[Check_mk (english)] host inventory oscillating

[Austin, Gordon]

I have many hosts configured in Check MK including using mk_inventory to successfully gather inventory data. However for some but not all hosts I've added in the past few months (more or less since I upgraded Check MK from 1.2.8 to 1.4.0p34) I see an odd oscillation:
At one point in time a full inventory is gathered and shows up via our Check MK website and on the monitor server under ...var/check_mk/inventory/<host> with lots of data (say 1522 items). Next day or so at a certain time of day(always the same), this has dropped to 91 items or less and Inventory History shows under Removed something like 1522.
Manually testing the agent and mk_inventory call, I always get the full data.
For the problem hosts, there is nothing in ...var/check_mk/persisted/<host>.
For the good hosts there is normal full data in persisted and inventory directories.

I don't know if this is a bug, or I have a bad rule somewhere. Once a day rings a bell but so far I can't find a rule for that. What might it be under?

Any suggestions?

Gord Austin
SENIOR TECHNICAL SPECIALIST
UNIX & LINUX SYSTEMS
BUSINESS TECHNOLOGY

[cid:***@01CF6A25.263C1640]

T 204.926.4773 C 431.336.2072
6th Floor 300 CARLTON STREET
WINNIPEG, MB R3B 2K6
***@postmedia.com<mailto:***@postmedia.com>

The information contained in this email is strictly confidential, and is only intended for the party(ies) to whom it is addressed. Any other use, dissemination, distribution, disclosure or copying is prohibited. If you have received this email in error please so advise by reply email. Thank you.

[Andreas Döhler]

Hi Gordon,

You mean the "Check_MK HW/SW Inventory" check right? Here are some points
you can check.
Check interval for this service should be a high value like 4 hours as in
most of my setups.
The check interval of your hosts is important. If your persistent data gets
to old and no new data is coming then your check is thinking that nothing
is there anymore.

The mk_inventory script sets the persistent time to defined interval plus 5
minutes. If you have hosts with an check interval of 10 minutes you will
get from time to time an empty response and have also no persistent data on
your monitoring server. This will result in the problem you mentioned.

Best regards
Andreas


Am Fr., 9. Nov. 2018 um 18:32 Uhr schrieb Austin, Gordon <

Post by Austin, Gordon
I have many hosts configured in Check MK including using mk_inventory to
successfully gather inventory data. However for some but not all hosts
I’ve added in the past few months (more or less since I upgraded Check MK
At one point in time a full inventory is gathered and shows up via our
Check MK website and on the monitor server under

var/check_mk/inventory/<host> with lots of data (say 1522 items). Next
day or so at a certain time of day(always the same), this has dropped to 91
items or less and Inventory History shows under Removed something like 1522.
Manually testing the agent and mk_inventory call, I always get the full data.
For the problem hosts, there is nothing in 
var/check_mk/persisted/<host>.
For the good hosts there is normal full data in persisted and inventory directories.
I don’t know if this is a bug, or I have a bad rule somewhere. Once a day
rings a bell but so far I can’t find a rule for that. What might it be
under?
Any suggestions?
*Gord Austin*
SENIOR TECHNICAL SPECIALIST
UNIX & LINUX SYSTEMS
BUSINESS TECHNOLOGY
T 204.926.4773 C 431.336.2072
6th Floor 300 CARLTON STREET
WINNIPEG, MB R3B 2K6
The information contained in this email is strictly confidential, and is
only intended for the party(ies) to whom it is addressed. Any other use,
dissemination, distribution, disclosure or copying is prohibited. If you
have received this email in error please so advise by reply email. Thank
you.
_______________________________________________
checkmk-en mailing list
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

[Austin, Gordon]

Yes that is the check. It has taken me some time to find it, but I have a rule “Normal check interval for service checks” (for Active checks), set to an interval of 1 day for Services: Check_MK HW/SW Inventory$ for all hosts.
My Host check interval is defaulting to 1 minute.
The mk_inventory.linux script is setting the interval to 14400 (seconds, equals 4 hours) and adding 600 (10 minutes) plus NOW to give the persist value.

So if I understand correctly, I need to either shorten the rule “Normal check interval for service checks” to less than 4 hours, or define the INVENTORY_INTERVAL the mk_inventory script looks for to be greater than 1 day, in seconds.

This makes sense, but I don’t understand why this is a problem on six of my servers but not several hundred others – all running the same version of mk_inventory.

Gord


From: Andreas Döhler [mailto:***@gmail.com]
Sent: Sunday, November 11, 2018 3:00 AM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

You mean the "Check_MK HW/SW Inventory" check right? Here are some points you can check.
Check interval for this service should be a high value like 4 hours as in most of my setups.
The check interval of your hosts is important. If your persistent data gets to old and no new data is coming then your check is thinking that nothing is there anymore.

The mk_inventory script sets the persistent time to defined interval plus 5 minutes. If you have hosts with an check interval of 10 minutes you will get from time to time an empty response and have also no persistent data on your monitoring server. This will result in the problem you mentioned.

Best regards
Andreas


Am Fr., 9. Nov. 2018 um 18:32 Uhr schrieb Austin, Gordon <***@postmedia.com<mailto:***@postmedia.com>>:
I have many hosts configured in Check MK including using mk_inventory to successfully gather inventory data. However for some but not all hosts I’ve added in the past few months (more or less since I upgraded Check MK from 1.2.8 to 1.4.0p34) I see an odd oscillation:
At one point in time a full inventory is gathered and shows up via our Check MK website and on the monitor server under 
var/check_mk/inventory/<host> with lots of data (say 1522 items). Next day or so at a certain time of day(always the same), this has dropped to 91 items or less and Inventory History shows under Removed something like 1522.
Manually testing the agent and mk_inventory call, I always get the full data.
For the problem hosts, there is nothing in 
var/check_mk/persisted/<host>.
For the good hosts there is normal full data in persisted and inventory directories.

I don’t know if this is a bug, or I have a bad rule somewhere. Once a day rings a bell but so far I can’t find a rule for that. What might it be under?

Any suggestions?

Gord Austin
SENIOR TECHNICAL SPECIALIST
UNIX & LINUX SYSTEMS
BUSINESS TECHNOLOGY

[cid:***@01CF6A25.263C1640]

T 204.926.4773 C 431.336.2072
6th Floor 300 CARLTON STREET
WINNIPEG, MB R3B 2K6
***@postmedia.com<mailto:***@postmedia.com>

The information contained in this email is strictly confidential, and is only intended for the party(ies) to whom it is addressed. Any other use, dissemination, distribution, disclosure or copying is prohibited. If you have received this email in error please so advise by reply email. Thank you.


_______________________________________________
checkmk-en mailing list
checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

[Andreas Döhler]

Hi Gordon,

You dont need to change the setting they are ok.
It would be a problem if your check interval of your “check_mk“ service
would be greater then 10 minutes.

I try to describe why :)

The inventory data generated is every 14400 seconds and is persistent for
15000 seconds.
If your check interval of your “check_mk“ service (not the Check_MK HW/SW
Inventory service)
Is now greater then 10 minutes it can happen that you reach the point last
query to system was at 14390 seconds of interval time (all fine here) next
query is then 15min later at 15300 seconds of interval. Here will be again
all data transfered. But between 15000 and 15300 if there happens a active
check of Check_MK HW/SW Inventory this service will not find any persistent
data to process and will give you an empty inventory.

Any other problems need more data about your system setup. This is then
also not so easy to debug.

Best regards
Andreas

Post by Austin, Gordon
Yes that is the check. It has taken me some time to find it, but I have a
rule “Normal check interval for service checks” (for Active checks), set to
an interval of 1 day for Services: Check_MK HW/SW Inventory$ for all hosts.
My Host check interval is defaulting to 1 minute.
The mk_inventory.linux script is setting the interval to 14400 (seconds,
equals 4 hours) and adding 600 (10 minutes) plus NOW to give the persist
value.
So if I understand correctly, I need to either shorten the rule “Normal
check interval for service checks” to less than 4 hours, or define the
INVENTORY_INTERVAL the mk_inventory script looks for to be greater than 1
day, in seconds.
This makes sense, but I don’t understand why this is a problem on six of
my servers but not several hundred others – all running the same version of
mk_inventory.
Gord
*Sent:* Sunday, November 11, 2018 3:00 AM
*To:* Austin, Gordon
*Subject:* Re: [Check_mk (english)] host inventory oscillating
Hi Gordon,
You mean the "Check_MK HW/SW Inventory" check right? Here are some points you can check.
Check interval for this service should be a high value like 4 hours as in
most of my setups.
The check interval of your hosts is important. If your persistent data
gets to old and no new data is coming then your check is thinking that
nothing is there anymore.
The mk_inventory script sets the persistent time to defined interval plus
5 minutes. If you have hosts with an check interval of 10 minutes you will
get from time to time an empty response and have also no persistent data on
your monitoring server. This will result in the problem you mentioned.
Best regards
Andreas
Am Fr., 9. Nov. 2018 um 18:32 Uhr schrieb Austin, Gordon <
I have many hosts configured in Check MK including using mk_inventory to
successfully gather inventory data. However for some but not all hosts
I’ve added in the past few months (more or less since I upgraded Check MK
At one point in time a full inventory is gathered and shows up via our
Check MK website and on the monitor server under

var/check_mk/inventory/<host> with lots of data (say 1522 items). Next
day or so at a certain time of day(always the same), this has dropped to 91
items or less and Inventory History shows under Removed something like 1522.
Manually testing the agent and mk_inventory call, I always get the full data.
For the problem hosts, there is nothing in 
var/check_mk/persisted/<host>.
For the good hosts there is normal full data in persisted and inventory directories.
I don’t know if this is a bug, or I have a bad rule somewhere. Once a day
rings a bell but so far I can’t find a rule for that. What might it be
under?
Any suggestions?
*Gord Austin*
SENIOR TECHNICAL SPECIALIST
<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
UNIX & LINUX SYSTEMS
BUSINESS TECHNOLOGY
T 204.926.4773 C 431.336.2072
6th Floor 300 CARLTON STREET
<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
WINNIPEG, MB R3B 2K6
<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
The information contained in this email is strictly confidential, and is
only intended for the party(ies) to whom it is addressed. Any other use,
dissemination, distribution, disclosure or copying is prohibited. If you
have received this email in error please so advise by reply email. Thank
you.
_______________________________________________
checkmk-en mailing list
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

[Austin, Gordon]

Hi Andreas, I don’t have anything specifically set for the “check_mk” service so it should be defaulting like almost all my services to 1 minute. In fact I have observed this, e.g. after deleting the inventory flag file.

Gord

From: Andreas Döhler [mailto:***@gmail.com]
Sent: Tuesday, November 13, 2018 3:03 PM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

You dont need to change the setting they are ok.
It would be a problem if your check interval of your “check_mk“ service would be greater then 10 minutes.

I try to describe why :)

The inventory data generated is every 14400 seconds and is persistent for 15000 seconds.
If your check interval of your “check_mk“ service (not the Check_MK HW/SW Inventory service)
Is now greater then 10 minutes it can happen that you reach the point last query to system was at 14390 seconds of interval time (all fine here) next query is then 15min later at 15300 seconds of interval. Here will be again all data transfered. But between 15000 and 15300 if there happens a active check of Check_MK HW/SW Inventory this service will not find any persistent data to process and will give you an empty inventory.

Any other problems need more data about your system setup. This is then also not so easy to debug.

Best regards
Andreas
Austin, Gordon <***@postmedia.com<mailto:***@postmedia.com>> schrieb am Di., 13. Nov. 2018, 20:56:
Yes that is the check. It has taken me some time to find it, but I have a rule “Normal check interval for service checks” (for Active checks), set to an interval of 1 day for Services: Check_MK HW/SW Inventory$ for all hosts.
My Host check interval is defaulting to 1 minute.
The mk_inventory.linux script is setting the interval to 14400 (seconds, equals 4 hours) and adding 600 (10 minutes) plus NOW to give the persist value.

So if I understand correctly, I need to either shorten the rule “Normal check interval for service checks” to less than 4 hours, or define the INVENTORY_INTERVAL the mk_inventory script looks for to be greater than 1 day, in seconds.

This makes sense, but I don’t understand why this is a problem on six of my servers but not several hundred others – all running the same version of mk_inventory.

Gord


From: Andreas Döhler [mailto:***@gmail.com<mailto:***@gmail.com>]
Sent: Sunday, November 11, 2018 3:00 AM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

You mean the "Check_MK HW/SW Inventory" check right? Here are some points you can check.
Check interval for this service should be a high value like 4 hours as in most of my setups.
The check interval of your hosts is important. If your persistent data gets to old and no new data is coming then your check is thinking that nothing is there anymore.

The mk_inventory script sets the persistent time to defined interval plus 5 minutes. If you have hosts with an check interval of 10 minutes you will get from time to time an empty response and have also no persistent data on your monitoring server. This will result in the problem you mentioned.

Best regards
Andreas


Am Fr., 9. Nov. 2018 um 18:32 Uhr schrieb Austin, Gordon <***@postmedia.com<mailto:***@postmedia.com>>:
I have many hosts configured in Check MK including using mk_inventory to successfully gather inventory data. However for some but not all hosts I’ve added in the past few months (more or less since I upgraded Check MK from 1.2.8 to 1.4.0p34) I see an odd oscillation:
At one point in time a full inventory is gathered and shows up via our Check MK website and on the monitor server under 
var/check_mk/inventory/<host> with lots of data (say 1522 items). Next day or so at a certain time of day(always the same), this has dropped to 91 items or less and Inventory History shows under Removed something like 1522.
Manually testing the agent and mk_inventory call, I always get the full data.
For the problem hosts, there is nothing in 
var/check_mk/persisted/<host>.
For the good hosts there is normal full data in persisted and inventory directories.

I don’t know if this is a bug, or I have a bad rule somewhere. Once a day rings a bell but so far I can’t find a rule for that. What might it be under?

Any suggestions?

Gord Austin
SENIOR TECHNICAL SPECIALIST
UNIX & LINUX SYSTEMS
BUSINESS TECHNOLOGY

Error! Filename not specified.

T 204.926.4773 C 431.336.2072
6th Floor 300 CARLTON STREET<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
WINNIPEG, MB R3B 2K6<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
***@postmedia.com<mailto:***@postmedia.com>

The information contained in this email is strictly confidential, and is only intended for the party(ies) to whom it is addressed. Any other use, dissemination, distribution, disclosure or copying is prohibited. If you have received this email in error please so advise by reply email. Thank you.


_______________________________________________
checkmk-en mailing list
checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

[Andreas Döhler]

Hi Gordon, this is strange it looks more like that your "Check_MK HW/SW
Inventory" check runs every minute and see's there a deleted file and in
the result empty data from your agent.
If you look at the service view of the "Check_MK HW/SW Inventory" what do
you have as "Service normal/retry check interval" there?
On my systems it is something like 240 m / 60 s.

Best regards
Andreas

Am Di., 13. Nov. 2018 um 22:20 Uhr schrieb Austin, Gordon <

Post by Austin, Gordon
Hi Andreas, I don’t have anything specifically set for the “check_mk”
service so it should be defaulting like almost all my services to 1
minute. In fact I have observed this, e.g. after deleting the inventory
flag file.
Gord
*Sent:* Tuesday, November 13, 2018 3:03 PM
*To:* Austin, Gordon
*Subject:* Re: [Check_mk (english)] host inventory oscillating
Hi Gordon,
You dont need to change the setting they are ok.
It would be a problem if your check interval of your “check_mk“ service
would be greater then 10 minutes.
I try to describe why :)
The inventory data generated is every 14400 seconds and is persistent for 15000 seconds.
If your check interval of your “check_mk“ service (not the Check_MK HW/SW
Inventory service)
Is now greater then 10 minutes it can happen that you reach the point last
query to system was at 14390 seconds of interval time (all fine here) next
query is then 15min later at 15300 seconds of interval. Here will be again
all data transfered. But between 15000 and 15300 if there happens a active
check of Check_MK HW/SW Inventory this service will not find any persistent
data to process and will give you an empty inventory.
Any other problems need more data about your system setup. This is then
also not so easy to debug.
Best regards
Andreas
Yes that is the check. It has taken me some time to find it, but I have a
rule “Normal check interval for service checks” (for Active checks), set to
an interval of 1 day for Services: Check_MK HW/SW Inventory$ for all hosts.
My Host check interval is defaulting to 1 minute.
The mk_inventory.linux script is setting the interval to 14400 (seconds,
equals 4 hours) and adding 600 (10 minutes) plus NOW to give the persist
value.
So if I understand correctly, I need to either shorten the rule “Normal
check interval for service checks” to less than 4 hours, or define the
INVENTORY_INTERVAL the mk_inventory script looks for to be greater than 1
day, in seconds.
This makes sense, but I don’t understand why this is a problem on six of
my servers but not several hundred others – all running the same version of
mk_inventory.
Gord
*Sent:* Sunday, November 11, 2018 3:00 AM
*To:* Austin, Gordon
*Subject:* Re: [Check_mk (english)] host inventory oscillating
Hi Gordon,
You mean the "Check_MK HW/SW Inventory" check right? Here are some points you can check.
Check interval for this service should be a high value like 4 hours as in
most of my setups.
The check interval of your hosts is important. If your persistent data
gets to old and no new data is coming then your check is thinking that
nothing is there anymore.
The mk_inventory script sets the persistent time to defined interval plus
5 minutes. If you have hosts with an check interval of 10 minutes you will
get from time to time an empty response and have also no persistent data on
your monitoring server. This will result in the problem you mentioned.
Best regards
Andreas
Am Fr., 9. Nov. 2018 um 18:32 Uhr schrieb Austin, Gordon <
I have many hosts configured in Check MK including using mk_inventory to
successfully gather inventory data. However for some but not all hosts
I’ve added in the past few months (more or less since I upgraded Check MK
At one point in time a full inventory is gathered and shows up via our
Check MK website and on the monitor server under

var/check_mk/inventory/<host> with lots of data (say 1522 items). Next
day or so at a certain time of day(always the same), this has dropped to 91
items or less and Inventory History shows under Removed something like 1522.
Manually testing the agent and mk_inventory call, I always get the full data.
For the problem hosts, there is nothing in 
var/check_mk/persisted/<host>.
For the good hosts there is normal full data in persisted and inventory directories.
I don’t know if this is a bug, or I have a bad rule somewhere. Once a day
rings a bell but so far I can’t find a rule for that. What might it be
under?
Any suggestions?
*Gord Austin*
SENIOR TECHNICAL SPECIALIST
UNIX & LINUX SYSTEMS
BUSINESS TECHNOLOGY
*Error! Filename not specified.*
T 204.926.4773 C 431.336.2072
6th Floor 300 CARLTON STREET
<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
WINNIPEG, MB R3B 2K6
<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
The information contained in this email is strictly confidential, and is
only intended for the party(ies) to whom it is addressed. Any other use,
dissemination, distribution, disclosure or copying is prohibited. If you
have received this email in error please so advise by reply email. Thank
you.
_______________________________________________
checkmk-en mailing list
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en
_______________________________________________
checkmk-en mailing list
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

[Austin, Gordon]

Hi Andreas, looking in WATO Host & Service Parameters / Monitoring Configuration, the Normal check interval for service checks is defaulting (to 1 minute) for most but I have a rule for the specific service “Check_MK HW/SW Inventory$” of a 1 day interval. The Retry interval for service checks I have set to 3 minutes.

Yesterday I tried setting the environment variable (referred to in mk_inventory, sourced from a config file) INVENTORY_INTERVAL one on problematic host to 28 hours (100800 seconds). This is used in building the Persist value in the output.
This appeared to work, the host has had significant inventory data saved in the persist directory on the monitoring server.
I’m giving it a few days to see what happens.

Gord

From: Andreas Döhler [mailto:***@gmail.com]
Sent: Thursday, November 15, 2018 3:01 PM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon, this is strange it looks more like that your "Check_MK HW/SW Inventory" check runs every minute and see's there a deleted file and in the result empty data from your agent.
If you look at the service view of the "Check_MK HW/SW Inventory" what do you have as "Service normal/retry check interval" there?
On my systems it is something like 240 m / 60 s.

Best regards
Andreas

Am Di., 13. Nov. 2018 um 22:20 Uhr schrieb Austin, Gordon <***@postmedia.com<mailto:***@postmedia.com>>:
Hi Andreas, I don’t have anything specifically set for the “check_mk” service so it should be defaulting like almost all my services to 1 minute. In fact I have observed this, e.g. after deleting the inventory flag file.

Gord

From: Andreas Döhler [mailto:***@gmail.com<mailto:***@gmail.com>]
Sent: Tuesday, November 13, 2018 3:03 PM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

You dont need to change the setting they are ok.
It would be a problem if your check interval of your “check_mk“ service would be greater then 10 minutes.

I try to describe why :)

The inventory data generated is every 14400 seconds and is persistent for 15000 seconds.
If your check interval of your “check_mk“ service (not the Check_MK HW/SW Inventory service)
Is now greater then 10 minutes it can happen that you reach the point last query to system was at 14390 seconds of interval time (all fine here) next query is then 15min later at 15300 seconds of interval. Here will be again all data transfered. But between 15000 and 15300 if there happens a active check of Check_MK HW/SW Inventory this service will not find any persistent data to process and will give you an empty inventory.

Any other problems need more data about your system setup. This is then also not so easy to debug.

Best regards
Andreas
Austin, Gordon <***@postmedia.com<mailto:***@postmedia.com>> schrieb am Di., 13. Nov. 2018, 20:56:
Yes that is the check. It has taken me some time to find it, but I have a rule “Normal check interval for service checks” (for Active checks), set to an interval of 1 day for Services: Check_MK HW/SW Inventory$ for all hosts.
My Host check interval is defaulting to 1 minute.
The mk_inventory.linux script is setting the interval to 14400 (seconds, equals 4 hours) and adding 600 (10 minutes) plus NOW to give the persist value.

So if I understand correctly, I need to either shorten the rule “Normal check interval for service checks” to less than 4 hours, or define the INVENTORY_INTERVAL the mk_inventory script looks for to be greater than 1 day, in seconds.

This makes sense, but I don’t understand why this is a problem on six of my servers but not several hundred others – all running the same version of mk_inventory.

Gord


From: Andreas Döhler [mailto:***@gmail.com<mailto:***@gmail.com>]
Sent: Sunday, November 11, 2018 3:00 AM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

You mean the "Check_MK HW/SW Inventory" check right? Here are some points you can check.
Check interval for this service should be a high value like 4 hours as in most of my setups.
The check interval of your hosts is important. If your persistent data gets to old and no new data is coming then your check is thinking that nothing is there anymore.

The mk_inventory script sets the persistent time to defined interval plus 5 minutes. If you have hosts with an check interval of 10 minutes you will get from time to time an empty response and have also no persistent data on your monitoring server. This will result in the problem you mentioned.

Best regards
Andreas


Am Fr., 9. Nov. 2018 um 18:32 Uhr schrieb Austin, Gordon <***@postmedia.com<mailto:***@postmedia.com>>:
I have many hosts configured in Check MK including using mk_inventory to successfully gather inventory data. However for some but not all hosts I’ve added in the past few months (more or less since I upgraded Check MK from 1.2.8 to 1.4.0p34) I see an odd oscillation:
At one point in time a full inventory is gathered and shows up via our Check MK website and on the monitor server under 
var/check_mk/inventory/<host> with lots of data (say 1522 items). Next day or so at a certain time of day(always the same), this has dropped to 91 items or less and Inventory History shows under Removed something like 1522.
Manually testing the agent and mk_inventory call, I always get the full data.
For the problem hosts, there is nothing in 
var/check_mk/persisted/<host>.
For the good hosts there is normal full data in persisted and inventory directories.

I don’t know if this is a bug, or I have a bad rule somewhere. Once a day rings a bell but so far I can’t find a rule for that. What might it be under?

Any suggestions?

Gord Austin
SENIOR TECHNICAL SPECIALIST
UNIX & LINUX SYSTEMS
BUSINESS TECHNOLOGY

Error! Filename not specified.

T 204.926.4773 C 431.336.2072
6th Floor 300 CARLTON STREET<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
WINNIPEG, MB R3B 2K6<https://maps.google.com/?q=300+CARLTON+STREET+%0D%0A+WINNIPEG,+MB+R3B+2K6&entry=gmail&source=g>
***@postmedia.com<mailto:***@postmedia.com>

The information contained in this email is strictly confidential, and is only intended for the party(ies) to whom it is addressed. Any other use, dissemination, distribution, disclosure or copying is prohibited. If you have received this email in error please so advise by reply email. Thank you.


_______________________________________________
checkmk-en mailing list
checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en
_______________________________________________
checkmk-en mailing list
checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>
Manage your subscription or unsubscribe
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

[Austin, Gordon]

My understanding of Check MK Inventory management is still very limited.
Under what circumstances does the Check MK monitor remove most/all of the inventory for a given host?

Gord Austin

[Arno Wijnhoven]

Hi Gordon,

What do you mean? Like ‘automatically’? I can’t think of a situation where you’d want that

It wouldn’t be right if Check_MK would update my inventory if it suddenly can’t get info about a previous existing hard drive, for instance.
This is a plain error (drive missing) and I want a notification, not an inventory update.

If some services don’t exist anymore, they will go into ‘Unknown’ state and you have to remove them –manually-.
When you go to the services of a host via WATO, you can click ‘Remove vanished services’.
You can set an automatic periodic inventory update though - but remember: with great power comes great responsibility ☺.

FYI, Check_MK user Marco Reale created a beginner guide (https://mathias-kettner.com/download/Marco_Reale_Check_MK_Beginner_guide.pdf).

Hope this helps,

Arno

From: checkmk-en [mailto:checkmk-en-***@lists.mathias-kettner.de] On Behalf Of Austin, Gordon
Sent: maandag 26 november 2018 17:57
To: <checkmk-***@lists.mathias-kettner.de> <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

My understanding of Check MK Inventory management is still very limited.
Under what circumstances does the Check MK monitor remove most/all of the inventory for a given host?

Gord Austin

[Andreas Döhler]

Hi Gordon,

to your question about removing inventory entries.
The inventory is removed if your inventory check gets no data. That means
nothing was found for this device and all entries are removed.
Easy to test if you do this with a PC - set inventory interval to 1 hour
and after you get the first inventory switch this PC off and wait 1 hour.
Then the next inventory check will fail and produce no data. Result will be
0 inventory entries.

@Arno - don't mix the HW/SW inventory with the normal service discovery.
All your thoughts are related to the normal service discovery :)

Best regards
Andreas

Am Di., 27. Nov. 2018 um 09:03 Uhr schrieb Arno Wijnhoven <

Post by Andreas Döhler
Hi Gordon,
What do you mean? Like ‘automatically’? I can’t think of a situation where
you’d want that

It wouldn’t be right if Check_MK would update my inventory if it suddenly
can’t get info about a previous existing hard drive, for instance.
This is a plain error (drive missing) and I want a notification, not an inventory update.
If some services don’t exist anymore, they will go into ‘Unknown’ state
and you have to remove them –manually-.
When you go to the services of a host via WATO, you can click ‘Remove
vanished services’.
with great power comes great responsibility J.
FYI, Check_MK user Marco Reale created a beginner guide (
https://mathias-kettner.com/download/Marco_Reale_Check_MK_Beginner_guide.pdf
).
Hope this helps,
Arno
Behalf Of *Austin, Gordon
*Sent:* maandag 26 november 2018 17:57
*Subject:* Re: [Check_mk (english)] host inventory oscillating
My understanding of Check MK Inventory management is still very limited.
Under what circumstances does the Check MK monitor remove most/all of the
inventory for a given host?
Gord Austin
_______________________________________________
checkmk-en mailing list
Manage your subscription or unsubscribe
https://lists.mathias-kettner.de/cgi-bin/mailman/listinfo/checkmk-en

[Arno Wijnhoven]

Thanks Andreas, you’re absolutely right.
Sorry ☺

Arno Wijnhoven

From: checkmk-en [mailto:checkmk-en-***@lists.mathias-kettner.de] On Behalf Of Andreas Döhler
Sent: dinsdag 27 november 2018 10:28
To: Austin, Gordon <***@postmedia.com>
Cc: <checkmk-***@lists.mathias-kettner.de> <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

to your question about removing inventory entries.
The inventory is removed if your inventory check gets no data. That means nothing was found for this device and all entries are removed.
Easy to test if you do this with a PC - set inventory interval to 1 hour and after you get the first inventory switch this PC off and wait 1 hour.
Then the next inventory check will fail and produce no data. Result will be 0 inventory entries.

@Arno - don't mix the HW/SW inventory with the normal service discovery.
All your thoughts are related to the normal service discovery :)

Best regards
Andreas

Am Di., 27. Nov. 2018 um 09:03 Uhr schrieb Arno Wijnhoven <***@vsnsystemen.nl<mailto:***@vsnsystemen.nl>>:
Hi Gordon,

What do you mean? Like ‘automatically’? I can’t think of a situation where you’d want that

It wouldn’t be right if Check_MK would update my inventory if it suddenly can’t get info about a previous existing hard drive, for instance.
This is a plain error (drive missing) and I want a notification, not an inventory update.

If some services don’t exist anymore, they will go into ‘Unknown’ state and you have to remove them –manually-.
When you go to the services of a host via WATO, you can click ‘Remove vanished services’.
You can set an automatic periodic inventory update though - but remember: with great power comes great responsibility ☺.

FYI, Check_MK user Marco Reale created a beginner guide (https://mathias-kettner.com/download/Marco_Reale_Check_MK_Beginner_guide.pdf).

Hope this helps,

Arno

From: checkmk-en [mailto:checkmk-en-***@lists.mathias-kettner.de<mailto:checkmk-en-***@lists.mathias-kettner.de>] On Behalf Of Austin, Gordon
Sent: maandag 26 november 2018 17:57
To: <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>> <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>>
Subject: Re: [Check_mk (english)] host inventory oscillating

My understanding of Check MK Inventory management is still very limited.
Under what circumstances does the Check MK monitor remove most/all of the inventory for a given host?

Gord Austin

_______________________________________________
checkmk-en mailing list
checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>
Manage your subscription or unsubscribe
https://lists.mathias-kettner.de/cgi-bin/mailman/listinfo/checkmk-en

[Austin, Gordon]

Hi Andreas, in this case one of my problematic servers is a test Linux host. (Almost all my hosts are Linux.)
After the inventory on the monitor has been mostly discarded, the status is always
OK

Check_MK HW/SW Inventory<https://wpgnagmon01.pmcorp.postmedia.com/Nagios_elwi/check_mk/view.py?host=wpgedworad1&service=Check_MK%20HW/SW%20Inventory&site=Nagios_elwi&view_name=service>

[Loading Image...]<javascript:void(0);>

OK - found 91 entries


On the Check_MK monitor I have for this host set the Normal check interval for service checks “Check_MK HW/SW Inventory$” to 12 minutes.
On the client side I have set INVENTORY_INTERVAL=600 and I modified /usr/bin/check_mk_agent to tee the plugins $skript to a new log file each run. Of course on most runs (once a minute) the mk_inventory output is empty (0 length file) due to the mk_inventory.last. file’s timestamp and the interval. Every 10 minutes I get a non-empty mk_inventory log file.
After six runs producing non-empty mk_inventory output I see that these all produced identical full output except for the persist timestamps.

In spite of this, the Inventory history of this host shows that
55m(inutes ago) 1523 new objects were identified; and
38m 1523 objects were removed
No later Inventory runs are identified in the Inventory history report.

Gord

From: Andreas Döhler [mailto:***@gmail.com]
Sent: Tuesday, November 27, 2018 3:28 AM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

to your question about removing inventory entries.
The inventory is removed if your inventory check gets no data. That means nothing was found for this device and all entries are removed.
Easy to test if you do this with a PC - set inventory interval to 1 hour and after you get the first inventory switch this PC off and wait 1 hour.
Then the next inventory check will fail and produce no data. Result will be 0 inventory entries.

@Arno - don't mix the HW/SW inventory with the normal service discovery.
All your thoughts are related to the normal service discovery :)

Best regards
Andreas

Am Di., 27. Nov. 2018 um 09:03 Uhr schrieb Arno Wijnhoven <***@vsnsystemen.nl<mailto:***@vsnsystemen.nl>>:
Hi Gordon,

What do you mean? Like ‘automatically’? I can’t think of a situation where you’d want that

It wouldn’t be right if Check_MK would update my inventory if it suddenly can’t get info about a previous existing hard drive, for instance.
This is a plain error (drive missing) and I want a notification, not an inventory update.

If some services don’t exist anymore, they will go into ‘Unknown’ state and you have to remove them –manually-.
When you go to the services of a host via WATO, you can click ‘Remove vanished services’.
You can set an automatic periodic inventory update though - but remember: with great power comes great responsibility ☺.

FYI, Check_MK user Marco Reale created a beginner guide (https://mathias-kettner.com/download/Marco_Reale_Check_MK_Beginner_guide.pdf).

Hope this helps,

Arno

From: checkmk-en [mailto:checkmk-en-***@lists.mathias-kettner.de<mailto:checkmk-en-***@lists.mathias-kettner.de>] On Behalf Of Austin, Gordon
Sent: maandag 26 november 2018 17:57
To: <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>> <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>>
Subject: Re: [Check_mk (english)] host inventory oscillating

My understanding of Check MK Inventory management is still very limited.
Under what circumstances does the Check MK monitor remove most/all of the inventory for a given host?

Gord Austin

_______________________________________________
checkmk-en mailing list
checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>
Manage your subscription or unsubscribe
https://lists.mathias-kettner.de/cgi-bin/mailman/listinfo/checkmk-en

[Austin, Gordon]

Hello again. Can someone tell me how the service check “Check_MK HW/SW Inventory” is performed?
I can see separately that the check_mk_agent is prodded into running once a minute (via port 6556) including launching mk_inventory, and that every $INTERVAL mk_inventory will actually produce output. This seems to be independent of the service check.

So what & how does service check “Check_MK HW/SW Inventory” operate?

Gord

From: checkmk-en [mailto:checkmk-en-***@lists.mathias-kettner.de] On Behalf Of Austin, Gordon
Sent: Tuesday, November 27, 2018 10:14 AM
To: Andreas Döhler
Cc: <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Andreas, in this case one of my problematic servers is a test Linux host. (Almost all my hosts are Linux.)
After the inventory on the monitor has been mostly discarded, the status is always
OK

Check_MK HW/SW Inventory<https://wpgnagmon01.pmcorp.postmedia.com/Nagios_elwi/check_mk/view.py?host=wpgedworad1&service=Check_MK%20HW/SW%20Inventory&site=Nagios_elwi&view_name=service>

[https://wpgnagmon01.pmcorp.postmedia.com/Nagios_elwi/check_mk/images/icon_menu.png]<javascript:void(0);>

OK - found 91 entries


On the Check_MK monitor I have for this host set the Normal check interval for service checks “Check_MK HW/SW Inventory$” to 12 minutes.
On the client side I have set INVENTORY_INTERVAL=600 and I modified /usr/bin/check_mk_agent to tee the plugins $skript to a new log file each run. Of course on most runs (once a minute) the mk_inventory output is empty (0 length file) due to the mk_inventory.last. file’s timestamp and the interval. Every 10 minutes I get a non-empty mk_inventory log file.
After six runs producing non-empty mk_inventory output I see that these all produced identical full output except for the persist timestamps.

In spite of this, the Inventory history of this host shows that
55m(inutes ago) 1523 new objects were identified; and
38m 1523 objects were removed
No later Inventory runs are identified in the Inventory history report.

Gord

From: Andreas Döhler [mailto:***@gmail.com]
Sent: Tuesday, November 27, 2018 3:28 AM
To: Austin, Gordon
Cc: <checkmk-***@lists.mathias-kettner.de>
Subject: Re: [Check_mk (english)] host inventory oscillating

Hi Gordon,

to your question about removing inventory entries.
The inventory is removed if your inventory check gets no data. That means nothing was found for this device and all entries are removed.
Easy to test if you do this with a PC - set inventory interval to 1 hour and after you get the first inventory switch this PC off and wait 1 hour.
Then the next inventory check will fail and produce no data. Result will be 0 inventory entries.

@Arno - don't mix the HW/SW inventory with the normal service discovery.
All your thoughts are related to the normal service discovery :)

Best regards
Andreas

Am Di., 27. Nov. 2018 um 09:03 Uhr schrieb Arno Wijnhoven <***@vsnsystemen.nl<mailto:***@vsnsystemen.nl>>:
Hi Gordon,

What do you mean? Like ‘automatically’? I can’t think of a situation where you’d want that

It wouldn’t be right if Check_MK would update my inventory if it suddenly can’t get info about a previous existing hard drive, for instance.
This is a plain error (drive missing) and I want a notification, not an inventory update.

If some services don’t exist anymore, they will go into ‘Unknown’ state and you have to remove them –manually-.
When you go to the services of a host via WATO, you can click ‘Remove vanished services’.
You can set an automatic periodic inventory update though - but remember: with great power comes great responsibility ☺.

FYI, Check_MK user Marco Reale created a beginner guide (https://mathias-kettner.com/download/Marco_Reale_Check_MK_Beginner_guide.pdf).

Hope this helps,

Arno

From: checkmk-en [mailto:checkmk-en-***@lists.mathias-kettner.de<mailto:checkmk-en-***@lists.mathias-kettner.de>] On Behalf Of Austin, Gordon
Sent: maandag 26 november 2018 17:57
To: <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>> <checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>>
Subject: Re: [Check_mk (english)] host inventory oscillating

My understanding of Check MK Inventory management is still very limited.
Under what circumstances does the Check MK monitor remove most/all of the inventory for a given host?

Gord Austin

_______________________________________________
checkmk-en mailing list
checkmk-***@lists.mathias-kettner.de<mailto:checkmk-***@lists.mathias-kettner.de>
Manage your subscription or unsubscribe
https://lists.mathias-kettner.de/cgi-bin/mailman/listinfo/checkmk-en

[Andreas Döhler]

Hi Gordon,

i try to describe how the "Check_MK HW/SW Inventory" works

If one check sends data with a persist timestamp in the header the the
server side remembers this data for the given time.
Now the plugin only needs to send new data with a new persist header before
the timestamp is reached on the server side.
If your server sees that the end of the timestamp from persist header is
reached he discards the data.

There is only one problem what i remember - do you have more then the
"HW/SW inventory" script what produce persistent output?
If yes all scripts must sent the data at the same time.

Best regards,
Andreas

Am Di., 27. Nov. 2018 um 17:56 Uhr schrieb Austin, Gordon <

Post by Austin, Gordon
Hello again. Can someone tell me how the service check “Check_MK HW/SW
Inventory” is performed?
I can see separately that the check_mk_agent is prodded into running once
a minute (via port 6556) including launching mk_inventory, and that every
$INTERVAL mk_inventory will actually produce output. This seems to be
independent of the service check.
So what & how does service check “Check_MK HW/SW Inventory” operate?
Gord
Behalf Of *Austin, Gordon
*Sent:* Tuesday, November 27, 2018 10:14 AM
*To:* Andreas Döhler
*Subject:* Re: [Check_mk (english)] host inventory oscillating
Hi Andreas, in this case one of my problematic servers is a test Linux
host. (Almost all my hosts are Linux.)
After the inventory on the monitor has been mostly discarded, the status is always
*OK*
Check_MK HW/SW Inventory
<https://wpgnagmon01.pmcorp.postmedia.com/Nagios_elwi/check_mk/view.py?host=wpgedworad1&service=Check_MK%20HW/SW%20Inventory&site=Nagios_elwi&view_name=service>
https://wpgnagmon01.pmcorp.postmedia.com/Nagios_elwi/check_mk/images/icon_menu.png]
OK - found 91 entries
On the Check_MK monitor I have for this host set the Normal check interval
for service checks “Check_MK HW/SW Inventory$” to 12 minutes.
On the client side I have set INVENTORY_INTERVAL=600 and I modified
/usr/bin/check_mk_agent to tee the plugins $skript to a new log file each
run. Of course on most runs (once a minute) the mk_inventory output is
empty (0 length file) due to the mk_inventory.last. file’s timestamp and
the interval. Every 10 minutes I get a non-empty mk_inventory log file.
After six runs producing non-empty mk_inventory output I see that these
all produced identical full output except for the persist timestamps.
In spite of this, the Inventory history of this host shows that
55m(inutes ago) 1523 new objects were identified; and
38m 1523 objects were removed
No later Inventory runs are identified in the Inventory history report.
Gord
*Sent:* Tuesday, November 27, 2018 3:28 AM
*To:* Austin, Gordon
*Subject:* Re: [Check_mk (english)] host inventory oscillating
Hi Gordon,
to your question about removing inventory entries.
The inventory is removed if your inventory check gets no data. That means
nothing was found for this device and all entries are removed.
Easy to test if you do this with a PC - set inventory interval to 1 hour
and after you get the first inventory switch this PC off and wait 1 hour.
Then the next inventory check will fail and produce no data. Result will
be 0 inventory entries.
@Arno - don't mix the HW/SW inventory with the normal service discovery.
All your thoughts are related to the normal service discovery :)
Best regards
Andreas
Am Di., 27. Nov. 2018 um 09:03 Uhr schrieb Arno Wijnhoven <
Hi Gordon,
What do you mean? Like ‘automatically’? I can’t think of a situation where
you’d want that

It wouldn’t be right if Check_MK would update my inventory if it suddenly
can’t get info about a previous existing hard drive, for instance.
This is a plain error (drive missing) and I want a notification, not an inventory update.
If some services don’t exist anymore, they will go into ‘Unknown’ state
and you have to remove them –manually-.
When you go to the services of a host via WATO, you can click ‘Remove
vanished services’.
with great power comes great responsibility J.
FYI, Check_MK user Marco Reale created a beginner guide (
https://mathias-kettner.com/download/Marco_Reale_Check_MK_Beginner_guide.pdf
).
Hope this helps,
Arno
Behalf Of *Austin, Gordon
*Sent:* maandag 26 november 2018 17:57
*Subject:* Re: [Check_mk (english)] host inventory oscillating
My understanding of Check MK Inventory management is still very limited.
Under what circumstances does the Check MK monitor remove most/all of the
inventory for a given host?
Gord Austin
_______________________________________________
checkmk-en mailing list
Manage your subscription or unsubscribe
https://lists.mathias-kettner.de/cgi-bin/mailman/listinfo/checkmk-en