Re: [Nagios-users] benchmark question
Hi Patrick, hi all, Dario B. Bestetti (OpServices) a écrit : - Patrick Morris patrick.mor...@hp.com escreveu: shadih rahman wrote: List, Is there any information as to what is the maximum number of services one can run on a single box with single instance of Nagios? I did not find any concrete data on this. I am running a single instance of nagios on a quad core 2.5 GHZ machine with with 4 Gigs of RAM. I have total of 7359 service check running on this box. I have also ndoutils running on the same box as backend. My total service checks is going to five fold very soon with a lot of nrpe checks. Now, my question is should I run multiple instance of nagios on the same box or a single instance will be able to handle about 3 service checks? In my experience, you'll run into issues right around 8,000 checks or so, depending no output verbosity, on a fairly stock setup. This is due to the size of the pipe used to temporarily store check results, at least on Linux (and you may be seeing it already with your number of checks). Around that point, even with quite a bit of tuning, the check results will fill the pipe in less than a second, which is the minimum amount of time I've been able to configure Nagios to flush it. When that happens, latencies go through the roof. Distributing the checks doesn't solve the problem if you're still sending the results to a centralized Nagios instance, since one machine still needs to process all of them. I'm working through this situation now, and it's looking like it may take a custom kernel with a larger pipe size to handle it. Shadih, we have a customer running over 20.000 service checks in a single box. The box is 2x Intel Dual-core Xeon 3Ghz with 8Gb RAM. The latency is around 0.7s. The checks are performed between 3min and 5min. They have around 40 simultaneous users. _ Dario B. BestettiOpServices I had to install a bunch of nagios for 40 polls. We ran some tests on bi-Xeon Dual-Core with 4Go. It seems the limit of Nagios 3 is around 25000 services by server, but we didn't use the interface and the recording to database is with Perl scripts of our own. Taking 2 services as a limit seem to be the right thing to do, perhaps less as 15000 with NDO. Regards, -- David Dumortier LINAGORA Service Management Monitoring -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null
Re: [Nagios-users] benchmark question
- David Dumortier david.dumort...@linagora.com escreveu: Hi Patrick, hi all, Dario B. Bestetti (OpServices) a écrit : - Patrick Morris patrick.mor...@hp.com escreveu: shadih rahman wrote: List, Is there any information as to what is the maximum number of services one can run on a single box with single instance of Nagios? I did not find any concrete data on this. I am running a single instance of nagios on a quad core 2.5 GHZ machine with with 4 Gigs of RAM. I have total of 7359 service check running on this box. I have also ndoutils running on the same box as backend. My total service checks is going to five fold very soon with a lot of nrpe checks. Now, my question is should I run multiple instance of nagios on the same box or a single instance will be able to handle about 3 service checks? In my experience, you'll run into issues right around 8,000 checks or so, depending no output verbosity, on a fairly stock setup. This is due to the size of the pipe used to temporarily store check results, at least on Linux (and you may be seeing it already with your number of checks). Around that point, even with quite a bit of tuning, the check results will fill the pipe in less than a second, which is the minimum amount of time I've been able to configure Nagios to flush it. When that happens, latencies go through the roof. Distributing the checks doesn't solve the problem if you're still sending the results to a centralized Nagios instance, since one machine still needs to process all of them. I'm working through this situation now, and it's looking like it may take a custom kernel with a larger pipe size to handle it. Shadih, we have a customer running over 20.000 service checks in a single box. The box is 2x Intel Dual-core Xeon 3Ghz with 8Gb RAM. The latency is around 0.7s. The checks are performed between 3min and 5min. They have around 40 simultaneous users. _ Dario B. BestettiOpServices I had to install a bunch of nagios for 40 polls. We ran some tests on bi-Xeon Dual-Core with 4Go. It seems the limit of Nagios 3 is around 25000 services by server, but we didn't use the interface and the recording to database is with Perl scripts of our own. Taking 2 services as a limit seem to be the right thing to do, perhaps less as 15000 with NDO. Regards, -- David Dumortier LINAGORA Service Management Monitoring -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null I forgot to tell that we did run some stress tests in this customer with up to 30.000 services and Nagios supported it very well. We also use our own broker to store data in a MySQL database. Regards, Dario -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null
[Nagios-users] benchmark question
List, Is there any information as to what is the maximum number of services one can run on a single box with single instance of Nagios? I did not find any concrete data on this. I am running a single instance of nagios on a quad core 2.5 GHZ machine with with 4 Gigs of RAM. I have total of 7359 service check running on this box. I have also ndoutils running on the same box as backend. My total service checks is going to five fold very soon with a lot of nrpe checks. Now, my question is should I run multiple instance of nagios on the same box or a single instance will be able to handle about 3 service checks? I already tested a dev environment with multiple instance of nagios with some hacking to init.d script, it started and monitored fine. What would be the preferred method of running nagios, single instance on a box or multiple instance on a box when we are dealing with high number of service checks? Please comment on this. Thanks -- Cordially, Shadhin Rahman -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null
Re: [Nagios-users] benchmark question
Do you have another box? I don't know of much of, if any, of an advantage to running multiple instances. The SMP will usually not help there because the other cores will be occupied executing plug-ins. If you had another box you could go with a distributed environment instead. Perhaps using SVN to keep record of config changes. From: shadih rahman [mailto:shadhi...@gmail.com] Sent: Monday, December 14, 2009 9:54 AM To: nagios-users@lists.sourceforge.net Subject: [Nagios-users] benchmark question List, Is there any information as to what is the maximum number of services one can run on a single box with single instance of Nagios? I did not find any concrete data on this. I am running a single instance of nagios on a quad core 2.5 GHZ machine with with 4 Gigs of RAM. I have total of 7359 service check running on this box. I have also ndoutils running on the same box as backend. My total service checks is going to five fold very soon with a lot of nrpe checks. Now, my question is should I run multiple instance of nagios on the same box or a single instance will be able to handle about 3 service checks? I already tested a dev environment with multiple instance of nagios with some hacking to init.d script, it started and monitored fine. What would be the preferred method of running nagios, single instance on a box or multiple instance on a box when we are dealing with high number of service checks? Please comment on this. Thanks -- Cordially, Shadhin Rahman This email and any attachments (Message) may contain legally privileged and/or confidential information. If you are not the addressee, or if this Message has been addressed to you in error, you are not authorized to read, copy, or distribute it, and we ask that you please delete it (including all copies) and notify the sender by return email. Delivery of this Message to any person other than the intended recipient(s) shall not be deemed a waiver of confidentiality and/or a privilege. -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null
Re: [Nagios-users] benchmark question
shadih rahman wrote: List, Is there any information as to what is the maximum number of services one can run on a single box with single instance of Nagios? I did not find any concrete data on this. I am running a single instance of nagios on a quad core 2.5 GHZ machine with with 4 Gigs of RAM. I have total of 7359 service check running on this box. I have also ndoutils running on the same box as backend. My total service checks is going to five fold very soon with a lot of nrpe checks. Now, my question is should I run multiple instance of nagios on the same box or a single instance will be able to handle about 3 service checks? In my experience, you'll run into issues right around 8,000 checks or so, depending no output verbosity, on a fairly stock setup. This is due to the size of the pipe used to temporarily store check results, at least on Linux (and you may be seeing it already with your number of checks). Around that point, even with quite a bit of tuning, the check results will fill the pipe in less than a second, which is the minimum amount of time I've been able to configure Nagios to flush it. When that happens, latencies go through the roof. Distributing the checks doesn't solve the problem if you're still sending the results to a centralized Nagios instance, since one machine still needs to process all of them. I'm working through this situation now, and it's looking like it may take a custom kernel with a larger pipe size to handle it. -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null
Re: [Nagios-users] benchmark question
- Patrick Morris patrick.mor...@hp.com escreveu: shadih rahman wrote: List, Is there any information as to what is the maximum number of services one can run on a single box with single instance of Nagios? I did not find any concrete data on this. I am running a single instance of nagios on a quad core 2.5 GHZ machine with with 4 Gigs of RAM. I have total of 7359 service check running on this box. I have also ndoutils running on the same box as backend. My total service checks is going to five fold very soon with a lot of nrpe checks. Now, my question is should I run multiple instance of nagios on the same box or a single instance will be able to handle about 3 service checks? In my experience, you'll run into issues right around 8,000 checks or so, depending no output verbosity, on a fairly stock setup. This is due to the size of the pipe used to temporarily store check results, at least on Linux (and you may be seeing it already with your number of checks). Around that point, even with quite a bit of tuning, the check results will fill the pipe in less than a second, which is the minimum amount of time I've been able to configure Nagios to flush it. When that happens, latencies go through the roof. Distributing the checks doesn't solve the problem if you're still sending the results to a centralized Nagios instance, since one machine still needs to process all of them. I'm working through this situation now, and it's looking like it may take a custom kernel with a larger pipe size to handle it. -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null Shadih, we have a customer running over 20.000 service checks in a single box. The box is 2x Intel Dual-core Xeon 3Ghz with 8Gb RAM. The latency is around 0.7s. The checks are performed between 3min and 5min. They have around 40 simultaneous users. _ Dario B. BestettiOpServices R. Luciana de Abreu, 471 - Sala 403 Porto Alegre, RS - CEP 90570-060 Fone 55(51)32753588 Mobile 55(51)81518218 Fax 55(51)32753588 Emaildario.beste...@opservices.com.br In God we trust, the rest we monitor ... _ -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev ___ Nagios-users mailing list Nagios-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null
