Hi Aaron,
I'll need you to add
d($process);d($process_name);
To Line 311 of lib/api.inc.php
And then run: /usr/local/bin/php -q poller2.php master 2
And show me the output at the top.
Javier
Aaron Daubman wrote:
Hi Javier,
Here's what the output from `ps -eo args` looks like under FreeBSD 5.3 about 10 minutes after re-enabling the jffnms cronjobs:
---snip--- [EMAIL PROTECTED](55)> ps -eo args -U jffnms ps: Process environment requires procfs(5) COMMAND /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/rrdtool - /usr/local/bin/rrdtool - /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/rrdtool - /usr/local/bin/rrdtool - /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/rrdtool - /usr/local/bin/rrdtool - /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/php -q poller2.php - 5 /usr/local/bin/php -q poller2.php - 5 sh -c /usr/local/bin/rrdtool - /usr/local/bin/rrdtool - sh -c /usr/local/bin/rrdtool - /usr/local/bin/rrdtool - /bin/sh -c cd $JFFNMS && $PHP -q poller2.php master $THREADS >/dev/null 2>&1 /bin/sh -c cd $JFFNMS && $PHP -q consolidate.php >/dev/null 2>&1 /usr/local/bin/php -q poller2.php master 2 /usr/local/bin/php -q consolidate.php ---snip---
it appears that is_process_running is actually looking for the process name as the last part of each line - which does not seem to be the case in my output above. Any suggestions?
killall is what I've been using, but the problem crops back up every ten minutes or so.
Thanks again, ~Aaron
On Mon, 21 Mar 2005 15:24:51 -0300, Javier Szyszlican <[EMAIL PROTECTED]> wrote:
Hi Aaron,
It seems our system against running a program twice is not working on your server.
Check the lib/api.inc.php is_process_running function.
We're running "ps -eo args" to check fi ap rocess is already running.
Also do a killall php to clean everything.
Javier
Aaron Daubman wrote:
Hello again,
My system is still getting hammered by jffnms php processes even after switching to the new poller. Is there anything I'm missing? Anything else I can check?
I'm attaching three files: ---------------------------------- crontab.txt - the JFFNMS user crontab on my system topout.txt - the output from `top -d1 400` psout.txt - the output from `ps -auxw -U jffnms`
Thanks again, ~Aaron
On Thu, 17 Mar 2005 17:19:58 -0300, Javier Szyszlican <[EMAIL PROTECTED]> wrote:
Hi Aaron,
Aaron Daubman wrote:
Hi Javier, et al;
Pardon if I don't provide enough useful information up front - let me know what would be useful to know and I'll include it.
I'm in the process of setting up a brand new JFFNMS 0.8.1 install on a Sunfire V20z dual opteron with 2G ram running FreeBSD 5.3-Stable SMP for amd64 with apache2, mysql 4.1.10 and php5.
Wow, nice boxes... :)
So far I'm monitoring 6 switches and a VPN concentrator with a total of 1191 interfaces.
Everything had been running fine for 1.5 weeks with just 3 switches (and maybe 600 interfaces). Early this week I added the rest of the hosts and after a while, at random, I started getting errors like this:
Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 128 bytes) in /opt/jffnms/lib/api.events.inc.php on line 288
Going on a previous post, I increased the memory limit in php.ini from 8M to 32M and that seemed to fix the problem.
Thats ok.
Today, a day after upping the php memory limit, I tried to log in to the box and could not. The console cli reacted very sluggishly. About 15 minutes later it seems the box keeled over and reloaded. Almost all available memory was used (it appeared by php) and the processor load would spike. I'm attaching a text file of the output of `ps -auxw` from 15 minutes before the box reloaded.
After the reload, the box seems to be functioning fine - I'm just waiting now for it to happen again.
What should I be looking at to diagnose root-cause? I wouldn't think 1000 interfaces would be too much for a box of this caliber to handle...
It should handle it fine, but you are using the old poller, and your pollers are stacking up, it seems some of your hosts take more than 5 minutes to be polled, and a new poller for that host starts.
This is a common problem on big installations, thats why we did the new poller (poller2), using it, you will not see any more stacking, and the load will reamain stable.
Check the included crontab in docs/examples/unix for usage.
Thanks! ~Aaron
Hope that helps.
Javier
-- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Javier Szyszlican, Project Leader, JFFNMS [EMAIL PROTECTED]
I hope JFFNMS or I were helpful to you, if you can, please donate at http://jffnms.org/donate
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ jffnms-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/jffnms-users
-- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Javier Szyszlican, Project Leader, JFFNMS [EMAIL PROTECTED]
I hope JFFNMS or I were helpful to you, if you can, please donate at http://jffnms.org/donate
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ jffnms-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/jffnms-users
