I've had one instance running for about an hour now and my netflow udp queues are still maxed out. This instance has about 160 flow exporters, but rate of flows (flows/sec) is not massive or anything. I'd guess it's about the same as another instance I have for the HQ LAN, which it's queue is empty! The diff is the LAN instance is not tracking remote hosts....
So, could the additional overhead / load be related to name resolution or more specifically GeoIP? I don't think GeoIP was around in 3.2, so maybe that whole process is slowing things down enough to cause me problems, especially on initial startup when there are a bunch of new hosts to discover info about? I'm hoping once it catches up everything will be fine? ________________________________ From: Gary Gatten Sent: Tuesday, May 12, 2009 3:57 PM To: '[email protected]'; '[email protected]' Subject: RE: high CPU load / can't keep up with netflow since upgrade to 3.3.8 PS: On 3.3.9 from SourceForge now - got tired of the "[warn] kevent: Bad file descriptor" errors. G ________________________________ From: Gary Gatten Sent: Tuesday, May 12, 2009 3:56 PM To: '[email protected]'; '[email protected]' Subject: RE: high CPU load / can't keep up with netflow since upgrade to 3.3.8 OK, under production load I'm having similar issues even with "HAVE_SNMP" undef'd. I tweaked some timers in globals-defines.h that is causing higher than typical memory usage. I don't have much RAM (768MB) so I'm doing a lot of swapping. This is part of the problem for sure, so at this point I'm not 100% sure if the SNMP (UtilLoop) had anything to do with this and if so how much? I do recall when first trying to get on 3.3.x some time ago I didn't tweak the timers as much, but I also had even less RAM then! More testing, but prolly not today! ________________________________ From: Gary Gatten Sent: Monday, May 11, 2009 6:42 PM To: '[email protected]'; '[email protected]' Subject: RE: high CPU load / can't keep up with netflow since upgrade to 3.3.8 Thanks for all the prompt replies on this! :-) Looks like the additional load is related to SNMP. I commented out and undef'd stuff related to "HAVE_SNMP" and SO FAR my CPU load is back to "normal" and the udp queues are being serviced promptly. Not a whole lot of traffic right now though, so will let you know for sure EOD tomorrow - I know you're all anxiously waiting! G ________________________________ From: Gary Gatten Sent: Monday, May 11, 2009 3:27 PM To: '[email protected]'; '[email protected]' Subject: RE: high CPU load / can't keep up with netflow since upgrade to 3.3.8 What's diff between 3.2.x and 3.3.x netflow plugin? I deleted the prefsCache.db and started adding netflow devices 1 at a time. Each device as between 10 and 30 netflow exporters, and not high volume ones; all T1 sites with ~ 15 users each. Anyway, all is going well until I hit 3 or 4 netflow devices, then cpu maxes out and queues start filling up. Number of flows / sec / min is as stable as a production environment can be - not drastic changes that would account for this high of load. If there something different with the threads? It seems to be acting differently - just a hunch. ________________________________ From: Gary Gatten Sent: Monday, May 11, 2009 11:08 AM To: '[email protected]'; [email protected] Subject: high CPU load / can't keep up with netflow since upgrade to 3.3.8 Been running 3.2.1 for years and CPU has always been fine. Avg, load on FreeBSD 6.0 is around .5, CPU around 40%, and udp port queues (netstat -a) are most always zero, but sometimes a couple get a little backed up for a few mins here and there. Now running 3.3.8 compiled from ports. Load now 1.8, CPU 100%, and most all udp netflow queues are maxed and not emptying, especially within one instance that has many flow exporters - about 160'ish: $ netstat -a | grep flow Ntop Instance 1 udp4 41200 0 *.netflow-Regn29 *.* udp4 41496 0 *.netflow-Regn16 *.* udp4 41456 0 *.netflow-Regn15 *.* udp4 40904 0 *.netflow-Regn11 *.* udp4 41544 0 *.netflow-Regn10 *.* udp4 41456 0 *.netflow-Regn02 *.* udp4 41552 0 *.netflow-Regn88 *.* udp4 40656 0 *.netflow-Regn53 *.* Ntop Instance 2 udp4 0 0 *.netflow-Inet *.* Ntop Instance 3 udp4 0 0 *.netflow-LANCore *.* 3.2.1 had its issues, but as far as netflow and cpu load it was fine. I'd like to get on a more recent version, but keep hitting roadblocks. Tried this back with 3.3.1 (I think?) with similar results. Nothing in logs I find interesting. Only interesting thing on compile what something about "sched.h": checking sys/sched.h presence... yes configure: WARNING: sys/sched.h: present but cannot be compiled configure: WARNING: sys/sched.h: check for missing prerequisite headers? configure: WARNING: sys/sched.h: see the Autoconf documentation configure: WARNING: sys/sched.h: section "Present But Cannot Be Compiled" configure: WARNING: sys/sched.h: proceeding with the preprocessor's result configure: WARNING: sys/sched.h: in the future, the compiler will take precedence I'm willing to do anything reasonable to help get this resolved. Any help would be greatly appreciated!!! TIA! Gary <font size="1"> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'> </div> "This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." </font>
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
