Swap will kill you... Memory: 2048M real, 1354M free, 314M swap in use, 1357M swap free
If that's ntop, then every time it has to walk the tables (i.e. purge), it's going to have to page it all in. No clue why it would be paged out with that much free, but it could be peak usage issues. Still, it says SIZE=161M RES=158M, which would seem not to be paging. I suggest you turn on the IDLE_PURGE DEBUG stuff (/* #define IDLE_PURGE_DEBUG */ in globals-define.h). This will then generate messages showing what's up with the purge... Just be aware it will generate A HUGE # of log messages. The interesting ones will be these: ntop[3237]: IDLE_PURGE(0): Scanning for idle hosts complete, 27 [out of 58] selected [MSGID8466797] ntop[3237]: IDLE_PURGE(0) 'eth1': 27 hosts deleted, elapsed time is 1.635329 seconds (0.060568 per host) [MSGID8477291] (so if you run --trace-level 4 --log-extra 2, you can grep 'em out by MSGID#) That will show us what's up with the purge. If it's not purge, I don't know what to say. You can try viewing the showMutex.html page - it might show which mutexes are locked... The other message you're seeing, ld.so.1: /usr/local/bin/ntop: fatal: relocation error: file /usr/local/lib/ntop/plugins/rrdPlugin.so: symbol rrd_clear_error: referenced symbol not found Is just a pass through when ntop tries to load the plugin. The loader doesn't like it. That message usually means the loader was unable to resolve an external reference at load time. No real idea why not... Unless... look at setSpecifiedUser() in util.c - maybe that work-around isn't quite right for FreeBSD... -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Steven Rudolph Sent: Monday, November 24, 2003 2:42 PM To: [EMAIL PROTECTED] Subject: RE: [Ntop] High CPU use 2.2.96 Let me update the condition. After running this for 1 day the few seconds turned into more like 5 minutes+ for web page queries. Occasionally it is quick like it used to be. My top capture after running all weekend: last pid: 4077; load averages: 1.70, 1.74, 1.74 37 processes: 35 sleeping, 2 on cpu CPU states: 14.5% idle, 69.5% user, 15.9% kernel, 0.0% iowait, 0.0% swap Memory: 2048M real, 1354M free, 314M swap in use, 1357M swap free PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 9039 ntop 9 20 0 161M 158M cpu/1 92.0H 73.01% ntop 3628 root 1 58 0 2672K 1736K cpu/0 0:12 0.09% top 336 mysql 14 58 0 48M 39M sleep 19.0H 0.06% mysqld 392 root 12 58 0 2744K 2232K sleep 34:51 0.02% mibiisa 223 root 9 53 0 88M 52M sleep 11:41 0.00% nscd 309 root 1 58 0 1720K 832K sleep 5:48 0.00% prngd 208 root 1 48 0 1968K 1216K sleep 1:01 0.00% cron 1 root 1 58 0 832K 624K sleep 0:49 0.00% init 10197 root 1 10 0 2792K 1912K sleep 0:22 0.00% sshd 15166 root 1 58 0 4992K 4424K sleep 0:16 0.00% perl 270 root 1 59 0 4408K 1832K sleep 0:16 0.00% sendmail 2606 root 3 58 0 2952K 2200K sleep 0:02 0.00% syslogd 15160 root 1 0 0 5056K 4520K sleep 0:00 0.00% swatch 287 root 1 0 0 1080K 808K sleep 0:00 0.00% mysqld_safe 75 root 5 20 0 2592K 2184K sleep 0:00 0.00% picld I do have these errors in the log viewer of the web page: 20/Nov/2003 13:57:05 **WARNING** Unable to load plugin '/usr/local/lib/ntop/plugins/rrdPlugin.so' 20/Nov/2003 13:57:05 **WARNING** Message is 'ld.so.1: /usr/local/bin/ntop: fatal: relocation error: file /usr/local/lib/ntop/plugins/rrdPlugin.so: symbol rrd_clear_error: referenced symbol not found' Strange warnings since /usr/local/lib/ntop/plugins/rrdPlugin.so is there with permissions: -rwxr-xr-x 1 ntop other 222084 Nov 20 11:10 rrdPlugin.so The second warning I have no clue about. Steve -----Original Message----- From: Burton M. Strauss III [mailto:[EMAIL PROTECTED] Sent: Thursday, November 20, 2003 1:36 PM To: [EMAIL PROTECTED] Subject: RE: [Ntop] High CPU use 2.2.96 Actually, that looks pretty good. 3335.0 + 8410.8 = 11745.8 Mbps or (assume 500 byte packets) around 3k packets / second. Spikes are probably the purge. In the 2.2.9x versions, it locks a mutex, which prevents the web pages from accessing the data. 2.2 just bulled ahead. I suppose we could detect that the mutex is locked and intercept the request with a try again later message, but it should only be for a few seconds every few minutes, so is it worth it? -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Steven Rudolph Sent: Thursday, November 20, 2003 12:10 PM To: [EMAIL PROTECTED] Subject: [Ntop] High CPU use 2.2.96 Running 2.2.96 from compiled CVS source a few days ago. gcc 3.2 libpcap-2003.11.11 gd-2.0.11 gdbm-1.8.3 in the FAQ. gawk-3.1.3 zlib-1.1.4 libpng-1.2.5 automake-1.7 autoconf-2.54 I compiled with --disable-mutexextrainfo as suggested by earlier threads. CPU use seems to be between 30% and up to 89% for ntop. This is on a dual proc E250 with 2GB RAM, with pretty much just this program running. Command line: /usr/local/bin/ntop -P /usr/local/share/ntop -i qfe3 -m aaa.bbb.ccc.0/24,aaa.bbb.ccc.0/24,aaa.bbb.ccc.0/24,aaa.bbb.ccc.0/24 -o -w 6666 -u ntop -d It may be a little high at times. But it averages about 10 to 20 Mb/s Max In:61.3 Mb/s (61.3%) Average In:3335.0 kb/s (3.3%) Current In:1921.5 kb/s (1.9%) Max Out:68.4 Mb/s (68.4%) Average Out:8410.8 kb/s (8.4%) Current Out:14.2 Mb/s (14.2%) On a previous version this seemed to be no trouble at all. The CPU seems to spike from about 30% to about 70/80% for ntop periodically. If the web page seems to refresh during this time the web page loads very slow. Any ideas/pointers? Thanks for your assistance. Steve Rudolph, CCSA, CCSE Network Security Engineer Internet Operations Center _______________________________________________ Ntop mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop
