On that version of FreeBSD, what's Jun 18 19:23:30 firestorm kernel: pid 54319 (ntop), uid 0: exited on signal 5 (core dumped)
(man 7 signal usually gives the # translations) You may need a switch to tell gdb to follow the children. Either in the script, or your config file, you probably do have -d specified. -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Yanek Korff Sent: Tuesday, June 21, 2005 8:19 PM To: [email protected] Subject: [Ntop] FreeBSD - crash after a few mins FreeBSD 5.3-STABLE #6: Thu Mar 3 19:09:01 EST 2005 CPU: Intel Pentium III (665.04-MHz 686-class CPU) real memory = 535486464 (510 MB) xl0: <3Com 3c905-TX Fast Etherlink XL> port 0xecc0-0xecff irq 9 at device 7.0 on pci1 xl1: <3Com 3c905C-TX Fast Etherlink XL> port 0xec00-0xec7f mem 0xfdfffc00-0xfdfffc7f irq 5 at device 12.0 on pci1 nsphy0: <DP83840 10/100 media interface> on miibus0 Recently installed NTOP on this FreeBSD system crashes after running. Sometimes after half an hour... somtimes after 10 minutes... and when run through GDB... immediately. I have tried to follow the GDB ultraMini tutorial, and get this: (gdb) run Starting program: /usr/local/bin/ntop --set-pcap-nonblocking --skip-version-check -i xl0,xl1 -w 127.0.0.1:3000 -W 0 -a /var/log/ntop.access.log -u root -K Sat Jun 18 19:24:53 2005 Initializing gdbm databases WARNING: --set-pcap-nonblocking requested Sat Jun 18 19:24:53 2005 ntop v.3.1 MT (SSL) Sat Jun 18 19:24:53 2005 Configured on Feb 23 2005 8:38:57, built on Feb 23 2005 08:41:18. Sat Jun 18 19:24:53 2005 Copyright 1998-2004 by Luca Deri <[EMAIL PROTECTED]> Sat Jun 18 19:24:53 2005 Get the freshest ntop from http://www.ntop.org/ Sat Jun 18 19:24:53 2005 Initializing ntop Sat Jun 18 19:24:53 2005 NONBLOCK: pcap_setnonblock(), device 0, succeeded Sat Jun 18 19:24:53 2005 **WARNING** NONBLOCK: cpu usage may be high - see man page and FAQ Sat Jun 18 19:24:53 2005 Checking xl0 for additional devices Sat Jun 18 19:24:53 2005 Resetting traffic statistics for device xl0 Sat Jun 18 19:24:53 2005 DLT: Device 0 [xl0] is 1, mtu 1514, header 14 Sat Jun 18 19:24:53 2005 NONBLOCK: pcap_setnonblock(), device 1, succeeded Sat Jun 18 19:24:53 2005 **WARNING** NONBLOCK: cpu usage may be high - see man page and FAQ Sat Jun 18 19:24:53 2005 Checking xl1 for additional devices Sat Jun 18 19:24:53 2005 Resetting traffic statistics for device xl1 Sat Jun 18 19:24:53 2005 DLT: Device 1 [xl1] is 1, mtu 1514, header 14 Sat Jun 18 19:24:53 2005 Initializing gdbm databases Sat Jun 18 19:24:53 2005 VENDOR: Loading MAC address table. Sat Jun 18 19:24:53 2005 VENDOR: Checking for MAC address table file Sat Jun 18 19:24:53 2005 VENDOR: File '/usr/local/etc/ntop/specialMAC.txt.gz' does not need to be reloaded Sat Jun 18 19:24:53 2005 VENDOR: ntop continues ok Sat Jun 18 19:24:53 2005 VENDOR: Checking for MAC address table file Sat Jun 18 19:24:53 2005 VENDOR: File '/usr/local/etc/ntop/oui.txt.gz' does not need to be reloaded Sat Jun 18 19:24:53 2005 VENDOR: ntop continues ok Sat Jun 18 19:24:53 2005 Fingeprint: Loading signature file. Sat Jun 18 19:24:53 2005 Fingeprint: ...loaded 1697 records Sat Jun 18 19:24:53 2005 INIT: Parent process is exiting (this is normal) Sat Jun 18 19:24:53 2005 INIT: Bye bye: I'm becoming a daemon... Program exited normally. And in /var/log/messages I see: Jun 18 19:23:27 firestorm ntop[54318]: ntop v.3.1 MT (SSL) Jun 18 19:23:27 firestorm ntop[54318]: Configured on Feb 23 2005 8:38:57, built on Feb 23 2005 08:41:18. Jun 18 19:23:27 firestorm ntop[54318]: Copyright 1998-2004 by Luca Deri <[EMAIL PROTECTED]> Jun 18 19:23:27 firestorm ntop[54318]: Get the freshest ntop from http://www.ntop.org/ Jun 18 19:23:27 firestorm ntop[54318]: Initializing ntop Jun 18 19:23:27 firestorm ntop[54318]: NONBLOCK: pcap_setnonblock(), device 0, succeeded Jun 18 19:23:27 firestorm ntop[54318]: **WARNING** NONBLOCK: cpu usage may be high - see man page and FAQ Jun 18 19:23:27 firestorm ntop[54318]: Checking xl0 for additional devices Jun 18 19:23:27 firestorm ntop[54318]: Resetting traffic statistics for device xl0 Jun 18 19:23:27 firestorm ntop[54318]: DLT: Device 0 [xl0] is 1, mtu 1514, header 14 Jun 18 19:23:27 firestorm ntop[54318]: NONBLOCK: pcap_setnonblock(), device 1, succeeded Jun 18 19:23:27 firestorm ntop[54318]: **WARNING** NONBLOCK: cpu usage may be high - see man page and FAQ Jun 18 19:23:27 firestorm ntop[54318]: Checking xl1 for additional devices Jun 18 19:23:27 firestorm ntop[54318]: Resetting traffic statistics for device xl1 Jun 18 19:23:27 firestorm ntop[54318]: DLT: Device 1 [xl1] is 1, mtu 1514, header 14 Jun 18 19:23:27 firestorm ntop[54318]: Initializing gdbm databases Jun 18 19:23:27 firestorm ntop[54318]: VENDOR: Loading MAC address table. Jun 18 19:23:27 firestorm ntop[54318]: VENDOR: Checking for MAC address table file Jun 18 19:23:27 firestorm ntop[54318]: VENDOR: File '/usr/local/etc/ntop/specialMAC.txt.gz' does not need to be reloaded Jun 18 19:23:27 firestorm ntop[54318]: VENDOR: ntop continues ok Jun 18 19:23:27 firestorm ntop[54318]: VENDOR: Checking for MAC address table file Jun 18 19:23:27 firestorm ntop[54318]: VENDOR: File '/usr/local/etc/ntop/oui.txt.gz' does not need to be reloaded Jun 18 19:23:27 firestorm ntop[54318]: VENDOR: ntop continues ok Jun 18 19:23:27 firestorm ntop[54318]: Fingeprint: Loading signature file. Jun 18 19:23:28 firestorm ntop[54318]: Fingeprint: ...loaded 1697 records Jun 18 19:23:28 firestorm ntop[54318]: INIT: Parent process is exiting (this is normal) Jun 18 19:23:28 firestorm ntop[54319]: INIT: Bye bye: I'm becoming a daemon... Jun 18 19:23:28 firestorm ntop[54319]: Now running as a daemon Jun 18 19:23:28 firestorm ntop[54319]: ASN: Checking for Autonomous System Number table file Jun 18 19:23:28 firestorm ntop[54319]: **WARNING** ASN: Unable to open file 'AS-list.txt' Jun 18 19:23:28 firestorm ntop[54319]: I18N: This instance of ntop does not support multiple languages Jun 18 19:23:28 firestorm ntop[54319]: IP2CC: Checking for IP address <-> Country Code mapping file Jun 18 19:23:28 firestorm ntop[54319]: IP2CC: Loading file '/usr/local/etc/ntop/p2c.opt.table.gz' Jun 18 19:23:29 firestorm ntop[54319]: IP2CC: ...found 52395 lines Jun 18 19:23:29 firestorm ntop[54319]: GDVERCHK: Guessing at libgd version Jun 18 19:23:30 firestorm kernel: pid 54319 (ntop), uid 0: exited on signal 5 (core dumped) Jun 18 19:24:53 firestorm ntop[54339]: Now running as a daemon Jun 18 19:24:54 firestorm ntop[54339]: ASN: Checking for Autonomous System Number table file Jun 18 19:24:54 firestorm ntop[54339]: **WARNING** ASN: Unable to open file 'AS-list.txt' Jun 18 19:24:54 firestorm ntop[54339]: I18N: This instance of ntop does not support multiple languages Jun 18 19:24:54 firestorm ntop[54339]: IP2CC: Checking for IP address <-> Country Code mapping file Jun 18 19:24:54 firestorm ntop[54339]: IP2CC: Loading file '/usr/local/etc/ntop/p2c.opt.table.gz' Jun 18 19:24:55 firestorm ntop[54339]: IP2CC: ...found 52395 lines Jun 18 19:24:55 firestorm ntop[54339]: GDVERCHK: Guessing at libgd version Jun 18 19:24:56 firestorm kernel: pid 54339 (ntop), uid 0: exited on signal 5 (core dumped) This represents running ntop once with -L and once without. I don't see why it's so insistent on daemonizing even when I do not supply the -d flag. Running ntop normally looks like this in the logs: Jun 18 16:03:19 firestorm ntop[22149]: ntop v.3.1 MT (SSL) Jun 18 16:03:19 firestorm ntop[22149]: Configured on Feb 23 2005 8:38:57, built on Feb 23 2005 08:41:18. Jun 18 16:03:19 firestorm ntop[22149]: Copyright 1998-2004 by Luca Deri <[EMAIL PROTECTED]> Jun 18 16:03:19 firestorm ntop[22149]: Get the freshest ntop from http://www.ntop.org/ Jun 18 16:03:19 firestorm ntop[22149]: Initializing ntop Jun 18 16:03:19 firestorm ntop[22149]: NONBLOCK: pcap_setnonblock(), device 0, succeeded Jun 18 16:03:19 firestorm ntop[22149]: **WARNING** NONBLOCK: cpu usage may be high - see man page and FAQ Jun 18 16:03:19 firestorm ntop[22149]: Checking xl0 for additional devices Jun 18 16:03:19 firestorm ntop[22149]: Resetting traffic statistics for device xl0 Jun 18 16:03:19 firestorm ntop[22149]: DLT: Device 0 [xl0] is 1, mtu 1514, header 14 Jun 18 16:03:19 firestorm ntop[22149]: NONBLOCK: pcap_setnonblock(), device 1, succeeded Jun 18 16:03:19 firestorm ntop[22149]: **WARNING** NONBLOCK: cpu usage may be high - see man page and FAQ Jun 18 16:03:19 firestorm ntop[22149]: Checking xl1 for additional devices Jun 18 16:03:19 firestorm ntop[22149]: Resetting traffic statistics for device xl1 Jun 18 16:03:19 firestorm ntop[22149]: DLT: Device 1 [xl1] is 1, mtu 1514, header 14 Jun 18 16:03:19 firestorm ntop[22149]: Initializing gdbm databases Jun 18 16:03:19 firestorm ntop[22149]: VENDOR: Loading MAC address table. Jun 18 16:03:19 firestorm ntop[22149]: VENDOR: Checking for MAC address table file Jun 18 16:03:19 firestorm ntop[22149]: VENDOR: File '/usr/local/etc/ntop/specialMAC.txt.gz' does not need to be reloaded Jun 18 16:03:19 firestorm ntop[22149]: VENDOR: ntop continues ok Jun 18 16:03:19 firestorm ntop[22149]: VENDOR: Checking for MAC address table file Jun 18 16:03:19 firestorm ntop[22149]: VENDOR: File '/usr/local/etc/ntop/oui.txt.gz' does not need to be reloaded Jun 18 16:03:19 firestorm ntop[22149]: VENDOR: ntop continues ok Jun 18 16:03:19 firestorm ntop[22149]: Fingeprint: Loading signature file. Jun 18 16:03:20 firestorm ntop[22149]: Fingeprint: ...loaded 1697 records Jun 18 16:03:20 firestorm ntop[22150]: INIT: Bye bye: I'm becoming a daemon... Jun 18 16:03:20 firestorm ntop[22149]: INIT: Parent process is exiting (this is normal) Jun 18 16:03:20 firestorm ntop[22150]: Now running as a daemon Jun 18 16:03:20 firestorm ntop[22150]: ASN: Checking for Autonomous System Number table file Jun 18 16:03:20 firestorm ntop[22150]: **WARNING** ASN: Unable to open file 'AS-list.txt' Jun 18 16:03:20 firestorm ntop[22150]: I18N: This instance of ntop does not support multiple languages Jun 18 16:03:20 firestorm ntop[22150]: IP2CC: Checking for IP address <-> Country Code mapping file Jun 18 16:03:20 firestorm ntop[22150]: IP2CC: Loading file '/usr/local/etc/ntop/p2c.opt.table.gz' Jun 18 16:03:21 firestorm ntop[22150]: IP2CC: ...found 52395 lines Jun 18 16:03:21 firestorm ntop[22150]: GDVERCHK: Guessing at libgd version Jun 18 16:03:21 firestorm ntop[22150]: GDVERCHK: ... as 2.0.21+ Jun 18 16:03:21 firestorm ntop[22150]: Initializing external applications Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Started thread (134617600) for network packet analyser Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Started thread (134618112) for fingerprinting Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Started thread (134618624) for idle hosts detection Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Started thread (137056256) for DNS address resolution Jun 18 16:03:21 firestorm ntop[22150]: Calling plugin start functions (if any) Jun 18 16:03:21 firestorm ntop[22150]: SSL is present but https is disabled: use -W <https port> for enabling it Jun 18 16:03:21 firestorm ntop[22150]: INITWEB: Initializing web server Jun 18 16:03:21 firestorm ntop[22150]: INITWEB: Initializing tcp/ip socket connections for web server Jun 18 16:03:21 firestorm ntop[22150]: INITWEB: Converted address '127.0.0.1' - binding to the specific interface Jun 18 16:03:21 firestorm ntop[22150]: INITWEB: Initialized socket, port 3000, address 127.0.0.1 Jun 18 16:03:21 firestorm ntop[22150]: INITWEB: Waiting for HTTP connections on 127.0.0.1 port 3000 Jun 18 16:03:21 firestorm ntop[22150]: INITWEB: Starting web server Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Started thread (137056768) for web server Jun 18 16:03:21 firestorm ntop[22150]: Listening on [xl0,xl1] Jun 18 16:03:21 firestorm ntop[22150]: Loading Plugins Jun 18 16:03:21 firestorm ntop[22150]: Searching for plugins in /usr/local/lib/ntop/plugins Jun 18 16:03:21 firestorm ntop[22150]: ICMP: Welcome to icmpWatchPlugin. (C) 1999-2004 by Luca Deri Jun 18 16:03:21 firestorm ntop[22150]: LASTSEEN: Welcome to LastSeenWatchPlugin. (C) 1999 by Andrea Marangoni Jun 18 16:03:21 firestorm ntop[22150]: NETFLOW: Welcome to NetFlow.(C) 2002-04 by Luca Deri Jun 18 16:03:21 firestorm ntop[22150]: PDA: Welcome to PDAPlugin. (C) 2001-2004 by L.Deri and W.Brock Jun 18 16:03:21 firestorm ntop[22150]: RRD: Welcome to rrdPlugin. (C) 2002-04 by Luca Deri. Jun 18 16:03:21 firestorm ntop[22150]: SNMP: Welcome to snmpPlugin. (C) 2004 by F.Fusco and G.Giardina Jun 18 16:03:21 firestorm ntop[22150]: SFLOW: Welcome to sFlow.(C) 2002-04 by Luca Deri Jun 18 16:03:21 firestorm ntop[22150]: XML: Welcome to xmldump plugin. (C) 2003-2004 by Burton Strauss Jun 18 16:03:21 firestorm ntop[22150]: Calling plugin start functions (if any) Jun 18 16:03:21 firestorm ntop[22150]: RRD: Welcome to the RRD plugin Jun 18 16:03:21 firestorm ntop[22150]: RRD: Mask for new directories is 0700 Jun 18 16:03:21 firestorm ntop[22150]: RRD: Mask for new files is 0066 Jun 18 16:03:21 firestorm ntop[22150]: RRD: Started thread (137057792) for data collection. Jun 18 16:03:21 firestorm ntop[22150]: Now running as requested user 'nobody' (65534:65534) Jun 18 16:03:21 firestorm ntop[22150]: INIT: Created pid file (/var/db/ntop/ntop.pid) Jun 18 16:03:21 firestorm ntop[22150]: Note: Reporting device initally set to 0 [xl0] Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Started thread (137058304) for network packet sniffing on xl0 Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Started thread (137058816) for network packet sniffing on xl1 Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Packet processor thread running... Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Fingerprint scan thread running... Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Idle host scan thread running... Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: Address resolution thread running... Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: web connections thread (22150) started... Jun 18 16:03:21 firestorm ntop[22150]: Note: SIGPIPE handler set (ignore) Jun 18 16:03:21 firestorm ntop[22150]: WEB: ntop's web server is now processing requests Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: rrd thread (137057792) started Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: pcapDispatch(xl0) thread running... Jun 18 16:03:21 firestorm ntop[22150]: THREADMGMT: pcapDispatch(xl1) thread running... Jun 18 16:08:46 firestorm ntop[22150]: **WARNING** RRD: rrd_update(/var/db/ntop/rrd/interfaces/xl0/ethernetPkts.rrd) error: illegal attempt to update using time 1119125326 when last update time is 1590073200 (minimum one second step) ... lots of these for a variety of RRDs for as much as half an hour until it just stops via: Jun 18 16:33:30 firestorm kernel: pid 22150 (ntop), uid 65534: exited on signal 11 Going from running to not running when not operating through GDB looks something like this in vmstat: 1 0 0 134376 213724 505 0 0 0 477 0 0 0 2233 164964 2192 19 81 0 1 0 0 134376 213724 523 0 0 0 496 0 24 0 2245 159212 2242 23 77 0 1 0 0 134376 213724 505 0 0 0 471 0 0 0 2213 162478 2164 18 82 0 1 0 0 134376 213724 523 0 0 0 488 0 0 0 2237 164145 2200 17 83 0 2 0 0 134376 213724 505 0 0 0 472 0 0 0 2249 162377 2205 23 77 0 1 0 0 134376 213724 523 0 0 0 495 0 6 0 2237 160353 2210 20 80 0 0 0 0 83632 249688 1266 0 0 0 10236 0 6 0 2324 42139 12978 15 66 20 0 0 0 83632 249684 503 0 0 0 470 0 2 0 2515 797 2699 1 9 90 0 0 0 83632 249684 520 0 0 0 492 0 0 0 2316 835 2348 3 7 90 0 0 0 83632 249684 503 0 0 0 471 0 0 0 2299 793 2307 1 10 89 This host used to have half as much memory as it does now. I can see I'm a bit CPU constrained too based on the log and vmstat output. I tried using the -b option but found that didn't affect things either. Could CPU limitations be causing the crash? It doesn't appear to be hogging memory, the amount of memory it's consuming (virtual, that is) is about 50 megs (40 resident). How can I continue to debug this? -Yanek. -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
