Interesting... it's the same fd 0 problem (fd 0 should be a legal file descriptor, but RH9 socket() code hates it and hangs - except on Luca's computer).
-----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Kouprie, Robbert Sent: Tuesday, September 16, 2003 5:07 AM To: '[EMAIL PROTECTED]' Subject: FW: [Ntop-dev] NetFlow plugin thread not started at ntop startup Put some print statements in the NetFlow plugin code. Especially this check after socket() does it: --- ntop/plugins/netflowPlugin.c.orig 2003-09-16 10:52:39.000000000 +0200 +++ ntop/plugins/netflowPlugin.c 2003-09-16 12:03:17.000000000 +0200 @@ -105,6 +105,7 @@ static void setNetFlowInSocket() { struct sockaddr_in sockIn; int sockopt = 1, i; + errno = 0; if(myGlobals.netFlowInSocket > 0) { traceEvent(CONST_TRACE_ALWAYSDISPLAY, "NETFLOW: Collector terminated"); @@ -112,8 +113,16 @@ } if(myGlobals.netFlowInPort > 0) { + myGlobals.netFlowInSocket = socket(AF_INET, SOCK_DGRAM, 0); + if((myGlobals.netFlowInSocket <= 0) || (errno != 0) ) { + traceEvent(CONST_TRACE_INFO, "NETFLOW: Unable to create a socket - returned %d, error is '%s'(%d)", + myGlobals.netFlowInSocket, strerror(errno), errno); + exit(-1); + } + traceEvent(CONST_TRACE_INFO, "NETFLOW: Created a socket (%d)", myGlobals.netFlowInSocket); + setsockopt(myGlobals.netFlowInSocket, SOL_SOCKET, SO_REUSEADDR,(char *)&sockopt, sizeof(sockopt)); sockIn.sin_family = AF_INET; Sep 16 11:20:44 x222168 ntop[10824]: Starting 'NetFlow' Sep 16 11:20:44 x222168 ntop[10824]: Processing Netflow white/black list parameter '0' Sep 16 11:20:44 x222168 ntop[10824]: NETFLOW: White list initialized to '' Sep 16 11:20:44 x222168 ntop[10824]: NETFLOW: Black list initialized to '' Sep 16 11:20:44 x222168 ntop[10824]: setNetFlowInSocket: called, netFlowInSocket = -1 Sep 16 11:20:44 x222168 ntop[23224]: NETFLOW: Unable to create a socket - returned 0, error is 'Success'(0) So, my guess is the NetFlow plugin (and probably others) also suffer the FILEDESCRIPTORBUG when creating a socket. Strangely it works when the plugin is stopped/started. Probably some more handles spent then... -- Robbert -----Oorspronkelijk bericht----- Van: Kouprie, Robbert [mailto:[EMAIL PROTECTED] Verzonden: maandag 15 september 2003 16:01 Aan: '[EMAIL PROTECTED]' Onderwerp: [Ntop-dev] NetFlow plugin thread not started at ntop startup Hi, Still using today's CVS. For some reason the NetFlow plugin just won't come up after starting ntop. Of course I enabled it via Admin -> Plugins, did multiple clean shutdowns (Admin -> Shutdown) to make sure the setting is written to the configuration file. Tried with and without -i <local_interface> on the command line, didn't work. Also, the logs say NetFlow plugin is loaded and started, NetFlow-device is created and active, default reporting is set to this device, but I just don't get to see any data. Startup logs are attached below. Note, that there's no "netFlow thread started". Also I have 7 ntop threads, whereas normal I would have 8. When I disable the NetFlow plugin, en enable it again, I see this: Sep 15 15:53:04 x222168 ntop[14882]: SECURITY: Loading items table Sep 15 15:53:06 x222168 ntop[14882]: IDLE_PURGE: FINISHED selection, 0 [out of 46] hosts selected Sep 15 15:53:06 x222168 ntop[14882]: IDLE_PURGE: Device 0: no hosts deleted Sep 15 15:53:06 x222168 ntop[14882]: IDLE_PURGE: FINISHED selection, 0 [out of 2] hosts selected Sep 15 15:53:06 x222168 ntop[14882]: IDLE_PURGE: Device 1: no hosts deleted Sep 15 15:53:10 x222168 ntop[14882]: NETFLOW: Thanks for using ntop NetFlow Sep 15 15:53:10 x222168 ntop[14882]: NETFLOW: Done Sep 15 15:53:10 x222168 ntop[14882]: Processing Netflow white/black list parameter '0' Sep 15 15:53:10 x222168 ntop[14882]: NETFLOW: White list initialized to '' Sep 15 15:53:10 x222168 ntop[14882]: NETFLOW: Black list initialized to '' Sep 15 15:53:10 x222168 ntop[14882]: NETFLOW: Collector listening on port 2055 Sep 15 15:53:10 x222168 ntop[14882]: **ERROR** NetFlow-device is already active - request ignored Sep 15 15:53:10 x222168 ntop[14882]: THREADMGMT: netFlow thread(1158913328) started And it works perfectly. Notice the "netFlow thread started" here. Also I have 8 ntop threads again. Regards, -- Robbert Sep 15 15:45:58 x222168 ntop[14820]: ntop v.2.2.94 MT (SSL) [i686-pc-linux-gnu (redhat Linux 9)] (09/15/03 02:47:01 PM build) Sep 15 15:45:58 x222168 ntop[14820]: Copyright 1998-2003 by Luca Deri <[EMAIL PROTECTED]> Sep 15 15:45:58 x222168 ntop[14820]: Get the freshest ntop from http://www.ntop.org/ Sep 15 15:45:58 x222168 ntop[14820]: Initializing ntop Sep 15 15:45:58 x222168 ntop[14820]: Initializing IP services Sep 15 15:45:58 x222168 ntop[14820]: PROTO_INIT: Processing protocol file: '/usr/local/etc/ntop/protocol.list', size: 512 Sep 15 15:45:58 x222168 ntop[14820]: Initializing network devices Sep 15 15:45:58 x222168 ntop[14820]: Adding network device eth0 Sep 15 15:45:58 x222168 kernel: device eth0 entered promiscuous mode Sep 15 15:45:58 x222168 ntop[14820]: **WARNING** Truncated network size (device eth0) to 1024 hosts (real netmask 255.255.252.0) Sep 15 15:45:58 x222168 ntop[14820]: MEMORY: ipTrafficMatrix base (no TrafficEntry) for interface 'eth0' is 4.05MB Sep 15 15:45:58 x222168 ntop[14820]: Resetting traffic statistics for device eth0 Sep 15 15:45:58 x222168 ntop[14820]: Initializing gdbm databases Sep 15 15:45:58 x222168 ntop[14820]: Now running as requested user 'ntop' (502:502) Sep 15 15:45:58 x222168 ntop[14820]: Opening database '/var/lib/ntop/prefsCache.db' Sep 15 15:45:58 x222168 ntop[14820]: Opening database '/var/lib/ntop/ntop_pw.db' Sep 15 15:45:58 x222168 ntop[14820]: Creating database '/var/lib/ntop/addressQueue.db' Sep 15 15:45:58 x222168 ntop[14820]: Opening database '/var/lib/ntop/dnsCache.db' Sep 15 15:45:58 x222168 ntop[14820]: Opening database '/var/lib/ntop/macPrefix.db' Sep 15 15:45:58 x222168 ntop[14820]: VENDOR: Loading MAC address table. Sep 15 15:45:58 x222168 ntop[14820]: VENDOR: Checking './specialMAC.txt.gz' Sep 15 15:45:58 x222168 ntop[14820]: VENDOR: File './specialMAC.txt.gz' does not need to be reloaded Sep 15 15:45:58 x222168 ntop[14820]: VENDOR: Checking './oui.txt.gz' Sep 15 15:45:58 x222168 ntop[14820]: VENDOR: File './oui.txt.gz' does not need to be reloaded Sep 15 15:45:58 x222168 ntop[14821]: INIT: Bye bye: I'm becoming a daemon... Sep 15 15:45:58 x222168 ntop[14821]: Now running as a daemon Sep 15 15:45:58 x222168 ntop[14820]: INIT: Parent process is exiting (this is normal) Sep 15 15:45:58 x222168 ntop[14821]: Processing -m | --local-subnets parameter '130.161.0.0/16,145.94.0.0/16,194.171.50.0/24' Sep 15 15:45:58 x222168 ntop[14821]: OSFP: Looking for OS fingerprint file, etter.passive.os.fp.gz Sep 15 15:45:58 x222168 ntop[14821]: OSFP: Checking './etter.passive.os.fp.gz' Sep 15 15:45:58 x222168 ntop: ntop startup succeeded Sep 15 15:45:58 x222168 ntop[14821]: OSFP: Checking '/usr/local/etc/ntop/etter.passive.os.fp.gz' Sep 15 15:45:58 x222168 ntop[14821]: OSFP: ...found! Sep 15 15:45:58 x222168 ntop[14821]: AS: Looking for ASN file, AS-list.txt.gz Sep 15 15:45:58 x222168 ntop[14821]: AS: Checking './AS-list.txt.gz' Sep 15 15:45:58 x222168 ntop[14821]: AS: Checking '/usr/local/etc/ntop/AS-list.txt.gz' Sep 15 15:45:58 x222168 ntop[14821]: AS: Checking '/etc/AS-list.txt.gz' Sep 15 15:45:58 x222168 ntop[14821]: **WARNING** AS: Unable to open file 'AS-list.txt.gz'. Sep 15 15:45:58 x222168 ntop[14821]: AS: ntop continues ok, but without ASN information. Sep 15 15:45:58 x222168 ntop[14821]: I18N: This instance of ntop does not support multiple languages Sep 15 15:45:58 x222168 ntop[14821]: IP2CC: Looking for IP address <-> Country code mapping file Sep 15 15:45:58 x222168 ntop[14821]: IP2CC: ...looking for file ./p2c.opt.table.gz Sep 15 15:45:58 x222168 ntop[14821]: IP2CC: ...looking for file ./p2c.opt.table Sep 15 15:45:58 x222168 ntop[14821]: IP2CC: ...does not exist Sep 15 15:45:58 x222168 ntop[14821]: IP2CC: ...looking for file /usr/local/etc/ntop/p2c.opt.table.gz Sep 15 15:45:58 x222168 ntop[14821]: IP2CC: reading file '/usr/local/etc/ntop/p2c.opt.table.gz' Sep 15 15:45:59 x222168 ntop[14821]: IP2CC: ......49056 records read Sep 15 15:45:59 x222168 ntop[14821]: IP2CC: ...looking for file /etc/p2c.opt.table.gz Sep 15 15:45:59 x222168 ntop[14821]: IP2CC: ...looking for file /etc/p2c.opt.table Sep 15 15:45:59 x222168 ntop[14821]: IP2CC: ...does not exist Sep 15 15:45:59 x222168 ntop[14821]: IP2CC: 49056 records read Sep 15 15:45:59 x222168 ntop[14821]: Initializing external applications Sep 15 15:45:59 x222168 ntop[14821]: Initializing semaphores, mutexes and threads Sep 15 15:45:59 x222168 ntop[14821]: NOTE: atfork() handler registered for mutexes, rc 0 Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Packet processor thread running... Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Started thread (1092623664) for network packet analyser Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Idle host scan thread running... Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Started thread (1101016368) for idle hosts detection Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Address resolution thread running... Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Started thread (1116949808) for DNS address resolution Sep 15 15:45:59 x222168 ntop[14821]: Starting Plugins Sep 15 15:45:59 x222168 ntop[14821]: Calling plugin start functions (if any) Sep 15 15:45:59 x222168 ntop[14821]: Plugins started... continuing with initialization Sep 15 15:45:59 x222168 ntop[14821]: MEMORY: Base memory load is 6.65MB (2220816+4743168) Sep 15 15:45:59 x222168 ntop[14821]: MEMORY: Base interface structure (no hashes loaded) is 0.27MB each Sep 15 15:45:59 x222168 ntop[14821]: MEMORY: or 0.27MB for 1 interfaces Sep 15 15:45:59 x222168 ntop[14821]: MEMORY: ipTraffixMatrix structure (no TrafficEntry loaded) is 4.00MB Sep 15 15:45:59 x222168 ntop[14821]: Sniffying... Sep 15 15:45:59 x222168 ntop[14821]: INIT: Created pid file (/var/lib/ntop/ntop.pid) Sep 15 15:45:59 x222168 ntop[14821]: Listening on [eth0] Sep 15 15:45:59 x222168 ntop[14821]: Now running as requested user 'ntop' (502:502) Sep 15 15:45:59 x222168 ntop[14821]: Loading Plugins Sep 15 15:45:59 x222168 ntop[14821]: Searching for plugins in /usr/local/lib/ntop/plugins Sep 15 15:45:59 x222168 ntop[14821]: Loading plugin '/usr/local/lib/ntop/plugins/icmpPlugin.so' Sep 15 15:45:59 x222168 ntop[14821]: ICMP: Welcome to icmpWatchPlugin. (C) 1999 by Luca Deri Sep 15 15:45:59 x222168 ntop[14821]: Loading plugin '/usr/local/lib/ntop/plugins/lastSeenPlugin.so' Sep 15 15:45:59 x222168 ntop[14821]: LASTSEEN: Welcome to LastSeenWatchPlugin. (C) 1999 by Andrea Marangoni Sep 15 15:45:59 x222168 ntop[14821]: Compiling filter 'ip' on interface eth0 Sep 15 15:45:59 x222168 ntop[14821]: Loading plugin '/usr/local/lib/ntop/plugins/netflowPlugin.so' Sep 15 15:45:59 x222168 ntop[14821]: NETFLOW: Welcome to NetFlow.(C) 2002 by Luca Deri Sep 15 15:45:59 x222168 ntop[14821]: Loading plugin '/usr/local/lib/ntop/plugins/nfsPlugin.so' Sep 15 15:45:59 x222168 ntop[14821]: NFS: Welcome to nfsWatchPlugin. (C) 1999 by Luca Deri Sep 15 15:45:59 x222168 ntop[14821]: Compiling filter 'port 2049' on interface eth0 Sep 15 15:45:59 x222168 ntop[14821]: Loading plugin '/usr/local/lib/ntop/plugins/pdaPlugin.so' Sep 15 15:45:59 x222168 ntop[14821]: PDA: Welcome to PDAPlugin. (C) 2001-2002 by L.Deri and W.Brock Sep 15 15:45:59 x222168 ntop[14821]: Loading plugin '/usr/local/lib/ntop/plugins/sflowPlugin.so' Sep 15 15:45:59 x222168 ntop[14821]: SFLOW: Welcome to sFlowPlugin. (C) 2002 by Luca Deri Sep 15 15:45:59 x222168 ntop[14821]: Compiling filter 'ip' on interface eth0 Sep 15 15:45:59 x222168 ntop[14821]: Loading plugin '/usr/local/lib/ntop/plugins/rrdPlugin.so' Sep 15 15:45:59 x222168 ntop[14821]: RRD: Welcome to rrdPlugin. (C) 2002 by Luca Deri. Sep 15 15:45:59 x222168 ntop[14821]: Starting Plugins Sep 15 15:45:59 x222168 ntop[14821]: Calling plugin start functions (if any) Sep 15 15:45:59 x222168 ntop[14821]: Starting 'rrdPlugin' Sep 15 15:45:59 x222168 ntop[14821]: RRD: Welcome to the RRD plugin Sep 15 15:45:59 x222168 ntop[14821]: RRD: Mask for new directories is 0700 Sep 15 15:45:59 x222168 ntop[14821]: RRD: Mask for new files is 0066 Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: rrd thread (1125342512) started Sep 15 15:45:59 x222168 ntop[14821]: RRD: Started thread (1125342512) for data collection. Sep 15 15:45:59 x222168 ntop[14821]: Starting 'sFlowPlugin' Sep 15 15:45:59 x222168 ntop[14821]: Starting 'PDAPlugin' Sep 15 15:45:59 x222168 ntop[14821]: Starting 'nfsWatchPlugin' Sep 15 15:45:59 x222168 ntop[14821]: Starting 'NetFlow' Sep 15 15:45:59 x222168 ntop[14821]: Processing Netflow white/black list parameter '0' Sep 15 15:45:59 x222168 ntop[14821]: NETFLOW: White list initialized to '' Sep 15 15:45:59 x222168 ntop[14821]: NETFLOW: Black list initialized to '' Sep 15 15:45:59 x222168 ntop[14821]: NETFLOW: Collector listening on port 2055 Sep 15 15:45:59 x222168 ntop[14821]: Creating dummy interface, 'NetFlow-device' Sep 15 15:45:59 x222168 ntop[14821]: NETFLOW: Export disabled at user request Sep 15 15:45:59 x222168 ntop[14821]: Starting 'LastSeenWatchPlugin' Sep 15 15:45:59 x222168 ntop[14821]: Starting 'icmpWatchPlugin' Sep 15 15:45:59 x222168 ntop[14821]: Plugins started... continuing with initialization Sep 15 15:45:59 x222168 ntop[14821]: SSL is present but https is disabled: use -W <https port> for enabling it Sep 15 15:45:59 x222168 ntop[14821]: Device 0. eth0 (active) Sep 15 15:45:59 x222168 ntop[14821]: Device 1. NetFlow-device (dummy) (active) Sep 15 15:45:59 x222168 ntop[14821]: Note: Reporting device initally set to 1 [NetFlow-device] Sep 15 15:45:59 x222168 ntop[14821]: WEB: Initializing web server Sep 15 15:45:59 x222168 ntop[14821]: WEB: Initializing tcp/ip socket connections for web server Sep 15 15:45:59 x222168 ntop[14821]: Initializing socket, port 3000, address (any) Sep 15 15:45:59 x222168 ntop[14821]: FILEDESCRIPTORBUG: Work-around activated Sep 15 15:45:59 x222168 ntop[14821]: FILEDESCRIPTORBUG: Creating 0, '/tmp/ntop-000014821-0' Sep 15 15:45:59 x222168 ntop[14821]: FILEDESCRIPTORBUG: Created file 0 - '/tmp/ntop-000014821-0'(11) Sep 15 15:45:59 x222168 ntop[14821]: FILEDESCRIPTORBUG: Creating 1, '/tmp/ntop-000014821-1' Sep 15 15:45:59 x222168 ntop[14821]: FILEDESCRIPTORBUG: Created file 1 - '/tmp/ntop-000014821-1'(12) Sep 15 15:45:59 x222168 ntop[14821]: FILEDESCRIPTORBUG: Creating 2, '/tmp/ntop-000014821-2' Sep 15 15:45:59 x222168 ntop[14821]: FILEDESCRIPTORBUG: Created file 2 - '/tmp/ntop-000014821-2'(13) Sep 15 15:45:59 x222168 ntop[14821]: WEB: Created a new socket (14) Sep 15 15:45:59 x222168 ntop[14821]: Initialized socket, port 3000, address (any) Sep 15 15:45:59 x222168 ntop[14821]: WEB: Waiting for HTTP connections on port 3000 Sep 15 15:45:59 x222168 ntop[14821]: WEB: Starting web server Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: web connections thread (14821) started... Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Started thread (1142127920) for web server Sep 15 15:45:59 x222168 ntop[14821]: WEB: Server started... continuing with initialization Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: pcap dispatch thread running... Sep 15 15:45:59 x222168 ntop[14821]: THREADMGMT: Started thread (1150520624) for network packet sniffing on eth0 Sep 15 15:46:59 x222168 ntop[14821]: IDLE_PURGE: FINISHED selection, 0 [out of 20] hosts selected Sep 15 15:46:59 x222168 ntop[14821]: IDLE_PURGE: Device 0: no hosts deleted Sep 15 15:46:59 x222168 ntop[14821]: IDLE_PURGE: FINISHED selection, 0 [out of 2] hosts selected Sep 15 15:46:59 x222168 ntop[14821]: IDLE_PURGE: Device 1: no hosts deleted _______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev _______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev _______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
