Roel, how didi you configure ntop precisely? The fact that ntop uses 1.8 Gb of RAM somehow means that you are monitoring (too) many hosts, or at least that you have configured ntop to produce too precise stats.
Cheers, Luca R.H.Hoek wrote: > Burton Strauss wrote on 4-6-2005 14:25: > > >How about the stats from the plugin itself? > > > The packetcounts from Ntop netflow are taken from the stats from the > plugin itself: > > Flow Statistics > Received Flows > Flow Senders 130.89.244.12 [2,251,319 pkts] > > Number of Packets Received 2,251,319 > Number of Packets with Bad Version 0 > Number of Packets Processed 2,251,319 > Number of Valid Flows Received 65,850,914 > Average Number of Flows per Packet 29.2 > Number of V1 Flows Received 0 > Number of V5 Flows Received 65,850,914 > Number of V7 Flows Received 0 > Number of V9 Flows Received 0 > > Discarded Flows > Number of Flows with Zero Packet Count 0 > Number of Flows with Zero Byte Count 0 > Number of Flows with Bad Data 0 > Number of Flows with Unknown Template 0 > Total Number of Flows Processed 65,850,914 > > Flowtools reports: > ------------------- > Jun 6 09:41:00 localhost flow-capture[27759]: STAT: now=1118043660 > startup=1117802440 src_ip=127.0.0.1 dst_ip=127.0.0.1 d_ver=5 > pkts=4590376 flows=133781439 lost=0 reset=0 filter_drops=0 > Jun 6 09:42:00 localhost flow-capture[27759]: STAT: now=1118043720 > startup=1117802440 src_ip=127.0.0.1 dst_ip=127.0.0.1 d_ver=5 > pkts=4592106 flows=133831723 lost=0 reset=0 filter_drops=0 > Jun 6 09:43:00 localhost flow-capture[27759]: STAT: now=1118043780 > startup=1117802440 src_ip=127.0.0.1 dst_ip=127.0.0.1 d_ver=5 > pkts=4595919 flows=133942507 lost=0 reset=0 filter_drops=0 > -------------------- > > Cisco Netflow reports: > -------------------- > UTWENTE-router>sh ip flow export > Flow export is enabled > Exporting flows to x.x.x.x (2055) > Exporting using source interface Vlan102 > Version 5 flow records > *21104941* flows exported in *703498* udp datagrams > 0 flows failed due to lack of export packet > 0 export packets were sent up to process level > 0 export packets were dropped due to no fib > 0 export packets were dropped due to adjacency issues > 0 export packets were dropped due to fragmentation failures > 0 export packets were dropped due to encapsulation fixup failures > 0 export packets were dropped enqueuing for the RP > 0 export packets were dropped due to IPC rate limiting > UTWENTE-router>sh mls nde > Netflow Data Export enabled > Exporting flows to x.x.x.x (2055) > Exporting flows from y.y.y.y (49744) > Version: 5= > Include Filter not configured > Exclude Filter not configured > Total Netflow Data Export Packets are: > *3892409* packets, 0 no packets, *112837207* records > Total Netflow Data Export Send Errors: > IPWRITE_NO_FIB = 0 > IPWRITE_ADJ_FAILED = 0 > IPWRITE_PROCESS = 0 > IPWRITE_ENQUEUE_FAILED = 0 > IPWRITE_IPC_FAILED = 0 > IPWRITE_MTU_FAILED = 0 > IPWRITE_ENCAPFIX_FAILED = 0 > UTWENTE-router>sho clo > 09:42:52.569 MET-DST Mon Jun 6 2005 > -------------------- > > This measurement runs from Fri 3jun > Cisco and Flowtools reports both the same packets/flows: > > Cisco: 21104941+112837207= 133,942,148 flowtools: 133,942,507 (flows) > Cisco: 703498+3892409= 4,595,907 flowtools: 4,595,919 (packets) > > Netflowplugin: 65,850,914 flows > Netflowplugin: 2,251,319 packets > > > N.B. counters are reset and read manualy on 'about' the same time. > (within 30 sec) > > >-----Burton > > >-----Original Message----- > >From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On > Behalf > >Of R.H.Hoek > >Sent: Friday, June 03, 2005 9:15 AM > >To: [email protected] > >Subject: [Ntop-dev] netflow-plugin misses packets > > >Dear Ntoppers, > > >Some weeks ago I already mentioned that I experience some problems > with Ntop > >in combination with Netflow from a Cisco6509 with supervisor2. > >Some of you gave me some hints, but nothing couldn't solve my problem > (see > >below). > > >The problem is that the Netflow-plugin misses about 40-60% of the > >netflowpackets send by the Cisco6509. Some figures: > > >Last night, in a period of about 17 hours the Cisco has sendout > >31,274,087 flows in 1,073,210 packets. > >The Netflow-plugin reports it received 16,035,229 valid flows in 548,345 > >packets. The plugin reports no discarded flow. > > >My first though is that it is not the hardware. I (can) run Ntop on two > >different systems, but experience on both the same problem: > >System1: > >Dual-Xeon 3.2GHz, 5GB RAM, with 4 disks in two hardware SCSI raidsets > >(RAID1)(18GB,73GB) and 100M/1G Ethernet interface with a 100Mbps > >networkconecction. On this system the OS is SuSE-UnitedLinux 8. (2.4 > kernel) > >System2(originally meant for probe): > >Dual-Xeon 3.2GHz, 3GB RAM, with 2 disks in a hardware SCSI raidsets > >(RAID1)(18GB) and 100M/1G Ethernet interface with a 100Mbps > >networkconecction. On this system the OS is Debian Sarge 2.6 kernel. > > >I have done testing with Ntop 3.1 and the latest CVS versions -> same > >results. With tcpdump I did some counting. The conclusion is that the > >packets send by Cisco6509 are received on the system Ntop is running > on, but > >are discarded/missed by the Netflow-plugin > > >Answers on some questions: > >1) > >Ntop is running with --interface-none, --track-local-host and the Netflow > >pseudo-nic is selected. > >2) > >Even, with RRD-plugin is set to low-detail, and data to dump is 'none' > >does not solve the problem > >3) > >for testing I have incremented MAX_SUBNET_HOSTS to 8192/16384 in > >globals-defines.h. > >Load avg 0.14 (98% idle) > >The memory usage is about 1.8GB (3.3 GB free) after 1 hour Ntop is > started. > >4) > >The load graph and (traffic)protocol-distribution should display the > total > >network load of the Cisco6509 when the pseudo-nic is selected. I have > seen > >this on a Ntop system with low netflowtraffic. > >5) > >In a test config the Cisco-Netflow is first send to system2 and > >redistributed with flow-fanout to system1 running Ntop. On system2 I do > >flow-capture and flow-stat. When I compare this output with > Ntop-output, the > >differences are very large as of 5min net workload and protocol > >distribution. > >6) > >The average netflowstream is about 20 packets/s. But I have seen > bursts of > >900 packets/s > > > > >I think that the last point is possibly the problem. With these > bursts the > >netflowplugin inputbuffer is overloaded ? > >Is there a way to tune this buffer? Or does anybody else has some hints? > > > >-- > > >Groeten, > > >Roel H.Hoek, SeniorNetworkmanager > >Dienst Informatietechnologie, Bibliotheek en Educatie (ITBE) Universiteit > >Twente, Postbus 217, 7500 AE Enschede kmr SP 422, telefoon: 053 - 489 > >4598, fax: 053 - 489 2383 > >e-mail: [EMAIL PROTECTED] http://www.utwente.nl/itbe > > _______________________________________________ > Ntop-dev mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-dev > > _______________________________________________ > Ntop-dev mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-dev > > > -- > > Groeten, > > Roel H.Hoek, SeniorNetworkmanager > Dienst Informatietechnologie, Bibliotheek en Educatie (ITBE) > Universiteit Twente, Postbus 217, 7500 AE Enschede > kmr SP 422, telefoon: 053 - 489 4598, fax: 053 - 489 2383 > e-mail: [EMAIL PROTECTED] http://www.utwente.nl/itbe > _______________________________________________ Ntop-dev mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-dev -- Luca Deri <[EMAIL PROTECTED]> http://luca.ntop.org/ Hacker: someone who loves to program and enjoys being clever about it - Richard Stallman _______________________________________________ Ntop-dev mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
