Read the man page and docs/FAQ...
* You probably want -x not -X -- limiting sessions is less useful than
limiting hosts.
* You probably do want -g | --track-local-hosts, but you then need to use -m
to tell ntop what IS local. That's covered in the FAQ.
* If you can limit the # of hosts, you may want to make them sticky (-c)
* If you only want netflow, then use -i none to disable the local capture.
* Swap == Paging for some class of machines. I'm talking swapinfo / swapon
/ vmstat type #s.
$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy id
wa
4 0 56 31932 169056 490044 0 0 30 28 24 7 96 2 2
0
^^^^^^^^ Are active swaps and that's
what kills you, when ntop walks the HostTraffic structure for things like
throughput, topN, IdlePurge, etc.
Just because the MACHINE gets to 700MB doesn't mean that the individual
processes do. Check top or the /proc files...
top - 16:55:29 up 8 days, 7:49, 2 users, load average: 2.11, 2.39, 2.22
Tasks: 55 total, 3 running, 52 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.0% us, 0.7% sy, 99.3% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si
Cpu1 : 0.0% us, 0.7% sy, 99.3% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 839756k total, 776912k used, 62844k free, 169140k buffers
Swap: 1012084k total, 56k used, 1012028k free, 490220k cached
PID TIME+ #C S %CPU %MEM nFLT VIRT SWAP CODE DATA SHR RES RUSER
COMMAND
2289 4:54.42 0 S 0.0 3.1 20 132m 107m 52 122m 2772 25m ntop
/usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf -d --use-sys
The 490220k cached (esp. w/ si so of zero) means it's not being actively
swapped, so we're ok. But it's on the edge...
-----Burton
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Neal
Rauhauser
Sent: Wednesday, June 29, 2005 11:00 AM
To: [email protected]
Subject: Re: [Ntop] ntop - high volume environment
Nowhere in here do you tell us how many hosts you are trying to monitor.
I want to see what individual hosts inside the 29 * 256 hosts are doing in
terms of total bandwidth, and occasionally we want to drill down and see
what application they're using. You're right that we don't care much about
remote nodes, unless there is some P2P stuff happening.
I took your advice and tried to limit the memory used - here with only
16384 TCP sessions I get a failure after less than ten minutes. Is that too
many sessions?
Wed Jun 29 16:35:04 2005 **ERROR** accessMutex() call 'handleSession'
failed (rc=11) [EMAIL PROTECTED]:1614] Wed Jun 29 16:35:04 2005
THREADMGMT: netFlow thread(141547520) terminated Wed Jun 29 16:35:16 2005
**WARNING** THREADMGMT: Address resolution thread terminated...
Wed Jun 29 16:35:29 2005 THREADMGMT: Idle Scan thread (134614016) terminated
Wed Jun 29 16:36:56 2005 THREADMGMT: Fingerprint Scan thread (134613504)
terminated Wed Jun 29 16:39:06 2005 **FATAL_ERROR** calloc(4) @
report.c:3881 returned NULL [no more memory?] Segmentation fault ntop# ntop
-X 16384 -w 4000 -P /var/db/ntop4000/ -i em1
Keep in mind I'm looking at Netflow exports, not direct packet capture.
I tried the -g option and all it showed me was a handful of RFC1918
addresses that shouldn't even be on the network. The 'local' address range
is a /30 of public space between two Cisco 3660 - should I make local the
whole public space the provider owns? How is this done when it is several
different IP address blocks?
>It's not the 29 * 256 of your local nets, unless you've used -m and -g
>| --track-local-hosts to tell ntop that. Otherwise, you are trying to
>monitor those 7.5K hosts + the 10, 12, 50 or 100 that each of them is
>in contact with. That can easily exhaust usable memory, even at the
>reduced per-host memory usage for each HostTraffic entry in the current
>CVS version. And regardless of how much memory you can throw at it.
>
>It's not just 'raw' memory, it's really how much ntop can grab w/o
>swapping, something that turns out to be incredibly difficult to
>determine. I've found that - even w/ 852M on Tigger (and the only
>other thing running on Tigger is my 'production' monitoring instance) -
>the real usable per-process memory is around 140M. After that,
>swapping starts and as I've discussed before, swapping kills you
>
>
I never see swap activated - the machine uses at most 768 meg of its gig
before the failure occurs. You're not talking paging here, are you? This is
some internal data structure thing?
>What you are trying to determine is the point at which ntop starts to swap.
>Then you can use the crude -X and/or -x switches to limit the number of
>HostTraffic entries.
>
>BUT: The best, long term answer is to look at the other switches, such
>as track-local-hosts and configure ntop properly.
>
>-----Burton
>
>
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
