See [BMS] in-line.
-----Burton
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of John
Barbieri
Sent: Thursday, September 15, 2005 1:46 PM
To: [email protected]
Subject: Re: [Ntop] Ntop and Netflows
That doesnt truley help me.
First off, I am not even listening in promiscuous mode, ntop is only
accepting netflows.
[BMS] You didn't give us your command line... Few people figure out -i none
w/o understanding ntop pretty well.
I am not running FreeBSD, I am running Fedora Core 4.
[BMS] Misread 5.4 as the version# of FreeBSD...
The load on the Server was around 0.10.....until all the free memory was
used, then it started going to swap. Ive never seen a program use 95% of
real memory (95% of 1GB). Thats just an insane amount of RAM for such a
little program.
[BMS] Did you read the articles and stuff about memory? It's totally
dependent upon what you are monitoring - if each of your 1000 users contacts
25 hosts, that's 26K hosts - or 100MB. More hosts, more contacts = more
memory.
[BMS] If you overflow into swap space - whatever the load - it's going to
hurt. There's no way to scan all hosts (for things like throughput, top 3
users, etc.) without, well, scanning all hosts. E.g.:
[BMS] There is a discrepancy in Linux between the system usage of memory and
what ntop is really using. That's a function of Linux's memory handling.
The OS grabs all but about 64M of memory and uses it for it's own buffers,
then doles it back as processes need it. That's why free (or vmstat) never
seem to show you what's being used in any meaningful way:
$ free
total used free shared buffers cached
Mem: 839756 779796 59960 0 321132 299228
-/+ buffers/cache: 159436 680320
Swap: 1012084 200 1011884
$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy id
wa
2 3 200 59716 321132 299228 0 0 10 9 26 23 92 1 7
0
$ top -p 32692,5357
Tasks: 2 total, 0 running, 2 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.3% us, 1.0% sy, 98.3% ni, 0.0% id, 0.0% wa, 0.3% hi, 0.0% si
Cpu1 : 0.3% us, 0.7% sy, 99.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 839756k total, 782840k used, 56916k free, 321196k buffers
Swap: 1012084k total, 200k used, 1011884k free, 302024k cached
PID TIME+ #C S %CPU %MEM nFLT VIRT SWAP CODE DATA SHR RES RUSER
COMMAND
32692 0:02.35 0 S 0.3 3.8 0 149m 118m 60 138m 2632 31m ntop
/devel/bin/ntop -i eth
5357 43:08.80 1 S 0.0 4.1 3 140m 106m 56 129m 2860 33m ntop
/usr/bin/ntop -i eth1,
[BMS] Different versions of Linux (kernel, glibc) also don't report memory
usage the same way. If you had shared with us the data from textinfo.html,
we might have seen your actual usage.
Memory allocation - data segment
arena limit, getrlimit(RLIMIT_DATA, ...).....-1
Allocated blocks (ordblks).....20
Allocated (arena).....5361664
Used (uordblks).....5229072
Free (fordblks).....132592
Memory allocation - mmapped
Allocated blocks (hblks).....8
Allocated bytes (hblkhd).....10059776
Memory Usage
IPX/SAP Hash Size (bytes).....1897
IP to country flag table (bytes).....1614732 (1.5 MB)
Bytes per entry.....30.8
IP to AS (Autonomous System) number table (bytes).....0 (0.0 MB)
Current memory usage.....15421440
Base memory usage.....13709312
Hosts stored (active+cache).....37 = (37 + 0)
(very) Approximate memory per host.....45.2KB
[BMS] Notice that while top says 149m, malloc() (glibc) only reports
allocating 15m...
Is there a way to have ntop dump what it knows (speed and protocols) to an
RRD file and then free up all the RAM?
[BMS] That's what idle purge does.
Im not interested in viewing all this data on the ntop site (although it
would be helpful) i just like how ntop dumps to the RRD files that i can
graph.
[BMS] If you don't want the benefits of ntop, then it's not the right tool
for you - look at snmp based tools like mrtg.
I only sent 1/5 of our traffic to the ntop box (1 router out of 4), and the
fact that it couldnt even handle that is kind of sad.
On one of the pdfs, rrdandntop,
"An ISP using ntop to monitor a couple of T3s needs a FAST computer and A
LOT of memory"
How fast is a fast computer and how much memory does it need?
[BMS] It depends on way too many factors. That's why there's data printed
in textinfo.html - so you can see what the per host memory is for YOUR
hosts.
If I disable TCP sessions, will ntop still dump to RRD files with the
correct protocol usage? Will it take some of the load off of the box?
[BMS] Yes - read the docs/FAQ articles on reducing workload.
I was hoping someone knew some tweaks/tricks to keep memory usage down so it
doesnt have to goto swap.
[BMS] We've talked about this endlessly - read the man page for the various
options and the articles in docs/FAQ about workload. Options such as
--track-local-hosts, filters, are the ones you want to look at. But
ultimately it depends on YOUR understanding of YOUR network so you can
collect the data YOU care about.
Burton Strauss wrote:
>In docs/FAQ and the back traffic for this list, we've frequently
>discussed ntop and memory usage. Reading that will give you a basis for
>framing reasonable questions...
>
>We've also discussed swap space (very bad) and the FreeBSD cpu issue
>(short answer for that one: get over it).
>
>>From docs/FAQ:
>
>Q4. I'm running out of memory.
>A. Basically ntop uses a lot of memory - it stores a chunk of
>information about each and every host it's monitoring. See "Q. Why
>does ntop use so much memory ?" and the following articles below.
>
>
>Re cpu, read the answer beginning:
>
>A. Also, understand that --set-pcap-nonblocking is going to increase
>ntop's cpu usage. It will probably come close to pegging the CPU at
>100%. Yet strangely other processes won't seem to be impacted. (Of
>course, you really should be running ntop on it's own host, anyway,
>right?).
>
>(FreeBSD 5.x just automates this process, so you don't need the switch.
>It's still the same WRT userland threads).
>
>-----Burton
>
>
>-----Original Message-----
>From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of
>John Barbieri
>Sent: Wednesday, September 14, 2005 3:43 PM
>To: [email protected]
>Subject: [Ntop] Ntop and Netflows
>
> Howdy there.
>
>
>
> Ive been playting around with ntop and netflows, and I have to say, i
> really like how ntop presents the data.
>
>
> the one thing i dont like is how much cpu power / memory is uses. On a
> dual opteron system with 1GB of ram, the load was about a 5.4 running
> fedora core 4. the load was low, until all the free memory was eaten
> up and it started to use swap.
>
>
> I was wondering if there was a way to use ntop as a netflow collector,
> but not use so much memory.
>
>
> also, this is probably for the wrong list, but if anyone knows of
> another collector / displayer out there similar to ntop, that would be
> great to. Ive been trying to use other programs such as cflow,
> flow-tools, cu-flow, flowscan, flowd etc.
>
> They did not seem to do what I wanted to (not to mention none of them
> compiled for me either =/)
>
>
>
> any help would be greatly appreciated.
>
>
> thank you
>
>
> John Barbieri
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
