Re: [pmacct-discussion] Checking to see if I understand this right

[Jim Archer]

Thanks Ruben!  Would you mind a few more questions?



Ruben Laban said:

> ... What we do is that we have pmacctd monitor the outside of our
> firewall contineously (which is the default for non-*flow setups,
> again: not sure if *flow setups are any
> diferent). During this monitoring it looks at all packets that pass by.
> Then
> it stores those by ip address in an in-memory table. This table is read
> and
> flushed every 5 minutes by a perl script written by myself. This perl
> script
> puts the data in to mysql-based backend. This way we have 5 minute
> interval
> data for each ip address we have in use. And allows us to calculations for
> total bandwidth, 95 percentile, etc.

Does the PERL script save every record, or does it perform some analysis
and save a summary?  If it saves all, what is the advantage over having
pmacct write the data to mySQL directly?

I don't understand what you mean by you have a 5 minute interval.  Do you
mean you have all the traffic data in 5 minute chunks summerized, or a
continuous line indexed every 5 minutes?

Also, do the memory tables maintained by pmacct have the ability to be
indexed or locked, so when your script reads the data other data can
continue to come in, so new data is not lost whiule you capture the old?

> The performance of pmacct by default is pretty decent. We currently have
> it it
> running on firewalls with ~3GHz CPUs and gigabit network interfaces. We
> have
> peaked at 180Mbps of traffic during which the cpu load went up to about
> 30%

Thats good to hear.  I am running SuperMicro PDSMI+ motherboards with 4GB
RAM and 2.66GHz dual core processors.  Vyatta tells me that, depending
upon the packet size, we should see performance in excess of a gigabit per
second.  I think much of that will depend upon the speed of the RAM and
the bus, so I got the fastest this board supports.

> (the firewalls also have about 3k of iptables rules active). The
> performance
> can be increased in a few ways: libpcap-mmap instead of the 'standard'
> libpcap or the use of PF_RING. The latter requires a kernel rebuild
> though.

Thanks for those tips!

> The bottomline is that any recent hardware (P4 based system) should be
> able
> to handle 100Mbps links without too much trouble.

I'm hoping to do a little better.  A 100 Mb/s link is 200 Mb/s, plus it
moves from interface to interface so figure 400 Mb/s.  Now I want two of
them per router, so 800 Mb/s.  We'll see if it works, or if I need better
routers ;)

Thanks again,

Jim


_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists