Michael Ralston wrote:
You might want to look at using something like ntop in
parallel with pmacctd... it's quite handy for identifying
unusual traffic.
Ntop is more an instantaneous thing though right? Like top is for CPU... I
need something that will log the unusual traffic so when I see a weird spike
on traffic graphs I can look at the logs and see where it came from.
It's a daemon that watches an interface and does various analysis
internally - the results are shown through a web page. All sorts of nice
reports, graphs, etc are available. By protocol, port, source,
destination, country, pie charts, etc...
I'm running 6 instances of pmacctd, with two aggregate
filters in each (IN vs OUT). Works great. The only
differences in the config files are the table table to write
to, and the sql_history parameter. RAM usage is fine -
perhaps 5-15 MB for each pmacctd. This system is counting for
around 8000 IP addresses.
What is the best way to tweak the SQL updates for performance? Ie...
sql_refresh_time, sql_history, sql_history_roundoff
My configuration file for monthly counters includes:
sql_refresh_time: 600
sql_optimize_clauses: true
sql_history: 1M
sql_table: acct_monthly
sql_table_version: 2
pmacctd_id: 1
And the ones for daily/hourly are similiar of course. As far as I know,
configuring with --enable-mmap is the best way to go if your platform
supports it.
I found that it's really important to have strip out any extra columns
from the table and have the correct indexes that will be used by the
UPDATE/INSERT queries, and also be used by any SELECT queries that you
may be doing... for example, my acct_daily table has 4 indexes. One each
for stamp_inserted, ip_src, and ip_dst, plus another for a SQL group
expression.
"acct_daily_dst" btree (ip_dst)
"acct_daily_expr" btree (date_trunc('month'::text, stamp_inserted))
"acct_daily_si" btree (stamp_inserted)
"acct_daily_si_src" btree (ip_src)
Best thing to do is take an UPDATE query that pmacctd is generating, and
using MySQL/PostgreSQL's "EXPLAIN SELECT ...." command to see what it's
actually doing.
I'm not running a postgres server... All my other software I mysql based and
I don't have the resources to run both :(
I switched, and don't regret it :-)
I tried setting ip_proto in the config and it says
WARN: ignoring unknown aggregation method: ip_proto.
Hmm. Maybe try just "proto". CONFIG-KEYS has the whole list of
aggregation methods.
Reason I am curious is because my snmp graphs are showing big traffic when
I'm getting a DOS attack, but when I check my pmacctd tables there is no
sign of the traffic there. I figured it had to be something not udp/tcp and
pmacctd was ignoring it?
Hmm. I think it's supposed to just insert either a NULL entry or a 0,
but perhaps Paolo will confirm :-)
Also, check that pmacctd is actually getting all the packets - libpcap0
and/or the kernel can drop packets if it can't keep up. There should be
messages in STDERR/STDOUT as to how many packets were dropped by the kernel.
Wim
