Hello everyone,
I have been testing pmacct (or rather sfacctd) to get traffic
information from sflow data, and it is working like a charm as long as
there is only a single sflow data source. Kudos to the developer!
I still have troubles when multiple devices send sflow samples to
sfacctd. To simplify things, I have build a very simple setup with two
computers, server and client (the scenarion follows, relevant
configuration files are at the bottom).
I would like to ask if anyone has used sfacctd or nfacctd to receive
netflow data from multiple exporters and has run into similar problems
or has found solutions?
Now to the scenario (Simple 2 PC Setup):
Both PCs have InMon Agent 5.6 running and monitoring their network card
(eth0). client and server send their sflow data to server.
server is running sfacctd and is collecting data from itself (server)
and client.
server is running an ftp server hosting a 100 MB Test file.
Now client downloads the file from server, and both InMon Agents send
sflow samples to server. When I look at the traffic information
afterwards, I see 200 MB traffic from server to client.
192.168.16.211 192.168.16.222
|PC1-client|<------ FTP ---------- |PC2-server|
: :
: : SFlow Samples to localhost
: \/
\--------- SFlow Samples --> sfacctd running on PC2 -
adding the samples up in one aggregate
In this simple setup, I could just disable one probe or divide the
traffic by two, but consider this setup:
:---> sfacctd < --- sflow ------------:
: : :
|Client1| ----- |Switch1| ---------------- |Switch2| ---|Switch3| ---
|Client3|
|
|
|Client2|
Now moving 100 MB from Client1 to the other clients would show up as 100
MB for Client2, but as 300 MB for Client3.
I have been looking for a way to deal with multiple flow sources, but
have found no good solution so far.
Right now I am experimenting with pre-tagging the data from different
sensors - in the simple two PC example, my pre-tag map looks like this:
id=11 ip=192.168.16.211
id=22 ip=192.168.16.222
Now if I transfer my testfile again from server to client, I get two
entries in the database, each with 100MB (one with agent_id=11, the
other with agent_id=22)
From there I could sort the situation out in sql (and eventually reduce
my aggregate primitives to a sensible set), but this would still mean
touching
the same data twice.
In theory, this could work, one disadvantage would be that the
pretag.map file would have to be maintained manually and data from
sensors not in the map would have to be rejected - and it depends on the
Agent-ID set right on the individual sensors.
Furthermore, I am concerned about the performance requirements with a
bunch of sensors involved. Another problem is that after the data is
inserted in the database, there is no information about the sample rate,
so data from a sensor wich samples every packet is treated as having the
same accuracy as a sensor which samples only one in a zillion packets.
There sure are more problems I have not even thought about. Has anyone
used nfacctd|sfacctd in a similar setup and could give me some pointers?
Best regards,
Gregor
Configuration:
=======================================================================================
sfacctd is invoked on 192.168.16.222 with "./sfacctd -f ./sfacctd.test.conf
InMon Agent is invoked
on 192.168.16.211 with "./sflsp -d eth0 -C 192.168.16.222 -s 64 -v -c
3655 -A 192.168.16.211"
on 192.168.16.222 with "./sflsp -d eth0 -C 192.168.16.222 -s 64 -v -c
3655 -A 192.168.16.222"
========================================================================================
Configuration Files:
sfacctd.test.conf:
!!!!!!!START!!!!!!!!
daemonize: false
debug: true
plugins: pgsql
!plugins: print
print_refresh_time: 20
aggregate:
tag,src_mac,dst_mac,src_host,dst_host,src_port,dst_port,proto,tos,vlan,flows
!
! I do intend to reduce the aggregate set, but during testing I want to
have as much information as possible
!
pre_tag_map: /root/pmetc/pretag.map
interface: eth0
promisc: false
sfacctd_port: 3655
sfacctd_ip: 192.168.16.222
sfacctd_renormalize: true
sql_db: pmacct
sql_data: typed
sql_passwd: ------
sql_table_version: 6
sql_table: acct_v6_%H
sql_refresh_time: 30
sql_history: 5m
sql_history_roundoff: m
sql_table_schema: /home/pmetc/pmacct-v6.pgsql.schema
snaplen: 750
!!!!!!!!END!!!!!!!!
pretag.map
!!!START!!!!
id=11 ip=192.168.16.211
id=22 ip=192.168.16.222
!!!END!!!
pmacct-v6.pgsql.schema
CREATE TABLE acct_v6_%H (
agent_id INT NOT NULL DEFAULT 0,
class_id CHAR(16) NOT NULL DEFAULT ' ',
mac_src macaddr NOT NULL DEFAULT '0:0:0:0:0:0',
mac_dst macaddr NOT NULL DEFAULT '0:0:0:0:0:0',
vlan INT NOT NULL DEFAULT 0,
as_src INT NOT NULL DEFAULT 0,
as_dst INT NOT NULL DEFAULT 0,
ip_src inet NOT NULL DEFAULT '0.0.0.0',
ip_dst inet NOT NULL DEFAULT '0.0.0.0',
port_src INT NOT NULL DEFAULT 0,
port_dst INT NOT NULL DEFAULT 0,
ip_proto SMALLINT NOT NULL DEFAULT 0,
tos INT NOT NULL DEFAULT 0,
packets INT NOT NULL,
bytes BIGINT NOT NULL,
flows INT NOT NULL DEFAULT 0,
stamp_inserted timestamp without time zone NOT NULL DEFAULT
'0000-01-01 00:00:00',
stamp_updated timestamp without time zone,
CONSTRAINT acct_v6_pk_%H PRIMARY KEY (agent_id, class_id,
mac_src, mac_dst, vlan, as_src, as_dst, ip_src, ip_dst, port_src,
port_dst, ip_proto, tos, stamp_inserted)
);
GRANT SELECT, INSERT, UPDATE, DELETE ON acct_v6_%H TO pmacct;
========================================================================================
SQL Result after ftp transfer (without pre-tag or having the InMon
Agents started without "-A <IP>"):
pmacct=# select * from acct_v6_11 where ip_src='192.168.16.222' and
ip_dst='192.168.16.211';
agent_id | class_id | mac_src | mac_dst |
vlan | as_src | as_dst | ip_src | ip_dst | port_src |
port_dst | ip_proto | tos | packets | bytes | flows |
stamp_inserted | stamp_updated
----------+------------------+-------------------+-------------------+------+--------+--------+----------------+----------------+----------+----------+----------+-----+---------+-----------+-------+---------------------+---------------------
0 | unknown | 00:0c:29:f5:2b:7d | 00:0c:29:b7:74:fa |
0 | 0 | 0 | 192.168.16.222 | 192.168.16.211 | 1025 |
32819 | 6 | 0 | 142207 | 211933362 | 0 | 2006-05-31
11:35:00 | 2006-05-31 11:38:59
(1 row)
SQL Result after ftp transfer (with pre-tag):
pmacct=# select * from acct_v6_11 where ip_src='192.168.16.222' and
ip_dst='192.168.16.211';
agent_id | class_id | mac_src | mac_dst |
vlan | as_src | as_dst | ip_src | ip_dst | port_src |
port_dst | ip_proto | tos | packets | bytes | flows |
stamp_inserted | stamp_updated
----------+------------------+-------------------+-------------------+------+--------+--------+----------------+----------------+----------+----------+----------+-----+---------+-----------+-------+---------------------+---------------------
11 | unknown | 00:0c:29:f5:2b:7d | 00:0c:29:b7:74:fa |
0 | 0 | 0 | 192.168.16.222 | 192.168.16.211 | 1025 |
32817 | 6 | 0 | 70806 | 105297084 | 0 | 2006-05-31
11:25:00 | 2006-05-31 11:25:59
22 | unknown | 00:0c:29:f5:2b:7d | 00:0c:29:b7:74:fa |
0 | 0 | 0 | 192.168.16.222 | 192.168.16.211 | 1025 |
32817 | 6 | 0 | 73033 | 107993862 | 0 | 2006-05-31
11:25:00 | 2006-05-31 11:25:59
(2 rows)
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
