Hi Bjorn, Is it possible part of the traffic, the outbound one, is VLAN tagged? It's the only thing that comes to mind; if this is the case you can solve it by rewriting the current filter as:
"vlan and src net 95.211.55.128/26" Let me know. If it does not ring a bell and/or solve, can you please send over privately a brief capture of the outbound traffic? Cheers, Paolo On Wed, May 22, 2013 at 10:01:23PM +0200, Björn van den Heuvel wrote: > Hi all, > > These past few days I've been testing with PMACCT. I ran into the following > problem. > > When I configure PMACCT to fill two tables, based on the example, the table > logging the outbound traffic stays empty. > If I dump everything in one table, I see both inbound traffic (several > records with each of my IP's as the ip_dst) and outbound traffic (several > records with each of my IP's as the ip_src). > > Also, tracing with tcpdump shows traffic flowing in both directions. > > My setup: > Debian Wheezy (x64) on a small Supermicro Intel Atom based server. eth0 is > connected to a port that is configured to copy all traffic from two ports > connected to redundant uplinks fiber (TX and RX are copied, for both links). > These uplinks pull, on average, between 5 and 20Mbit/s at all times. > Eth1 is connected to our management LAN. The nics are on-board intel 1000mbit > nics. > > Here is my config that appears to work, but only fills the acct_in table > (acct_out stays empty): > ! pmacctd configuration > ! > interface: eth0 > ! > ! storage methods > plugins: mysql[in],mysql[out] > sql_host: localhost > sql_user: ****** > sql_passwd: ********* > sql_db: pmacct > sql_refresh_time: 300 > sql_history: 5m > sql_history_roundoff: m > sql_dont_try_update: true > aggregate[in]: dst_host > aggregate[out]: src_host > aggregate_filter[in]: dst net 95.211.55.128/26 > aggregate_filter[out]: src net 95.211.55.128/26 > sql_table[in]: acct_in > sql_table[out]: acct_out > > > Here's the relevant debug output: > root@flow01:~/pmacct-0.14.3# pmacctd -f /etc/pmacct/pmacctd.conf -d > DEBUG ( /etc/pmacct/pmacctd.conf ): plugin name/type: 'default'/'core'. > DEBUG ( /etc/pmacct/pmacctd.conf ): plugin name/type: 'in'/'mysql'. > DEBUG ( /etc/pmacct/pmacctd.conf ): plugin name/type: 'out'/'mysql'. > DEBUG ( /etc/pmacct/pmacctd.conf ): interface:eth0 > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_host:localhost > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_user:******** > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_passwd:*** > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_db:pmacct > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_refresh_time:300 > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_history:5m > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_history_roundoff:m > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_dont_try_update:true > DEBUG ( /etc/pmacct/pmacctd.conf ): aggregate[in]:dst_host > DEBUG ( /etc/pmacct/pmacctd.conf ): aggregate[out]:src_host > DEBUG ( /etc/pmacct/pmacctd.conf ): aggregate_filter[in]:dst net > 95.211.55.128/26 > DEBUG ( /etc/pmacct/pmacctd.conf ): aggregate_filter[out]:src net > 95.211.55.128/26 > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_table[in]:acct_in > DEBUG ( /etc/pmacct/pmacctd.conf ): sql_table[out]:acct_out > DEBUG ( /etc/pmacct/pmacctd.conf ): debug:true > INFO ( in/mysql ): 229376 bytes are available to address shared memory > segment; buffer size is 224 bytes. > INFO ( in/mysql ): Trying to allocate a shared memory segment of 6422528 > bytes. > INFO ( out/mysql ): 229376 bytes are available to address shared memory > segment; buffer size is 224 bytes. > INFO ( out/mysql ): Trying to allocate a shared memory segment of 6422528 > bytes. > OK ( default/core ): link type is: 1 > ^C( out/mysql ) *** Purging queries queue *** > ( in/mysql ) *** Purging queries queue *** > ( out/mysql ) *** Purging cache - START *** > ( in/mysql ) *** Purging cache - START *** > ( out/mysql ) *** Purging cache - END (QN: 0, ET: 0) *** > DEBUG ( in/mysql ): INSERT INTO `acct_in` (stamp_updated, stamp_inserted, > ip_dst, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src, packets, > bytes) VALUES (FROM_UNIXTIME(1369251855), FROM_UNIXTIME(1369251600), > '95.211.55.154', 0, 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0', 379, > 563254) > > ######## A whole bunch more of these inserts into in/mysql. Nothing about > out/mysql though... > > ( in/mysql ) *** Purging cache - END (QN: 29, ET: 0) *** > OK: Exiting ... > > 1384 packets received by filter > 0 packets dropped by kernel > > And with this config, I get both inbound and outbound traffic in MySQL (i > can/will use this as a temporary workaround): > ! pmacctd configuration > ! > ! > ! > !daemonize: true > !pidfile: /var/run/pmacctd1.pid > !syslog: daemon > ! > ! on this interface > interface: eth0 > ! > ! storage methods > plugins: mysql > sql_host: localhost > sql_user: **** > sql_passwd: ** > sql_db: pmacct > aggregate: src_host,dst_host > sql_table: acct > sql_refresh_time: 300 > sql_history: 5m > sql_history_roundoff: m > > > Things I did to try to get it to work: > Used both v0.14.0 (debian packaged version) and v0.14.3 (compiled myself). > Both show the same behavior. > Other things I tried; > - With and without SQL history > - with and without global "aggregate" clause > - with and without aggregate_filter[*] clauses (without filter actually also > writes to the acct_out table!) > - etc... > > My goals is to be able to measure both inbound and outbound traffic for the > VPSes and servers we host. I'd appreciate any help or tips. Thanks! > > Best regards, > Björn van den Heuvel > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
