Re: [pmacct-discussion] pmacct count only 5% of SYN packets.

2010-02-11 Thread Yavetskiy Yuriy 
Sorry, pmacct correctly calculates SYNs. The problem was in duplicated 
entries, this caused lost 95% of the data. This problem occurs only with 
tables version 7 (which are used as IDS). For traffic calculation I use 
a table version 1 without any problems. I've corrected the problem with 
the SYNs calculation by adding id autoincrement (and periodic zeroing of 
it) in the table version 7, now SYNs are being calculated correctly. 
However, this caused a problem, during one timestamp base grows up to 
500 megabytes. Please advise, why pmacct creates duplicated entries? 
Alsa I often see in log ERROR ( min-ddos/mysql ): FUNCTION 
pmacct.DROM_UNIXTIME does not exist#012 and You have an error in your 
SQL syntax; check the manual that corresponds to your MySQL server 
version for the right syntax to use near 'FROM]UNIXTIME(1265882756), 
FROM_UNIXTIME(1265882580), 0, '89.184.64.34', '193.17' at line 1#012  
(in first error I see that pmacct missplaced F and D, in second I 
see that . and ] are misplaced).


Here is my config:

! pmacctd configuration
!
!
!
debug: false
daemonize: true
pidfile: /var/run/pmacctd.pid
syslog: daemon
interface: eth2
promisc: true
plugin_buffer_size: 1024000
plugin_pipe_size: 40960
aggregate[min]: src_mac, dst_mac, src_host, dst_host
aggregate[min-ids]: src_host, dst_host, dst_port, proto, tcpflags
aggregate[hourly-in]: dst_host
aggregate[hourly-out]: src_host
plugins: mysql[min], mysql[min-ids], mysql[hourly-in], mysql[hourly-out]
networks_file[min]: /etc/pmacct/networks.list
networks_file[hourly-in]: /etc/pmacct/networks.list
networks_file[hourly-out]: /etc/pmacct/networks.list
sql_table[min]: acct
sql_table[min-ids]: acct_ids
sql_table[hourly-in]: acct_base_in
sql_table[hourly-out]: acct_base_out
sql_host: 10.7.10.2
sql_user: pmacct
sql_passwd: **
sql_db: pmacct
sql_table_version[min]: 1
sql_table_version[min-ids]: 7
sql_table_version[hourly-in]: 1
sql_table_version[hourly-out]: 1
sql_dont_try_update: true
sql_multi_values: 100
sql_locking_style: row

sql_history_roundoff[min]: m
sql_history[min]: 1m
sql_refresh_time[min]: 60
sql_history_roundoff[min-ids]: m
sql_history[min-ids]: 1m
sql_refresh_time[min-ids]: 60

sql_history_roundoff[hourly-in]: m
sql_history[hourly-in]: 30m
sql_refresh_time[hourly-in]: 1800
sql_history_roundoff[hourly-out]: m
sql_history[hourly-out]: 30m
sql_refresh_time[hourly-out]: 1800
sql_recovery_logfile[min]: /var/lib/pmacct/recovery_in_log
sql_recovery_logfile[hourly-in]: /var/lib/pmacct/recovery_log_in_base
sql_recovery_logfile[hourly-out]: /var/lib/pmacct/recovery_log_out_base



--
WBR
Yavetskiy Yuriy
ULTI-RIPE


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] pmacct count only 5% of SYN packets.

2010-02-09 Thread Paolo Lucente 
Hi Yuriy,

You have also other means to get a count of the TCP/SYN packets out
of pmacct. I would suggest one for troubleshooing purposes with the
goal to check where the issue lies:

* keep the 'tcpflags' primitive out of the 'aggregate' directive
* add a 'pcap_filter' directive to the config; it supports filters
  in tcpdump syntax and you can feed it with the same filter you
  use in tcpdump to count TCP/SYN packets. 

I would like to know if counting TCP/SYN packets this way makes
pmacct matching the numbers you get out of tcpdump.

Btw, if you like this strategy, instead of resorting to tcpflags, 
it can be refined so to make it co-existing with other things you
might want to do with the tool (ie. by using tagging or replacing
the 'pcap_filter' with an 'aggregate_filter').

Cheers,
Paolo


On Tue, Feb 09, 2010 at 12:39:52PM +0200, Yavetskiy Yuriy wrote:
 Hello.

 I'm running Debian with vanila 2.6.32 kernel.
 I've compiled pmacct 0.11.6 with pf_ring (transparent mode 1).
 Packets pass through brigde of 2 interfaces, one interface in promisc  
 mode (both interfaces are intel 82576, NAPI, LRO, RSS).
 Load on this brigde is 800mbps and 100 kpps (90% idle on each of 8 cores).
 I'm using pmacct to count traffic through brigde.
 I've noticed, that snmp data and pmacct's data are the same (99% similar  
 in MB).
 But if I use tcpflags in aggregation (src_host, dst_host, dst_port,  
 proto, tcpflags), I see (compare pmacct's data on bridge with tcpdump on  
 packet's destination host) that only 5% of packets with flag 2 (SYN) are  
 counted.
 What could be the problem?


 -- 
 WBR
 Yavetskiy Yuriy
 ULTI-RIPE


 ___
 pmacct-discussion mailing list
 http://www.pmacct.net/#mailinglists

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists