Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-12-01 Thread Vaggelis Koutroumpas
Hello Paolo, I guess I was wrong about the numbers not being off too much. I had to wait for more data to be collected. As time passes the total bytes accounted are getting way off. What would be the maximum accepted discrepancy in an ideal setup? I know that there will be differences between SNM

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-12-01 Thread Vaggelis Koutroumpas
Hello Mario, Yes they include everything AFAIK, but we don't have any multicast traffic and the the broadcast traffic is very little on our VLANs (mostly standard LAMP servers). Plus the uplink interfaces (which I am monitoring/exporting flows for) do not handle any broadcast traffic (except the s

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-11-30 Thread Jentsch, Mario
Vaggelis Koutroumpas > Sent: Sunday, November 29, 2015 12:23 AM > To: Paolo Lucente; pmacct-discussion@pmacct.net > Subject: Re: [pmacct-discussion] nfacct total bytes inconsistencies > > It seems that the new server shows the same behavior after all :( > > > mysql>

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-11-29 Thread Paolo Lucente
Hi Vaggelis, In your previous email it seems that for some period of time numbers were not 'that off'; is it a behaviour that you manage to reproduce if you stop/start nfacctd? I essentially wonder if it was a coincidence or there is effectively some degradation. Also, i see that you use 'aggrega

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-11-29 Thread Vaggelis Koutroumpas
Hello Markus, On the new server I don't have any drops at all. root@netflow:~# cat /proc/net/udp sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops 696: :0044 : 07 : 00: 0

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-11-29 Thread Markus Weber
On 28.11.2015 21:22, Vaggelis Koutroumpas wrote: Now, checking the udp drop counters on the old server, indeed I see some 25000+ drops. That counter seem to increase during the refresh time of the sql plugin. Not always though. Is there a connection between the drops and the mysql insert/update p

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-11-28 Thread Vaggelis Koutroumpas
It seems that the new server shows the same behavior after all :( mysql> SELECT ( -> SELECT concat(truncate((sum(bytes)/1024/1024/1024),2), 'GB') as bytesFROM hourlyWHERE ip_dst = '0.0.0.0' AND stamp_inserted BETWEEN '2015-11-28 20:00:00' AND '2015-11-28 23:59:59' -> ) as t

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-11-28 Thread Vaggelis Koutroumpas
Hi Paolo, > Posed I'm no expert of RouterOS; if it has a NetFlow export process, > can you check if it pegs at 100% CPU? Or if anything suspicious emerges > from the router logs? The netflow process runs at 0.1-0.2% CPU (on a 36core router). Unfortunately RouterOS' netflow options and stats are v

Re: [pmacct-discussion] nfacct total bytes inconsistencies

2015-11-28 Thread Paolo Lucente
Hi Vaggelis, I fully agree SNMP should be taken as ground truth; I have a couple of suggestions how to start troubleshooting this. Posed I'm no expert of RouterOS; if it has a NetFlow export process, can you check if it pegs at 100% CPU? Or if anything suspicious emerges from the router logs?

[pmacct-discussion] nfacct total bytes inconsistencies

2015-11-27 Thread Vaggelis Koutroumpas
Hello, I am using nfacct with Mikrotik RouterOS to account for the traffic our clients do each month. I do aggregation per IP to get the total bytes for each IP for all our prefixes. nfacct seems to be working fine with Mikrotik (it receives the flows without any errors when running in debug mode