Hi Vaggelis, In your previous email it seems that for some period of time numbers were not 'that off'; is it a behaviour that you manage to reproduce if you stop/start nfacctd? I essentially wonder if it was a coincidence or there is effectively some degradation.
Also, i see that you use 'aggregate_filter' in order to split inbound from outbound traffic; do you have any VLAN-tagged traffic? If yes, then that traffic would not be captured by your current filters. If not sure you can check this with a simple pmacct configuration like: plugin: memory[test] aggregate_filter[test]: vlan aggregate[test]: src_host, dst_host, src_port, dst_port, proto, vlan When doing a 'pmacct -s' to query the memory table, you should see no results (since the NetFlow v9 templates are sent every 60 secs based on your RouterOS configuration, please wait some 120 secs before drafting conclusions). Should all of this still not bring no anything conclusive, is remote- access to your collector box a possibility? If yes, we can follow-up privately: i'd be more than happy to have a look myself. Cheers, Paolo On Sun, Nov 29, 2015 at 01:22:34AM +0200, Vaggelis Koutroumpas wrote: > It seems that the new server shows the same behavior after all :( > > > mysql> SELECT ( > -> SELECT concat(truncate((sum(bytes)/1024/1024/1024),2), 'GB') > as bytes FROM hourly WHERE ip_dst = '0.0.0.0' AND stamp_inserted > BETWEEN '2015-11-28 20:00:00' AND '2015-11-28 23:59:59' > -> ) as total_out, ( > -> SELECT concat(truncate((sum(bytes)/1024/1024/1024),2), 'GB') > as bytes FROM hourly WHERE ip_src = '0.0.0.0' AND stamp_inserted > BETWEEN '2015-11-28 20:00:00' AND '2015-11-28 23:59:59' > -> ) as total_in; > +-----------+----------+ > | total_out | total_in | > +-----------+----------+ > | 101.03GB | 15.43GB | > +-----------+----------+ > 1 row in set (0.05 sec) > > While at the same time-frame observium reports higher 'total out' and > less 'total in' http://prntscr.com/983ers > > I guess the 'total in' discrepancy is acceptable. But the 'total out' is > over 6Gbytes off! > > If I increase the time-frame then the totals are more off. > > mysql> SELECT ( > -> SELECT concat(truncate((sum(bytes)/1024/1024/1024),2), 'GB') > as bytes FROM hourly WHERE ip_dst = '0.0.0.0' AND stamp_inserted > BETWEEN '2015-11-28 19:00:00' AND '2015-11-28 23:59:59' > -> ) as total_out, ( > -> SELECT concat(truncate((sum(bytes)/1024/1024/1024),2), 'GB') > as bytes FROM hourly WHERE ip_src = '0.0.0.0' AND stamp_inserted > BETWEEN '2015-11-28 19:00:00' AND '2015-11-28 23:59:59' > -> ) as total_in; > +-----------+----------+ > | total_out | total_in | > +-----------+----------+ > | 129.60GB | 19.46GB | > +-----------+----------+ > 1 row in set (0.02 sec) > > Observium: http://prntscr.com/983nxa > > Here the 'total out' is 8GBytes off. > While 'total in' seems to be a little off but in acceptable range. > > > There are no drops AFAICT. > > root@netflow:~# netstat -s | grep Udp\: -A 5 > Udp: > 817211 packets received > 688 packets to unknown port received. > 122 packet receive errors > 14971 packets sent > RcvbufErrors: 122 > > Those 122 errors are there for hours (before 20:00:00 of my query). > > root@netflow:~# cat /proc/net/udp > sl local_address rem_address st tx_queue rx_queue tr tm->when > retrnsmt uid timeout inode ref pointer drops > 696: 00000000:0044 00000000:0000 07 00000000:00000000 00:00000000 > 00000000 0 0 10611 2 ffff88007b36c780 0 > 751: 00000000:307B 00000000:0000 07 00000000:00000000 00:00000000 > 00000000 0 0 10580 2 ffff88007b36cb00 0 > > > I've also installed munin to monitor the performance of the server. > MySQL does on average 40 queries/s. > The server load is steadily 0.1 > The avg incoming packets are ~40pps > > So the server is pretty much idle to lose any data. > > Any ideas what else to check? > What would be an acceptable 'off percentage' of the bytes in comparison > with SNMP measurements? > > > Thanks. _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
