Hello Paolo,
I guess I was wrong about the numbers not being off too much. I had to
wait for more data to be collected. As time passes the total bytes
accounted are getting way off.
What would be the maximum accepted discrepancy in an ideal setup?
I know that there will be differences between SNMP measurements, but how
much difference is considered normal? (I know it's kind of a vague question)
Restarting nfacctd did not change anything.
I also restarted the whole box just in case.
The UDP drop counters still stay unaffected
Udp:
78879 packets received
590 packets to unknown port received.
0 packet receive errors
25 packets sent
sl local_address rem_address st tx_queue rx_queue tr tm->when
retrnsmt uid timeout inode ref pointer drops
101: :A1F1 : 07 : 00:
00 10613 2 88013a354780 0
575: 017F:2BCB : 07 : 00:
1100 10201 2 8800bacfdac0 0
1720: :0044 : 07 : 00:
00 10630 2 88013a354400 0
Regarding the VLAN traffic, I do have VLAN traffic, but Mirkotik does
not export this field as far as I can tell from the netflow template.
DEBUG ( default/core ): NfV9 agent : X.X.X.X:0
DEBUG ( default/core ): NfV9 template type : flow
DEBUG ( default/core ): NfV9 template ID : 257
DEBUG ( default/core ):
-
DEBUG ( default/core ): |pen | field type | offset |
size |
DEBUG ( default/core ): | 0 | ip version | 0 |
1 |
DEBUG ( default/core ): | 0 | IPv6 src addr | 1 |
16 |
DEBUG ( default/core ): | 0 | IPv6 src mask | 17 |
1 |
DEBUG ( default/core ): | 0 | input snmp | 18 |
4 |
DEBUG ( default/core ): | 0 | IPv6 dst addr | 22 |
16 |
DEBUG ( default/core ): | 0 | IPv6 dst mask | 38 |
1 |
DEBUG ( default/core ): | 0 | output snmp| 39 |
4 |
DEBUG ( default/core ): | 0 | IPv6 next hop | 43 |
16 |
DEBUG ( default/core ): | 0 | L4 protocol| 59 |
1 |
DEBUG ( default/core ): | 0 | tcp flags | 60 |
1 |
DEBUG ( default/core ): | 0 | tos| 61 |
1 |
DEBUG ( default/core ): | 0 | L4 src port| 62 |
2 |
DEBUG ( default/core ): | 0 | L4 dst port| 64 |
2 |
DEBUG ( default/core ): | 0 | 31 | 66 |
4 |
DEBUG ( default/core ): | 0 | 64 | 70 |
4 |
DEBUG ( default/core ): | 0 | last switched | 74 |
4 |
DEBUG ( default/core ): | 0 | first switched | 78 |
4 |
DEBUG ( default/core ): | 0 | in bytes | 82 |
4 |
DEBUG ( default/core ): | 0 | in packets | 86 |
4 |
DEBUG ( default/core ): | 0 | in dst mac | 90 |
6 |
DEBUG ( default/core ): | 0 | out src mac| 96 |
6 |
DEBUG ( default/core ):
-
DEBUG ( default/core ): Netflow V9/IPFIX record size : 102
DEBUG ( default/core ):
What drives me crazy is that if I do controlled data transfers for long
periods of time, nfacctd counts everything properly. I can see the rate
at which the bytes counter increases in the database, with which doing
the calculations results in exactly the mbit/s I am doing transfers at.
So it seems that RouterOS does export the flows properly and nfacctd
does measure the bytes properly.
And yet, when checking the results on another IP (which has normal web
traffic) then the data are always off and getting worse as time goes by.
What's even stranger is that the Download bytes (which is always less in
reality) is measured slightly higher in nfacctd (from a few MB to a few
hundred MB).
While upload data is measured less than what is actually going through
the wire. (from 1GB to 3-4GB less per hour, depending on how much
traffic the server has at any given hour)
Unfortunately the collector box is not accessible from the internet. I
understand that this would help you identify the issue much quicker than
explaining to me every possible solution to try.
I'll try to get permission to allow you access (via VPN or something) if
nothing else works.
I really do appreciate the offer to help! :)
I noticed today some sporadic info messages on nfacctd output.
INFO: expecting flow '657566' but received '657621'
collector=0.0.0.0:2055 agent=X.X.X.X1:0
INFO: expecting flow '657677' but received '657738'
collector=0.0.0.0:2055 agent=X.X.X.X:0
Is this normal? Does that mean that it lost a flow somewhere and that's
why it throws this INFO message?
I have increased the buffers:
plugin_pipe_size: 268435456
plugin_buffer_size: 268435
nfacctd_pipe_size:
