One extra non-obvious thing - depending on the router configuration it's
possible that netflow only processes ipv4 traffic, so ipv6 will remain
uncounted.
On Wed, Jun 7, 2017 at 9:52 AM, Lambert Hoogeveen <lambe...@clear.net.nz>
wrote:
> Another one to consider is sampling.
> If you're using this then that could be another reason for the disparity.
>
> Still, the answers from Brian and Adrian are the most likely cause though.
> The way traffic is counted is fundamentally different between Cacti and
> Netflow.
> Cacti reads the SNMP interface statistics (which includes ALL traffic
> down to Layer2), the Netflow collector looks at the exported IP Netflow
> packets.
> Taking all this into account, there are going to be differences between
> the two.
>
> Regards,
> Lambert
>
>
> On 07/06/17 01:07, Brian Candler wrote:
> > On 06/06/2017 13:25, nfsen-discuss-requ...@lists.sourceforge.net wrote:
> >> At this point, I got some divergence of data comparing
> >> Cacti/nfsen/nfdump,
> >> e.g:
> >>
> >> timeslot Jun 02 2017 - 11:55 - Jun 02 2017 - 15:50
> >> cacti: 650G
> >> nfsen: 617G
> >> nfdump: 575G
> >
> > nfsen just runs nfdump to show and aggregate flows. In fact, if you
> > look at the bottom section of the screen, it gives you the exact
> > nfdump command line it runs. So comparing the command line you ran
> > with the command line nfsen ran may help.
> >
> > If you're post-processing the output from nfdump, you might can use
> > the '-N' option to get figures printed as byte counts rather scaled to
> > different units.
> >
> > Actually, unit scaling appears to be one of your problems. Notice that
> > 575GiB (575x1024^3) is the same as 617GB (617*1000^3). Use numfmt
> > --to-si instead of --to-iec to make them match (*)
> >
> > As for the difference with Cacti: firstly, are you adding in and out
> > together? Otherwise you'll need to configure your nfdump queries with
> > filters to separate inbound and outbound traffic.
> >
> > Other differences can be due to nfcapd showing flows which don't pass
> > through the interface which Cacti is monitoring, or vice versa - this
> > isn't usually a problem if you're talking about a router interface
> > rather than a switch interface.
> >
> > It can also be to do with the handling of long-lived flows. For
> > example, there could be a long flow which was in progress at 15:50
> > (like a long download) which hadn't completed.
> >
> > You didn't say what router you're using, but if you configure it to
> > expire flows after 5 minutes, usually you'll find the flow data aligns
> > better with Cacti.
> >
> > Another possibility is to do with packet headers: that is, maybe your
> > Cacti interface counters are counting the full frames with ethernet
> > headers, and nfdump is just looking at the IP packets.
> >
> > Yet another possibility is non-IP traffic traversing the interface,
> > which Cacti will count but nfdump won't. (This includes ARP, along
> > with rarities like Netbeui, IPX and IS-IS). You might also find that
> > your router doesn't generate flows for broadcast traffic, but the
> > interface counters will count it.
> >
> > If you want to pin this down, set up a "quiet" test network, read the
> > interface counters with snmpwalk, send a known number of test packets
> > of known size, and check again. Compare with the flow data you receive.
> >
> > Regards,
> >
> > Brian.
> >
> > (*) It is arguable which is correct to use for this application.
> >
> > Communication systems always use power-of-ten units: e.g. 64Kbps is
> > 64,000 bits per second, and gigabit ethernet is 1,000,000,000 bits per
> > second. The nfdump manpage says it uses multiples of 1000.
> >
> > Computers traditionally use power-of-two units, especially for RAM.
> > However, hard drive manufacturers use power-of-ten units, since a
> > 500GB drive sounds better than 465GiB.
> >
> > So you have to be clear which you're using. If you are charging per
> > GB, tell your users whether this means 1000x1000x1000 bytes or
> > 1024x1024x1024 bytes.
> >
> >
> > ------------------------------------------------------------
> ------------------
> >
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Nfsen-discuss mailing list
> > Nfsen-discuss@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
> >
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Nfsen-discuss mailing list
> Nfsen-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
