Hi Peter, I think I worked out what was going on here. Softflowd was configured to generate Netflow for both the Outside and Inside interface traffic flows and my test flows go through both.
It appears that softflowd is seeing the flows on both interfaces and sending it twice (there's a softwflowd process launched for each interface on the pfSense) and nfdump sees then as separate flows. Is that expected behaviour? See the first and last line of the nfdump snippet below. The timestamp is slightly different, as are the no. of packets and byte counts. 2015-11-26 10:07:57.093 295.378 UDP 172.22.37.250:32995 -> 128.18.1.1:5001 75353 112.9 M 1 2015-11-26 10:07:57.381 295.080 ICMP 128.18.1.250:0 -> 128.18.1.254:8.0 284 22720 1 2015-11-26 10:07:57.382 295.080 ICMP 128.18.1.254:0 -> 128.18.1.250:0.0 284 22720 1 2015-11-26 10:07:57.481 291.978 UDP 10.103.50.8:45588 -> 228.1.2.1:45588 298 65000 1 2015-11-26 10:07:58.050 289.956 OSPF 10.103.50.1:0 -> 224.0.0.5:0 60 3840 1 2015-11-26 10:07:59.573 287.863 UDP 10.103.50.7:45588 -> 228.1.2.1:45588 184 48182 1 2015-11-26 10:07:57.092 289.553 UDP 172.22.37.250:32995 -> 128.18.1.1:5001 73867 110.7 M 1 If I configure softflowd to only generate Netflow for the Inside interface then I get the correct counts. Regards, GB -----Original Message----- From: Peter Haag [mailto:ph...@users.sourceforge.net] Sent: 21 November 2015 12:15 To: Garrett Burke <gbu...@egenera.com>; nfdump-discuss@lists.sourceforge.net Subject: Re: [Nfdump-discuss] Double counting with pfsense and softflowd Hi Garrett, I'm not aware of a problem with softflowd. The records btw are around 7s apart - so it looks unlikely, that it is the same same flow. If you do not have nsel records you may use the std formats to display the records e.g. -o line If you can not get arount this, I need to check with pfsense. Cheers - Peter On 20.11.15 17:39, Garrett Burke wrote: > All, > > I'm using pfSense 2.2.4 with softflowd 1.2.1 exporting Netflow v5 packets to > nfsen with nfdump: Version: NSEL-NEL1.6.11 and I'm seeing double counting of > the bps. > > If I generate a 10Mbps flow through the pfSense firewall with iperf, it's > being displayed as 20Mbps. The pfSense counters show it correctly as 10Mbps. > > It looks like softflowd is sending the records twice, as I see the following > in the nfcapd files: > > # nfdump -r nfcapd.201511201555 > Date first seen Event XEvent Proto Src IP Addr:Port > Dst IP Addr:Port X-Src IP Addr:Port X-Dst IP Addr:Port In Byte > Out Byte > 2015-11-20 15:50:22.588 IGNORE Ignore UDP 172.22.37.250:55138 -> > 128.18.1.1:5001 0.0.0.0:0 -> 0.0.0.0:0 382.7 M > 0 > 2015-11-20 15:50:29.099 IGNORE Ignore UDP 172.22.37.250:55138 -> > 128.18.1.1:5001 0.0.0.0:0 -> 0.0.0.0:0 386.5 M > 0 > > Has anyone else seen this? > > Is there a way to get nfsen/nfdump to ignore the duplicates (if that is what > they are)? > > Thks, > GB > > -- > Garrett Burke > VP Engineering > Egenera Inc. | Converge. Unify. Simplify. > 00-353-1-9022868 (office) > > http://www.egenera.com > http://blog.egenera.com > http://www.facebook.com/#!/pages/Egenera/74312707811 > http://twitter.com/#!/Egenera > http://www.linkedin.com/company/7909?trk=tyah > > > > > ---------------------------------------------------------------------- > -------- > > > > _______________________________________________ > Nfdump-discuss mailing list > Nfdump-discuss@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > -- Be nice to your netflow data. Use NfSen and nfdump :) ------------------------------------------------------------------------------ Go from Idea to Many App Stores Faster with Intel(R) XDK Give your users amazing mobile app experiences with Intel(R) XDK. Use one codebase in this all-in-one HTML5 development environment. Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs. http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140 _______________________________________________ Nfdump-discuss mailing list Nfdump-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
