On 2013-01-07, at 2:03 AM, Adrian Popa <adrian.popa...@gmail.com> wrote:
> If you are worried instead about the low volume of traffic seen from this AS, > keep in mind the following: > 1. You are probably using sampling on your router. NFSEN accounts for > sampling and tries to guesstimate some of the values. I am sampling. 1:1000. Maybe I don't quite understand sampling. Sampling doesn't quelch the number of flow records exported to the collector, it quelches the number of packets that are processed by the device in order to create the flow record. Is that accurate? So I just re-ran the math from the output below. Let's take this one for argument's sake: 2013-01-03 10:10:43.424 0.016 any 30513 2( 0.0) 2000( 0.0) 3.0 M( 0.0) 125000 1.5 G 1500 So what that is saying is that the statistic entry for AS30513 was first seen on 2013-01-03 10:10:43.424, consists of 16ms worth of data where 2 flows totalling 3MB of data volume spread across 2000 packets was collected within those 16ms. The flow records have no knowledge of pps, bps or bpp, so nfdump calculates those values based on the data that it knows about; time (16ms), volume (3MB) and total number of packets based on the exported flow records received by nfcapd. So if this is true, then trying to use bps as a statistic orderby will never provide you with decent results because those values are calculated based on data that might have been quelched based on the way the sampling works. If this is correct, it seems to me like sampling is bad (but I can't actually not sample or else my routers drop netflow packets; they can only handle 100k across the entire box), but I understand why it exists. So if sampling is the root cause of all these "bad" calculations, it would stand to reason that one should set the sampling rate as close to 1:1 as possible? > 2. You may have some spoofed traffic in your network that sends few packets > (hence the very short duration), but because of sampling, you get a high > count of packets (and usually this is a "round" number). > > On Sat, Jan 5, 2013 at 9:44 AM, Peter Haag <ph...@users.sourceforge.net> > wrote: > Hi Jason, > Looking at your output, I can not find something weird. Please keep in mind: > Each flow has two ASes, so and so see on how many flows these ASes appear. > Your second example makes it clear: You filter for 'as 30513' which results > in two flows - AS 30513 <-> AS 0. AS 0 means the exporting router has no AS > info. These resulting two flows are now ordered by AS and by bps as requested. > Each AS appears in each flow -> in 100% of all flows. > > The same math is now applied for your first run. But you only have the flows > of the first top 10 ASes by bps. In % the digits are way below what can be > displayed. You may also use -N to prevent scaling (K, M, G, T) in order to > see the actual number. To sum up, you would need to output of all seen ASes > -n 0 . > > Hope, this helps, otherwise let me know, if I can help > > - Peter > > On 4/1/13 5:20 PM, Jason Lixfeld wrote: > > Hi there, > > > > So I'm just playing around with my first 36 hours worth of data and I'm > > seeing some stuff that looks sort of off: > > > > ** nfdump -M > > /opt/nfsen/profiles-data/live/bfr01-hudson:bfr01-mowat:bfr01-front -T -R > > 2013/01/02/nfcapd.201301022305:2013/01/04/nfcapd.201301041055 -n 10 -s > > as/bps > > nfdump filter: > > any > > Top 10 AS ordered by bps: > > Date first seen Duration Proto AS Flows(%) > > Packets(%) Bytes(%) pps bps bpp > > 2013-01-02 22:39:46.290 130797.681 any 0 21.1 M(85.9) > > 42.2 G(87.5) 30.0 T(88.5) 322585 1.8 G 710 > > 2013-01-03 10:10:43.424 0.016 any 30513 2( 0.0) > > 2000( 0.0) 3.0 M( 0.0) 125000 1.5 G 1500 > > 2013-01-03 08:53:20.734 0.015 any 37957 2( 0.0) > > 2000( 0.0) 1.5 M( 0.0) 133333 810.7 M 760 > > 2013-01-04 10:23:02.606 0.017 any 35414 2( 0.0) > > 2000( 0.0) 1.5 M( 0.0) 117647 727.5 M 773 > > 2013-01-03 14:25:51.067 0.017 any 33428 2( 0.0) > > 2000( 0.0) 1.5 M( 0.0) 117647 692.7 M 736 > > 2013-01-03 13:37:35.176 0.039 any 46676 1( 0.0) > > 2000( 0.0) 2.8 M( 0.0) 51282 582.6 M 1420 > > 2013-01-04 00:43:04.529 0.048 any 15347 1( 0.0) > > 2000( 0.0) 2.8 M( 0.0) 41666 473.3 M 1420 > > 2013-01-03 15:58:33.535 0.077 any 47045 1( 0.0) > > 3000( 0.0) 4.3 M( 0.0) 38961 442.6 M 1420 > > 2013-01-02 23:02:16.952 129445.016 any 22822 4.0 M(16.2) > > 8.9 G(18.5) 6.4 T(19.0) 68835 398.2 M 723 > > 2013-01-03 14:52:54.865 0.031 any 19354 2( 0.0) > > 2000( 0.0) 1.5 M( 0.0) 64516 379.9 M 736 > > > > Summary: total flows: 24583165, total bytes: 33.9 T, total packets: 48.2 G, > > avg bps: 2.1 G, avg pps: 368688, avg bpp: 702 > > Time window: 2013-01-02 22:39:34 - 2013-01-04 10:59:43 > > Total flows processed: 24583165, Blocks skipped: 0, Bytes read: 2261849088 > > Sys: 8.970s flows/second: 2740403.8 Wall: 10.563s flows/second: 2327242.5 > > > > Lines 1 and 9 seem OK, but lines 2-8,10 look really weird; the math just > > doesn't add up. > > > > If I filter specifically on AS 30513: > > > > ** nfdump -M > > /opt/nfsen/profiles-data/live/bfr01-hudson:bfr01-mowat:bfr01-front -T -R > > 2013/01/02/nfcapd.201301022305:2013/01/04/nfcapd.201301041055 -n 10 -s > > as/bps > > nfdump filter: > > AS 30513 > > Top 10 AS ordered by bps: > > Date first seen Duration Proto AS Flows(%) > > Packets(%) Bytes(%) pps bps bpp > > 2013-01-03 10:10:43.424 0.016 any 0 2(100.0) > > 2000(100.0) 3.0 M(100.0) 125000 1.5 G 1500 > > 2013-01-03 10:10:43.424 0.016 any 30513 2(100.0) > > 2000(100.0) 3.0 M(100.0) 125000 1.5 G 1500 > > > > Summary: total flows: 2, total bytes: 3.0 M, total packets: 2000, avg bps: > > 1.5 G, avg pps: 125000, avg bpp: 1500 > > Time window: 2013-01-02 22:39:34 - 2013-01-04 10:59:43 > > Total flows processed: 24583165, Blocks skipped: 0, Bytes read: 2261849088 > > Sys: 7.574s flows/second: 3245367.9 Wall: 8.594s flows/second: 2860278.3 > > > > I have no idea how to even begin going about troubleshooting this, so any > > thoughts are welcomed. > > > > Thanks again in advance. > > ------------------------------------------------------------------------------ > > Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and > > much more. Get web development skills now with LearnDevNow - > > 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. > > SALE $99.99 this month only -- learn more at: > > http://p.sf.net/sfu/learnmore_122812 > > _______________________________________________ > > Nfsen-discuss mailing list > > Nfsen-discuss@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/nfsen-discuss > > > > -- > Be nice to your netflow data. Use NfSen and nfdump :) > > ------------------------------------------------------------------------------ > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft > MVPs and experts. SALE $99.99 this month only -- learn more at: > http://p.sf.net/sfu/learnmore_122912 > _______________________________________________ > Nfsen-discuss mailing list > Nfsen-discuss@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfsen-discuss > ------------------------------------------------------------------------------ Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 _______________________________________________ Nfsen-discuss mailing list Nfsen-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
