Shane, I'm not too familiar with flow-rpt2rrd, so I can't comment.
>Does flowscan suffer from the same "data degradation"? I wouldn't call it degradation, but yes, anything that uses RRDTool is subject to it's consolidation. If you're after pretty graphs, flowscan is the way to go, and your coding effort will be between minimal and none. 95th percentile stats based on RRDTool in any forma are difficult due to it's consolidation. There are 95th percentile scripts about, but they are fundamentally flawed due to the way RRDTool works, unless you store large amounts of data at short intervals, which kind of defeats the purpose of using RRDTool in the first place. If you want 95th percentile stats, I suggest compiling flow-tools with MySQL support and using flow-export f3 to load up a MySQL database and get the 95th information using a suitably crafted SQL statement. I've done something similar and it works a treat. Databases tend to grow fairly large, fairly quickly, though. Regards, Leigh. ----- Original Message ----- From: "Shane Dawalt" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Tuesday, September 20, 2005 11:36 PM Subject: Re: [Flow-tools] rrd and netflow > Leigh, > > I'm using the flow-rpt2rrd translation script. (Admittantly, I had to > translate flow-rpt2rrd to perl because RRD required python > 2.3.4 and > 2.3.5 won't install on my AMD64 box due to some odd RPM problem.) I've > extracted dumps of the data going to the rrdtool API and it is correct, > so I'm confident my perl version is working. > > I was aware RRD performed creative math on the data ... but I wasn't > aware that it didn't store the actual information. I thought the math > was done when the data was fetched from the database. > > But in looking at the flowscan link you sent (and some documents > thereon), I notice one perplexing issue; it too uses RRD. Does flowscan > suffer from the same "data degradation"? > > Shane > > Leigh Sharpe wrote: > > >How are you getting your data from flow files into RRDs? > >Have you looked at flowscan? (http://net.doit.wisc.edu/~plonka/FlowScan/) > > > >Remember, you won't get the same absolute values out of RRDTool as you put > >in. When you enter an absolute value in bytes, RRDTool stores the bytes/sec > >over that time, not the actual byte count. Therefore, storing the bps > >figures in an RRD is pretty much useless. > >Looking at your packet count, however, reveals the following: > >From your second sample.: > > > > > > > >>TAGSTRING,99402240,150869,313287.741105,22.944365,52452000.000000 > >> > >> > >In 5 minutes (300 secs), there were 150869 packets. > >150869/300=502.9 > >From your RRD: packets=5.0290042074e+02 > >ie 502.9 packets/sec. > > > >Looks like you might be on the right track, but you need to read up on how > >RRDTool works. > > > > > > > > > >----- Original Message ----- > >From: "Shane Dawalt" <[EMAIL PROTECTED]> > >To: <[email protected]> > >Sent: Tuesday, September 20, 2005 6:27 AM > >Subject: [Flow-tools] rrd and netflow > > > > > > > > > >> So I've been working towards a way to store flow-report data for > >>5-minute flow files. RRD seemed the best way. However, after going > >>through the pain on my AMD64 box, I've arrived at a rather odd problem. > >>Shown below are the first two records for 5-minute flows. The first > >>group is from flow-report. The second group is from fetching data from > >>the rrd database. > >> > >> From flow-report: > >># tag: TAGSTRING-TAGS > >># first-flow 1127145749 Mon Sep 19 12:02:29 2005 > >># last-flow 1127146045 Mon Sep 19 12:07:25 2005 > >># recn: source-tag*,octets,packets,avg-bps,min-bps,max-bps > >>TAGSTRING,125699150,192982,354640.397566,22.630000,57176000.000000 > >># first-flow 1127146044 Mon Sep 19 12:07:24 2005 > >># last-flow 1127146344 Mon Sep 19 12:12:24 2005 > >>TAGSTRING,99402240,150869,313287.741105,22.944365,52452000.000000 > >> > >> From "rrdtool fetch <rrdfile> AVERAGE --start '12:00 09/19/05' --end > >>'start +1 hour'": > >> > >>timestamp octets min-bps avg-bps > >>packets max-bps > >> > >>1127145600: 3.2446881292e+05 6.6090286387e-02 9.7143092605e+02 > >>5.9940418858e+02 1.0812780420e+05 > >>1127145900: 2.2599923079e+05 6.5043735241e-02 9.6301899034e+02 > >>5.0290042074e+02 7.1764395351e+04 > >> > >>The numbers from rrdtool aren't even close to those from flow-report. I > >>cannot believe they are correct ... but I've never played with rrdtool > >>before so I might be entering the fetch request wrong. (But if they are > >>correct then what are they telling me?) I suspect rrdtool may have a > >>problem running in 64-bit, but I don't have a 32-bit box to try it on. > >>Anyone care to comment? > >> > >> Shane > >> > >>_______________________________________________ > >>Flow-tools mailing list > >>[EMAIL PROTECTED] > >>http://mailman.splintered.net/mailman/listinfo/flow-tools > >> > >> > >> > > > > > > > > _______________________________________________ > Flow-tools mailing list > [EMAIL PROTECTED] > http://mailman.splintered.net/mailman/listinfo/flow-tools > _______________________________________________ Flow-tools mailing list [EMAIL PROTECTED] http://mailman.splintered.net/mailman/listinfo/flow-tools
