Peter Haag wrote:
> Hi Maurizio,
>
> -------- Original Message --------
> From: Maurizio Molina <[EMAIL PROTECTED]>
> To: [email protected]
> Subject: Re:[Nfsen-discuss] How to configure sampling rate
> Date: Tue Sep 19 2006 14:36:33 GMT+0200 (CEST)
>
> >Peter Haag wrote:
>
> >>-------- Original Message --------
> >>From: Szymon Trocha <[EMAIL PROTECTED]>
> >>To: [email protected]
> >>Subject: [Nfsen-discuss] How to configure sampling rate
> >>Date: Thu Aug 31 2006 13:16:06 GMT+0200 (CEST)
> >>
> >>>Hi all,
> >>>I'm a new user of this application.
> >>>I have two sources of NetFlow v5 using two different sampling rates
> >>>each. How can I configure nfsen to display the exact and real
> traffic or
> >>>packet levels as it seems the graphs show much less traffic than it
> goes
> >>>through the routers (rather single Mbs then hundreds of Mbs).
> >>>I'm using nfsen-1.2.4 and nfdump-1.5.2
> >>
> >>Up to know, you can not configure sampling rates, however a simple
> >>multiply may do the trick.
> >>I can prepare a patch for 1.2.4. Stay tuned.
>
> >I'd like to remark that the re-normalization of netflow data after
> >sampling brings two issues:
> >1) multiplying the number of packets and bytes by the inverse of the
> >sampling rate to obtain the real figure is an unbiased estimator (i.e.
> >it is correct), while for the number of flows it is not. Therefore, if
> >one chooses to give the re-normalized figure of the flows anyway, one
> >should at least add a warning that the information is not
> >representative of the real figure.
> >2) Also for packets and bytes, the re-normalized figure is affected by
> >an uncertainty that should be reported along with the figure, not to
> >give the user eccessive "trust" in potentially wrong figures. There are
> >simple statistical formulas that can be applied and I'd be happy to
> >discuss them in the list, if there is consensus that this can be helpful.
>
>
> Yes - I'm aware of that. I'd appreciate any links/pointers/formulas
> regarding sampling. So far I did not find any useful information, why
> I still hesitated to implement simple math.
The theory we need to apply for assessing the precision of
re-normalization of sampled data is indeed rather simple. Let me try to
summarise it.
- Let n be all the sampled packets you receive in a certain time
interval T (e.g. T = 5min. = 300s) and from a certain set of sources
(you then know that the total traffic was N=n/r, where r is your
sampling rate).
- Let h be a subset of the n (sampled) packets in which you're interest
(e.g. they are the packets of a given profile you set).
- You're interested in knowing H, the REAL number of packets of the
subset before sampling.
- Of course, you can estimate H as H'=N*(h/n) = N*p' ; p' is the
"estimated proportion"
- Now the problem is: how close is H' to the real H? (This is the same
to say: how close is p' to the real proportion p=H/N)?
- Let's put it slightly differently: immagine that we set a "precision
target" for the relative error on p', e.g. we say that we want
[|p'-p|/p] to be lower that "eps" with a certain confidence level
"alpha". Common numbers can be: eps=0.05, alpha=0.95 (note: alpha and
eps don't need to sum to 1, it's just for this example...).
- Statistical theory says that the "precision target" is satisfied if p'
> Z_p / (n + Z_p), where Z_p is a value that depends on alpha and eps.
For example, if (eps=0.05 and alpha=0.95, Z_p=1536)
That's all. Attached, a picture showing the relationship between the
packet rate and the proportin that can be reliably estimated for this
packet rate, assuming a time interval of 5 min, and Z_p=1536.
Regards,
Maurizio
P.S. Z_p = [(Z_1-alpha/2)/eps]^2, where Z_1-alpha/2 can be get from
statistical tables of the inverse of teh normal distribution
>
> - Peter
>
> >Regards,
> >Maurizio
>
> >> - Peter
> >>
> >>>Thank you in advance and regards,
> >>
> >>--
> >>_______ SWITCH - The Swiss Education and Research Network ______
> >>Peter Haag, Security Engineer, Member of SWITCH CERT
> >>PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
> >>SWITCH, Limmatquai 138, CH-8001 Zurich, Switzerland
> >>E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/security
>
>
> >-------------------------------------------------------------------------
> >Using Tomcat but need to do more? Need to support web services, security?
> >Get stuff done quickly with pre-integrated technology to make your job
> >easier
> >Download IBM WebSphere Application Server v.1.0.1 based on Apache
> Geronimo
> >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >_______________________________________________
> >Nfsen-discuss mailing list
> >[email protected]
> >https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
>
> >-------------------------------------------------------------------------
> >Take Surveys. Earn Cash. Influence the Future of IT
> >Join SourceForge.net's Techsay panel and you'll get the chance to
> share your
> >opinions on IT & business topics through brief surveys -- and earn cash
> >http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >_______________________________________________
> >Nfsen-discuss mailing list
> >[email protected]
> >https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
>
> --
> _______ SWITCH - The Swiss Education and Research Network ______
> Peter Haag, Security Engineer, Member of SWITCH CERT
> PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
> SWITCH, Limmatquai 138, CH-8001 Zurich, Switzerland
> E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/security
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nfsen-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
