-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
As just discussed together during the break, I'd happily will look into that.
- Peter
- -------- Original Message --------
From: Maurizio Molina <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Subject: Re:[Nfsen-discuss] How to configure sampling rate
Date: Fri Sep 22 2006 12:53:46 GMT+0200 (CEST)
> Peter Haag wrote:
>
>> Hi Maurizio,
>>
>> -------- Original Message --------
>> From: Maurizio Molina <[EMAIL PROTECTED]>
>> To: [email protected]
>> Subject: Re:[Nfsen-discuss] How to configure sampling rate
>> Date: Tue Sep 19 2006 14:36:33 GMT+0200 (CEST)
>>
>>> Peter Haag wrote:
>>>> -------- Original Message --------
>>>> From: Szymon Trocha <[EMAIL PROTECTED]>
>>>> To: [email protected]
>>>> Subject: [Nfsen-discuss] How to configure sampling rate
>>>> Date: Thu Aug 31 2006 13:16:06 GMT+0200 (CEST)
>>>>
>>>>> Hi all,
>>>>> I'm a new user of this application.
>>>>> I have two sources of NetFlow v5 using two different sampling rates
>>>>> each. How can I configure nfsen to display the exact and real
>> traffic or
>>>>> packet levels as it seems the graphs show much less traffic than it
>> goes
>>>>> through the routers (rather single Mbs then hundreds of Mbs).
>>>>> I'm using nfsen-1.2.4 and nfdump-1.5.2
>>>> Up to know, you can not configure sampling rates, however a simple
>>>> multiply may do the trick.
>>>> I can prepare a patch for 1.2.4. Stay tuned.
>>> I'd like to remark that the re-normalization of netflow data after
>>> sampling brings two issues:
>>> 1) multiplying the number of packets and bytes by the inverse of the
>>> sampling rate to obtain the real figure is an unbiased estimator (i.e.
>>> it is correct), while for the number of flows it is not. Therefore, if
>>> one chooses to give the re-normalized figure of the flows anyway, one
>>> should at least add a warning that the information is not
>>> representative of the real figure.
>>> 2) Also for packets and bytes, the re-normalized figure is affected by
>>> an uncertainty that should be reported along with the figure, not to
>>> give the user eccessive "trust" in potentially wrong figures. There are
>>> simple statistical formulas that can be applied and I'd be happy to
>>> discuss them in the list, if there is consensus that this can be helpful.
>>
>> Yes - I'm aware of that. I'd appreciate any links/pointers/formulas
>> regarding sampling. So far I did not find any useful information, why
>> I still hesitated to implement simple math.
>
> The theory we need to apply for assessing the precision of
> re-normalization of sampled data is indeed rather simple. Let me try to
> summarise it.
> - Let n be all the sampled packets you receive in a certain time
> interval T (e.g. T = 5min. = 300s) and from a certain set of sources
> (you then know that the total traffic was N=n/r, where r is your
> sampling rate).
> - Let h be a subset of the n (sampled) packets in which you're interest
> (e.g. they are the packets of a given profile you set).
> - You're interested in knowing H, the REAL number of packets of the
> subset before sampling.
> - Of course, you can estimate H as H'=N*(h/n) = N*p' ; p' is the
> "estimated proportion"
> - Now the problem is: how close is H' to the real H? (This is the same
> to say: how close is p' to the real proportion p=H/N)?
> - Let's put it slightly differently: immagine that we set a "precision
> target" for the relative error on p', e.g. we say that we want
> [|p'-p|/p] to be lower that "eps" with a certain confidence level
> "alpha". Common numbers can be: eps=0.05, alpha=0.95 (note: alpha and
> eps don't need to sum to 1, it's just for this example...).
> - Statistical theory says that the "precision target" is satisfied if p'
>> Z_p / (n + Z_p), where Z_p is a value that depends on alpha and eps.
> For example, if (eps=0.05 and alpha=0.95, Z_p=1536)
> That's all. Attached, a picture showing the relationship between the
> packet rate and the proportin that can be reliably estimated for this
> packet rate, assuming a time interval of 5 min, and Z_p=1536.
> Regards,
> Maurizio
>
> P.S. Z_p = [(Z_1-alpha/2)/eps]^2, where Z_1-alpha/2 can be get from
> statistical tables of the inverse of teh normal distribution
>
>> - Peter
>>
>>> Regards,
>>> Maurizio
>>>> - Peter
>>>>
>>>>> Thank you in advance and regards,
>>>> --
>>>> _______ SWITCH - The Swiss Education and Research Network ______
>>>> Peter Haag, Security Engineer, Member of SWITCH CERT
>>>> PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
>>>> SWITCH, Limmatquai 138, CH-8001 Zurich, Switzerland
>>>> E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/security
>>
>>> -------------------------------------------------------------------------
>>> Using Tomcat but need to do more? Need to support web services, security?
>>> Get stuff done quickly with pre-integrated technology to make your job
>>> easier
>>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>> _______________________________________________
>>> Nfsen-discuss mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>>
>>> -------------------------------------------------------------------------
>>> Take Surveys. Earn Cash. Influence the Future of IT
>>> Join SourceForge.net's Techsay panel and you'll get the chance to
>> share your
>>> opinions on IT & business topics through brief surveys -- and earn cash
>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>>> _______________________________________________
>>> Nfsen-discuss mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>>
>> --
>> _______ SWITCH - The Swiss Education and Research Network ______
>> Peter Haag, Security Engineer, Member of SWITCH CERT
>> PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
>> SWITCH, Limmatquai 138, CH-8001 Zurich, Switzerland
>> E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/security
>
>
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Nfsen-discuss mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
- --
_______ SWITCH - The Swiss Education and Research Network ______
Peter Haag, Security Engineer, Member of SWITCH CERT
PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
SWITCH, Limmatquai 138, CH-8001 Zurich, Switzerland
E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/security
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)
iQCVAwUBRRPf3f5AbZRALNr/AQIm3AQAmou3Ix+cOBd6kVSXJRkN6Fc27AdZK4KR
mEie0Z5zrX1JZPVve4kCorrb6PluJf06MpWcL2CBcZh+0qCg/6NPtej7ujg3dGrp
VEovRiV3gqv6tTxQUTHXhp6oUqIkqnNO6dp7/Md3/4K+q78EH58xRqvSOzBdkumD
nQeigNCnUUQ=
=0Q8o
-----END PGP SIGNATURE-----
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nfsen-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
