Hi Luca,
our tests are trying to put as much stress as we can on nprobe from a
flows-per-second point of view, and since we don't have dedicated HW
generators we generate 1 packet flows, TCP, of the smallest size
possible. The flows/s (and pk/s as well, since flows are 1 pkts long
flows) are in these tests 100K/s. There are six virtual machines
generating this traffic, so 6 different source IPs. Each of them chooses
randomly a destination port Xi (i=1,....,6), then generates 60,000
packets incrementing each time deterministically the destination port by
one (Di, Di+1, Di+2, ...), then the loop repeats itself with every VM
incrementing deterministically Xi by one.
So, at the end, two nested loops on the src and dst ports. This means
actually low "entropy" (a lot of subsequent pkt headers will differ by
one bit only) but this is similar to what happens e.g. in a port
scanning. A "good" hash should distribute randomly also with 1 bit
variations, but I know this is difficult. What hash is nprobe using, BTW?
Here below, the probability of long lists that we expect with a
perfectly uniform hash distribution for that load, hash size of 2^23,
flows kept in the flow cache for 11 seconds (10 sec idle tout and 1 sec
scan cycle). Bucket sizes of 16 elements are clearly indicating an
uneven distribution of the hash.
Cheers,
Maurizio
K P(list>k)
0 1.00E+00
1 1.16E-01
2 7.54E-03
3 3.30E-04
4 1.08E-05
5 2.83E-07
6 6.19E-09
7 1.16E-10
8 1.90E-12
9 2.77E-14
10 3.63E-16
11 4.33E-18
12 4.73E-20
13 4.77E-22
14 4.47E-24
15 3.91E-26
16 3.20E-28
17 2.47E-30
18 1.80E-32
19 1.24E-34
20 8.15E-37
On 19.04.2011 18:07, Luca Deri wrote:
Daniel
it looks your traffic is not easy balance-able then. What kind of packets are
you using?
Luca
On Apr 19, 2011, at 5:53 PM, Daniel Aschwanden wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi Luca,
On 04/19/2011 05:33 PM, Luca Deri wrote:
Daniel
if you have 100k concurrent flows, -w 256000 will do the job
Luca
Unfortunately not (-w 262144):
19/Apr/2011 17:40:39 [nprobe.c:1459] ---------------------------------
19/Apr/2011 17:40:39 [nprobe.c:1460] Average traffic: [105.431 K pps][68
Mb/sec]
19/Apr/2011 17:40:39 [nprobe.c:1465] Current traffic: [142.662 K pps][88
Mb/sec]
19/Apr/2011 17:40:39 [nprobe.c:1471] Current flow export rate: [12653.5
flows/sec]
19/Apr/2011 17:40:39 [nprobe.c:1474] Flow drops: [export queue too
long=0][too many flows=0]
19/Apr/2011 17:40:39 [nprobe.c:1480] Flow Buckets:
[active=532231][allocated=532231][toBeExported=0][frags=0]
19/Apr/2011 17:40:39 [nprobe.c:1516] Processed packets: 10332302 (max
bucket search: 14)
19/Apr/2011 17:40:39 [nprobe.c:1519] Flow Collection: [collected pkts:
0][processed flows: 0]
19/Apr/2011 17:40:39 [nprobe.c:222] Packet stats: 10332424/0 pkts
rcvd/dropped [0.0%] [Last 285324/0 pkts rcvd/dropped]
19/Apr/2011 17:40:42 [nprobe.c:1459] ---------------------------------
19/Apr/2011 17:40:42 [nprobe.c:1460] Average traffic: [105.521 K pps][68
Mb/sec]
19/Apr/2011 17:40:42 [nprobe.c:1465] Current traffic: [108.445 K pps][67
Mb/sec]
19/Apr/2011 17:40:42 [nprobe.c:1471] Current flow export rate: [29935.3
flows/sec]
19/Apr/2011 17:40:42 [nprobe.c:1474] Flow drops: [export queue too
long=0][too many flows=0]
19/Apr/2011 17:40:42 [nprobe.c:1480] Flow Buckets:
[active=605659][allocated=606136][toBeExported=477][frags=0]
19/Apr/2011 17:40:42 [nprobe.c:1516] Processed packets: 10657626 (max
bucket search: 16)
19/Apr/2011 17:40:42 [nprobe.c:1519] Flow Collection: [collected pkts:
0][processed flows: 0]
19/Apr/2011 17:40:42 [nprobe.c:222] Packet stats: 10657745/0 pkts
rcvd/dropped [0.0%] [Last 325321/0 pkts rcvd/dropped]
I know in theory the probability of occurance of list size> 16 should
be quite small.
I'm even more confused about that a obviously too big hash (2^23)
doesn't decrease the list size.
Any suggestions?
Thanks a lot!
Cheers, Dani
- --
daniel aschwanden
junior engineer
open systems ag
raeffelstrasse 29
ch-8045 zurich
t: +41 44 455 74 00
f: +41 44 455 74 01
[email protected]
http://www.open.ch
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iF4EAREIAAYFAk2tsBMACgkQxIzmH53Qg6HBtgD+Jxvl+v+YGAuwFgnq0b9fmsjR
Ve9Pa7yiAlaRO568asABAJTRmorB8vIggS/VqZ/JfUs255AZdMOHKmWEQDd9txPA
=rX7B
-----END PGP SIGNATURE-----
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
---
If you can not measure it, you can not improve it - Lord Kelvin
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
--
Maurizio Molina
Product Manager
open systems ag
raeffelstrasse 29
ch-8045 zurich
t: +41 44 455 74 00
f: +41 44 455 74 01
[email protected]
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
