To seperate into N cmaps, why not use hash value divided by N? Simon Horman <ho...@ovn.org> 于2024年5月1日周三 19:06写道：
> On Wed, Apr 24, 2024 at 02:44:54PM +0200, Felix Huettner via dev wrote: > > Currently conntrack uses a single large cmap for all connections stored. > > This cmap contains all connections for all conntrack zones which are > > completely separate from each other. By separating each zone to its own > > cmap we can significantly optimize the performance when using multiple > > zones. > > > > The change fixes a similar issue as [1] where slow conntrack zone flush > > operations significantly slow down OVN router failover. The difference is > > just that this fix is used whith dpdk, while [1] was when using the ovs > > kernel module. > > > > As we now need to store more cmap's the memory usage of struct conntrack > > increases by 524280 bytes. Additionally we need 65535 cmaps with 128 > > bytes each. This leads to a total memory increase of around 10MB. > > > > Running "./ovstest test-conntrack benchmark 4 33554432 32 1" shows no > > real difference in the multithreading behaviour against a single zone. > > > > Running the new "./ovstest test-conntrack benchmark-zones" show > > significant speedups as shown below. The values for "ct execute" are for > > acting on the complete zone with all its entries in total (so in the > > first case adding 10,000 new conntrack entries). All tests are run 1000 > > times. > > > > When running with 1,000 zones with 10,000 entries each we see the > > following results (all in microseconds): > > "./ovstest test-conntrack benchmark-zones 10000 1000 1000" > > > > +------+--------+---------+---------+ > > | Min | Max | 95%ile | Avg | > > +------------------------+------+--------+---------+---------+ > > | ct execute (commit) | | | | | > > | with commit | 2266 | 3505 | 2707.06 | 2592.06 | > > | without commit | 2411 | 12730 | 4432.50 | 2736.78 | > > +------------------------+------+--------+---------+---------+ > > | ct execute (no commit) | | | | | > > | with commit | 699 | 1238 | 886.15 | 722.67 | > > | without commit | 700 | 3377 | 1934.42 | 803.53 | > > +------------------------+------+--------+---------+---------+ > > | flush full zone | | | | | > > | with commit | 619 | 1122 | 901.36 | 679.15 | > > | without commit | 618 | 105078 | 64591 | 2886.46 | > > +------------------------+------+--------+---------+---------+ > > | flush empty zone | | | | | > > | with commit | 0 | 5 | 1.00 | 0.64 | > > | without commit | 54 | 87469 | 64520 | 2172.25 | > > +------------------------+------+--------+---------+---------+ > > > > When running with 10,000 zones with 1,000 entries each we see the > > following results (all in microseconds): > > "./ovstest test-conntrack benchmark-zones 1000 10000 1000" > > > > +------+--------+---------+---------+ > > | Min | Max | 95%ile | Avg | > > +------------------------+------+--------+---------+---------+ > > | ct execute (commit) | | | | | > > | with commit | 215 | 287 | 231.88 | 222.30 | > > | without commit | 214 | 1692 | 569.18 | 285.83 | > > +------------------------+------+--------+---------+---------+ > > | ct execute (no commit) | | | | | > > | with commit | 68 | 97 | 74.69 | 70.09 | > > | without commit | 68 | 300 | 158.40 | 82.06 | > > +------------------------+------+--------+---------+---------+ > > | flush full zone | | | | | > > | with commit | 47 | 211 | 56.34 | 50.34 | > > | without commit | 48 | 96330 | 63392 | 63923 | > > +------------------------+------+--------+---------+---------+ > > | flush empty zone | | | | | > > | with commit | 0 | 1 | 1.00 | 0.44 | > > | without commit | 3 | 109728 | 63923 | 3629.44 | > > +------------------------+------+--------+---------+---------+ > > > > Comparing the averages we see: > > * a moderate performance improvement for conntrack_execute with or > > without commiting of around 6% to 23% > > * a significant performance improvement for flushing a full zone of > > around 75% to 99% > > * an even more significant improvement for flushing empty zones since we > > no longer need to check any unrelated connections > > Very nice numbers indeed. > > > [1] 9ec849e8aa869b646c372fac552ae2609a4b5f66 > > > > Signed-off-by: Felix Huettner <felix.huettner@mail.schwarz> > > Acked-by: Simon Horman <ho...@ovn.org> > > ... > > _______________________________________________ > dev mailing list > d...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > -- hepeng _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
