Re: [Devel] [PATCH rh7 00/19] netfilter/conntrack: use a single hashtable for all namespaces
Committed to 3.10.0-1127.8.2.vz7.151.4 kernel. -- Best regards, Konstantin Khorenko, Virtuozzo Linux Kernel Team On 05/22/2020 11:10 AM, Konstantin Khorenko wrote: There is some software which creates new net namespaces often, for example: - "very secure FTP daemon" (vsftpd) - be default create a new network namespace per connection. - "phpsessionclean.service" (ionclean) - native php mechanism for old sessions garbage collection, if "PrivateNetwork=true" config is set (also default). Currently a netns creation triggers new conntrack hashtable allocation (65K), which is heavy. The current patchset makes all namespaces to use a single hash for conntracks (and similarly nat bysrc hash). Testing results: // unfixed kernel # free; time for i in `seq 1 1000`; do ip netns add a$i; done; free totalusedfree shared buff/cache available Mem:8008232 328524 7456876 776 222832 7443024 Swap: 8127484 0 8127484 real0m32.014s user0m0.227s sys 0m30.453s totalusedfree shared buff/cache available Mem:8008232 1469060 6282088 776 257084 6287440 Swap: 8127484 0 8127484 creating 1000 netns via "ip" utility. It takes 32 sec and eats > 1Gb RAM. // the kernel with current patchset # free; time for i in `seq 1 1000`; do ip netns add a$i; done; free totalusedfree shared buff/cache available Mem:8008080 260508 6905880 744 841692 7507772 Swap: 8127484 0 8127484 real0m3.589s user0m0.180s sys 0m3.157s totalusedfree shared buff/cache available Mem:8008080 619232 6530296 744 858552 7140864 Swap: 8127484 0 8127484 3(!) sec and only 350 Mb used! Test was run on a VM (just rebooted) with 8Gb RAM, so the memory is not fragmented and not close to the limit. https://jira.sw.ru/browse/PSBM-103515 Andrey Ryabinin (1): ms/netfilter: nf_conntrack: Fix possible possible crash on module loading. Florian Westphal (14): ms/netfilter: conntrack: don't attempt to iterate over empty table ms/netfilter: conntrack: use nf_ct_key_equal() in more places ms/netfilter: conntrack: small refactoring of conntrack seq_printf ms/netfilter: conntrack: check netns when comparing conntrack objects ms/netfilter: conntrack: make netns address part of hash ms/netfilter: conntrack: use a single hashtable for all namespaces ms/netfilter: conntrack: consider ct netns in early_drop logic ms/netfilter: conntrack: check netns when walking expect hash ms/netfilter: conntrack: use get_random_once for nat and expectations ms/netfilter: conntrack: make netns address part of expect hash ms/netfilter: conntrack: use a single expectation table for all namespaces ms/netfilter: conntrack: make netns address part of nat bysrc hash ms/netfilter: conntrack: use a single nat bysource table for all namespaces ms/netfilter: conntrack: use single slab cache Konstantin Khorenko (1): revert RH patch 9837-net-netfilter-nf_conntrack-don-t-resize-NULL-or-free.patch Liping Zhang (1): ms/netfilter: conntrack: do not dump other netns's conntrack entries via proc Vasily Khoruzhick (1): ms/netfilter: conntrack: fix calculation of next bucket number in early_drop Zhang Chunyu (1): netfilter: xt_MARK: Add ARP support include/net/netfilter/nf_conntrack_core.h | 2 + include/net/netfilter/nf_conntrack_expect.h| 1 + include/net/netns/conntrack.h | 9 - include/net/netns/hash.h | 2 +- net/ipv4/netfilter/nf_conntrack_l3proto_ipv4.c | 2 +- .../netfilter/nf_conntrack_l3proto_ipv4_compat.c | 47 ++-- net/netfilter/nf_conntrack_core.c | 255 ++--- net/netfilter/nf_conntrack_expect.c| 83 --- net/netfilter/nf_conntrack_helper.c| 8 +- net/netfilter/nf_conntrack_netlink.c | 29 ++- net/netfilter/nf_conntrack_standalone.c| 16 +- net/netfilter/nf_nat_core.c| 44 ++-- net/netfilter/nfnetlink_cttimeout.c| 6 +- net/netfilter/xt_mark.c| 1 + 14 files changed, 265 insertions(+), 240 deletions(-) ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
[Devel] [PATCH rh7 00/19] netfilter/conntrack: use a single hashtable for all namespaces
There is some software which creates new net namespaces often, for example: - "very secure FTP daemon" (vsftpd) - be default create a new network namespace per connection. - "phpsessionclean.service" (ionclean) - native php mechanism for old sessions garbage collection, if "PrivateNetwork=true" config is set (also default). Currently a netns creation triggers new conntrack hashtable allocation (65K), which is heavy. The current patchset makes all namespaces to use a single hash for conntracks (and similarly nat bysrc hash). Testing results: // unfixed kernel # free; time for i in `seq 1 1000`; do ip netns add a$i; done; free totalusedfree shared buff/cache available Mem:8008232 328524 7456876 776 222832 7443024 Swap: 8127484 0 8127484 real0m32.014s user0m0.227s sys 0m30.453s totalusedfree shared buff/cache available Mem:8008232 1469060 6282088 776 257084 6287440 Swap: 8127484 0 8127484 creating 1000 netns via "ip" utility. It takes 32 sec and eats > 1Gb RAM. // the kernel with current patchset # free; time for i in `seq 1 1000`; do ip netns add a$i; done; free totalusedfree shared buff/cache available Mem:8008080 260508 6905880 744 841692 7507772 Swap: 8127484 0 8127484 real0m3.589s user0m0.180s sys 0m3.157s totalusedfree shared buff/cache available Mem:8008080 619232 6530296 744 858552 7140864 Swap: 8127484 0 8127484 3(!) sec and only 350 Mb used! Test was run on a VM (just rebooted) with 8Gb RAM, so the memory is not fragmented and not close to the limit. https://jira.sw.ru/browse/PSBM-103515 Andrey Ryabinin (1): ms/netfilter: nf_conntrack: Fix possible possible crash on module loading. Florian Westphal (14): ms/netfilter: conntrack: don't attempt to iterate over empty table ms/netfilter: conntrack: use nf_ct_key_equal() in more places ms/netfilter: conntrack: small refactoring of conntrack seq_printf ms/netfilter: conntrack: check netns when comparing conntrack objects ms/netfilter: conntrack: make netns address part of hash ms/netfilter: conntrack: use a single hashtable for all namespaces ms/netfilter: conntrack: consider ct netns in early_drop logic ms/netfilter: conntrack: check netns when walking expect hash ms/netfilter: conntrack: use get_random_once for nat and expectations ms/netfilter: conntrack: make netns address part of expect hash ms/netfilter: conntrack: use a single expectation table for all namespaces ms/netfilter: conntrack: make netns address part of nat bysrc hash ms/netfilter: conntrack: use a single nat bysource table for all namespaces ms/netfilter: conntrack: use single slab cache Konstantin Khorenko (1): revert RH patch 9837-net-netfilter-nf_conntrack-don-t-resize-NULL-or-free.patch Liping Zhang (1): ms/netfilter: conntrack: do not dump other netns's conntrack entries via proc Vasily Khoruzhick (1): ms/netfilter: conntrack: fix calculation of next bucket number in early_drop Zhang Chunyu (1): netfilter: xt_MARK: Add ARP support include/net/netfilter/nf_conntrack_core.h | 2 + include/net/netfilter/nf_conntrack_expect.h| 1 + include/net/netns/conntrack.h | 9 - include/net/netns/hash.h | 2 +- net/ipv4/netfilter/nf_conntrack_l3proto_ipv4.c | 2 +- .../netfilter/nf_conntrack_l3proto_ipv4_compat.c | 47 ++-- net/netfilter/nf_conntrack_core.c | 255 ++--- net/netfilter/nf_conntrack_expect.c| 83 --- net/netfilter/nf_conntrack_helper.c| 8 +- net/netfilter/nf_conntrack_netlink.c | 29 ++- net/netfilter/nf_conntrack_standalone.c| 16 +- net/netfilter/nf_nat_core.c| 44 ++-- net/netfilter/nfnetlink_cttimeout.c| 6 +- net/netfilter/xt_mark.c| 1 + 14 files changed, 265 insertions(+), 240 deletions(-) -- 2.15.1 ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
