> I'm working on a proxy type program, using REDIRECT to catch (tcp) traffic,
> and I'm seeing severe network degradation above ~2000 connection.
>
> (computer: 1Gb p3, 2Gb memory, kernel 2.4.18 + aa1 patch)
Two questions first: how many packets/second is these 2000 connections?
And, do you have a feeling about how far the system would scale if it
were'nt for connection tracking.
Also, what is the total number of conntrack entries you have? Do this:
grep conntrack /proc/slabinfo
and show us the output line, please.
> I've profiled the kernel and found that > 50% of the cpu time is in
> __ip_conntrack_find
Ouch.
> - is there a patch to make connection tracking use a
> more scalable data structure (as I understand it uses a list), or to
> improve it's performance?
Please look a bit more careful at the code. While __ip_conntrack_find()
itself operates on a list, this is only the "inner loop" of a hash table
implementation. The setup is the usual "array of pointers to hash lists,
and hopefully they don't collide" setup. Nothing really wrong with that,
in principle, as far as data structures go.
The most likely cause for that 50% CPU usage, is that the hash table
is too small for your application. Contrary to ip_conntrack_max, the
total number of entries in the conntracking, the hash table size is
not modifiable at runtime. In ip_conntrack_core.c, that size is gotten
from the variable ip_conntrack_htable_size. This variable is computed
at boot / module load time, depending on the amount of RAM in your system.
IF you run ip_conntrack as a module, you can override the computed hash
table size by specifying "hashsize=XXX" as a module load parameter.
You can see the active value chosen for the hash table size, in syslog:
ip_conntrack (XXX buckets, YYY max)
The XXX is the number I'm talking about. Given ZZZ active conntrack entries
(as seen in /proc/slabinfo) you'll have, on average, a list size within
__ip_conntrack_find() of ZZZ/XXX.
The secondary reason for overly long lists, would be a bad hash function.
If you want to find out whether that could be the case, you could instrument
__ip_conntrack_find() to count the length of each list during traversal,
remember that somewhere, and occasionally printk() an average, minimum,
and maximum.
Hope this helps...
Patrick
