On Wed, 2002-04-17 at 21:16, Harald Welte wrote: > Have you tried without conntrack? I'm not so sure it really accounts > for most of the performance problems. Jamal Hadi Selmi and Robert Olsson > have made several performance analysis on linux routers. You can find > their test results in several papers published in conference proceedings. > > The major problem is that starting with a certain point, the kernel never > leaves the interrupt handler to process the network rx softirq. This is > a little bit better if you have a card (and driver) supporting the interrupt > mitigation scheme of 2.4.x kernels. > > You can easily have 300kpps routed (without conntrack) with their NAPI > changes which are in 2.5.x (patches for 2.4.x exist). NAPI switches the > network boards into polling mode as soon as a certain load is reached. > This means packets are really dropped on the network board and not within > the interrupt handler or the backlog queue.
I'm quite familiar with NAPI and the NAPI-enabled tulipdriver, have been bugtesting it before it went into 2.5 and using it in production routers for quite some time now. I've had quite alot of contact with Robert. And IIRC Robert has mailed results to this list some time ago pointing out that just loading ip_conntrack reduces the number of pps the router can process to about half. This is when routing a lot of small udp packets all matching the same conntrack entry. And in a quite recent test he's shown that simple routing (without any netfilter stuff) is slower on an SMP machine than on UP. The problem is with DDoS-attacks, conntrack falls down on it's knees even on UP. All those >100kpps creates a new connections in conntrack. The main problem seems to start when the conntrack table is full and it has to start looking for entries to evict. The router becomes _very_ unresponsive. All the inserts and deletes takes the global rwlock in write mode and then performance goes down the drain even if the other cpu wants to access another bucket which isn't affected by the insert/delete. > *sigh* I wish I once had a couple of wire-speed traffic generators and > a decent SMP x86 box to do some netfilter/conntrack benchmarking/profiling > and as a result do some performance optimizations. It would be enough > to be in a lab with reasonable equipment for one or two weeks.... I've routed over 120kpps while performing NAT on each packet on a dual pIII 600, it's been a while since I did that so I can't remember how much cpu it used or anything like that. just beeing able to route a lot of packets in a few streams is nice but it's not what's needed in reality, not in my reality anyway :) I need routers that can withstand some abuse without making an old 8086 cpu feel fast. Joakim Axelsson and me have been bugfixing and stresstesting the new superlimit match which uses another module Joakim wrote called hashslot which he's rewriting into hashslab and implementing real locking. If that works out well I might rewrite conntrack to use that. But first I have to go over all the locking in conntrack to see if it'll be possible to use hashslab. Otherwise I'll simply rewrite the locking to be per hashbucket. And I'll try to implement an eviction queue so we don't have to search a lot of lists to find entries to evict. We'll see how it works out. -- /Martin Never argue with an idiot. They drag you down to their level, then beat you with experience.
