Since most network devices have a single status register for both receiver and transmit (and errors and the like), which needs a lock to protect access, you will likely end up with serious thrashing of moving the lock between cpus. > Any ways to measure the trashing of locks?
Since most network devices have a single status register for both receiver and transmit (and errors and the like) > These register accesses will be mostly within the irq handler which I plan on keeping on the same processor. The network driver is actually tg3. Will looks closely into the driver. Thx, Venkat -----Original Message----- From: Lennart Sorensen [mailto:[EMAIL PROTECTED] Sent: Thursday, September 13, 2007 1:45 PM To: Venkat Subbiah Cc: Chris Snook; linux-kernel@vger.kernel.org Subject: Re: irq load balancing On Thu, Sep 13, 2007 at 01:31:39PM -0700, Venkat Subbiah wrote: > Doing it in a round-robin fashion will be disastrous for performance. > Your cache miss rate will go through the roof and you'll hit the slow > paths in the network stack most of the time. > > Most of the work in my system is spent in enrypt/decrypting traffic. > Right now all this is done in a tasklet within the softirqd and hence > all landing up on the same CPU. > On the receive side it'a packet handler that handles the traffic. On the > tx side it's done within the transmit path of the packet. So would > re-architecting this to move the rx packet handler to a different kernel > thread(with smp affinity to one CPU) and tx to a different kernel > thread(with SMP affinity to a different CPU) be advisable. > What's the impact on cache miss and slowpath/fastpath in network stack. Since most network devices have a single status register for both receiver and transmit (and errors and the like), which needs a lock to protect access, you will likely end up with serious thrashing of moving the lock between cpus. -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
