Re: [WIP][PATCHES] Network xmit batching - tg3 support

2007-07-04 Thread jamal
On Wed, 2007-04-07 at 09:49 +0530, Krishna Kumar2 wrote: Do you see any contention for tx_lock which can justify having a prep handler? As I understood, no other CPU can be in the xmit code at the same time since RUNNING bit is held. Hence getting this lock early or late should not matter for

Re: [WIP][PATCHES] Network xmit batching - tg3 support

2007-07-03 Thread jamal
On Mon, 2007-02-07 at 17:21 -0700, Michael Chan wrote: [Matt, please include the count in the fix per previous email] Only base_flags and mss are needed and these can be determined right before sending the frame. So is it better not to store these in the skb-cb at all? long answer: My goal

Re: [WIP][PATCHES] Network xmit batching - tg3 support

2007-07-03 Thread David Miller
From: jamal [EMAIL PROTECTED] Date: Tue, 03 Jul 2007 09:09:48 -0400 [It sounds very dangerous to me the way skb-cb is being used by the vlan code (i.e requires human intervention/knowledge to catch it as an issue). I had no freaking idea the vlan code was using it. Maybe a huge comment

Re: [WIP][PATCHES] Network xmit batching - tg3 support

2007-07-03 Thread jamal
On Tue, 2007-03-07 at 12:31 -0700, Matt Carlson wrote: I had planned on using netperf, but pktgen sounds like a more controlled environment. Thanks for the tip. I can help more if you use pktgen - netperf is more involved. Also pktgen is much closer to the driver so it would let you see

Re: [WIP][PATCHES] Network xmit batching - tg3 support

2007-07-03 Thread Krishna Kumar2
Hi Jamal, J Hadi Salim [EMAIL PROTECTED] wrote on 07/03/2007 06:56:20 PM: On Mon, 2007-02-07 at 17:21 -0700, Michael Chan wrote: [Matt, please include the count in the fix per previous email] long answer: My goal with storing these values and computing them was to do certain things that

Re: [WIP][PATCHES] Network xmit batching - tg3 support

2007-07-02 Thread Michael Chan
On Mon, 2007-07-02 at 14:20 -0700, Matt Carlson wrote: Also, I think the count, max_per_txd, and nr_frags fields of the tg3_tx_cbdata struct are not needed. The count field is not needed also. +struct tg3_tx_cbdata { + u32 base_flags; + int count; + unsigned int

Re: FSCKED clock sources WAS(Re: [WIP][PATCHES] Network xmit batching

2007-06-25 Thread jamal
On Thu, 2007-21-06 at 20:45 +0400, Evgeniy Polyakov wrote: On Thu, Jun 21, 2007 at 11:54:17AM -0400, jamal ([EMAIL PROTECTED]) wrote: Evgeniy, did you sync on the batching case with the git tree? My tree contains following commits: Latest mainline commit:

Re: FSCKED clock sources WAS(Re: [WIP][PATCHES] Network xmit batching

2007-06-25 Thread jamal
On Thu, 2007-21-06 at 12:55 -0400, Benjamin LaHaise wrote: You should qualify that as 'Old P4 Xeon', as the Core 2 Xeons are leagues better. The Xeon hardware is not that old - about a year or so (and so is the opteron). BTW, how could you tell this was old Xeon? cheers, jamal - To

Re: FSCKED clock sources WAS(Re: [WIP][PATCHES] Network xmit batching

2007-06-25 Thread Benjamin LaHaise
On Mon, Jun 25, 2007 at 12:59:54PM -0400, jamal wrote: On Thu, 2007-21-06 at 12:55 -0400, Benjamin LaHaise wrote: You should qualify that as 'Old P4 Xeon', as the Core 2 Xeons are leagues better. The Xeon hardware is not that old - about a year or so (and so is the opteron). BTW, how

Re: [WIP][PATCHES] Network xmit batching

2007-06-25 Thread Rick Jones
Evgeniy Polyakov wrote: On Thu, Jun 21, 2007 at 02:00:07PM -0700, Rick Jones ([EMAIL PROTECTED]) wrote: Simple test included test - desktop and vice versa traffic with 128 and 4096 block size in netperf-2.4.3 setup. Is that in conjunction with setting the test-specific -D to set TCP_NODELAY,

Re: [WIP][PATCHES] Network xmit batching

2007-06-22 Thread Evgeniy Polyakov
On Thu, Jun 21, 2007 at 02:00:07PM -0700, Rick Jones ([EMAIL PROTECTED]) wrote: Simple test included test - desktop and vice versa traffic with 128 and 4096 block size in netperf-2.4.3 setup. Is that in conjunction with setting the test-specific -D to set TCP_NODELAY, or was Nagle left-on?

FSCKED clock sources WAS(Re: [WIP][PATCHES] Network xmit batching

2007-06-21 Thread jamal
On Tue, 2007-19-06 at 15:28 -0700, David Miller wrote: Converting pktgen over to ktime_t might be a nice cleanup. Would that really solve it? i.e doesnt it still tie to what the clock source is? I had a friend of mine (Robert, you know Jeremy) and results are slightly different from what

Re: FSCKED clock sources WAS(Re: [WIP][PATCHES] Network xmit batching

2007-06-21 Thread Evgeniy Polyakov
On Thu, Jun 21, 2007 at 11:54:17AM -0400, jamal ([EMAIL PROTECTED]) wrote: Evgeniy, did you sync on the batching case with the git tree? My tree contains following commits: Latest mainline commit: fa490cfd15d7ce0900097cc4e60cfd7a76381138 Latest batch commit:

Re: FSCKED clock sources WAS(Re: [WIP][PATCHES] Network xmit batching

2007-06-21 Thread Benjamin LaHaise
On Thu, Jun 21, 2007 at 12:08:19PM -0400, jamal wrote: The results in the table for opteron and xeon are swapped when cutnpasting from a larger test result. So Opteron is the one with better results. In any case - off for the day over here. You should qualify that as 'Old P4 Xeon', as the

Re: [WIP][PATCHES] Network xmit batching

2007-06-21 Thread Rick Jones
On Tue, 2007-06-19 at 17:21 +0400, Evgeniy Polyakov wrote: Hi. On Thu, Jun 07, 2007 at 07:43:49AM -0400, jamal ([EMAIL PROTECTED]) wrote: Folks, we need help. Please run this on different hardware. Evgeniy, i thought this kind of stuff excites you, no? ;- (wink, wink). Only the sender

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Evgeniy Polyakov
On Tue, Jun 19, 2007 at 05:21:48PM +0400, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: Simple test included test - desktop and vice versa traffic with 128 and 4096 block size in netperf-2.4.3 setup. I.e. it is not pktgen, but usual userspace application, which should not use new batch methods.

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Evgeniy Polyakov
Hi. On Thu, Jun 07, 2007 at 07:43:49AM -0400, jamal ([EMAIL PROTECTED]) wrote: Folks, we need help. Please run this on different hardware. Evgeniy, i thought this kind of stuff excites you, no? ;- (wink, wink). Only the sender needs the patch but the receiver must be a more powerful machine

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Evgeniy Polyakov
On Tue, Jun 19, 2007 at 06:00:39PM +0400, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: On Tue, Jun 19, 2007 at 05:33:33PM +0400, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: On Tue, Jun 19, 2007 at 05:21:48PM +0400, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: Simple test included test -

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread jamal
On Tue, 2007-19-06 at 17:21 +0400, Evgeniy Polyakov wrote: I've ran several simple tests with desktop e1000 adapter I managed to find. Mucho gracias Evgeniy. Test machine is amd athlon64 3500+ with 1gb of ram. Another point is dektop core duo 3.4 ghz with 2 gb of ram and sky2 driver.

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread jamal
On Tue, 2007-19-06 at 18:00 +0400, Evgeniy Polyakov wrote: pktgen reults are quite poor: batch (changed from default script: count reduced, clone increased to 10k) 241319pps 115Mb/sec mainline (the same script, on start it wrote about unsupported batch_low parameter: 497520pps 238Mb/sec

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Evgeniy Polyakov
On Tue, Jun 19, 2007 at 12:28:49PM -0400, jamal ([EMAIL PROTECTED]) wrote: On Tue, 2007-19-06 at 18:00 +0400, Evgeniy Polyakov wrote: pktgen reults are quite poor: batch (changed from default script: count reduced, clone increased to 10k) 241319pps 115Mb/sec mainline (the same

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Evgeniy Polyakov
On Tue, Jun 19, 2007 at 12:28:49PM -0400, jamal ([EMAIL PROTECTED]) wrote: In my case, I have: # CONFIG_TICK_ONESHOT is not set # CONFIG_NO_HZ is not set # CONFIG_HIGH_RES_TIMERS is not set $ cat ./.config | egrep CONFIG_TICK_ONESHOT|CONFIG_NO_HZ|CONFIG_HIGH_RES_TIMERS $ I will try to turn

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Evgeniy Polyakov
On Tue, Jun 19, 2007 at 12:32:48PM -0400, jamal ([EMAIL PROTECTED]) wrote: pktgen reults are quite poor: batch (changed from default script: count reduced, clone increased to 10k) 241319pps 115Mb/sec BTW, dont turn on the cloning - leave it as 1 so we can have every packet alloced so

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Robert Olsson
jamal writes: What is your kernel config in regards to HRES timers? Robert mentioned to me that the clock source maybe causing issues with pktgen (maybe even qos). Robert, insights? pktgen heavily uses gettimeofday. I was using tsc as clock source with our opterons in the lab. In late

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread jamal
On Tue, 2007-19-06 at 19:35 +0200, Robert Olsson wrote: But Evgeniy is most likely using the same clocksource both for the mainline and batch tests so there must be something different... Iam curious though which one, from my notes on my laptop for that test machine: # cat

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread Evgeniy Polyakov
On Tue, Jun 19, 2007 at 01:48:20PM -0400, jamal ([EMAIL PROTECTED]) wrote: On Tue, 2007-19-06 at 19:35 +0200, Robert Olsson wrote: But Evgeniy is most likely using the same clocksource both for the mainline and batch tests so there must be something different... Iam curious though

Re: [WIP][PATCHES] Network xmit batching

2007-06-19 Thread David Miller
From: Robert Olsson [EMAIL PROTECTED] Date: Tue, 19 Jun 2007 19:35:45 +0200 pktgen heavily uses gettimeofday. I was using tsc as clock source with our opterons in the lab. In late 2.6.20 gettimeofday was changed so tsc couldn't be used on opterons (pktgen at least). To give you an

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread Evgeniy Polyakov
On Thu, Jun 07, 2007 at 06:23:16PM -0400, jamal ([EMAIL PROTECTED]) wrote: On Thu, 2007-07-06 at 20:13 +0400, Evgeniy Polyakov wrote: Actually I wonder where the devil lives, but I do not see how that patchset can improve sending situation. Let me clarify: there are two possibilities to

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread jamal
KK, On Fri, 2007-08-06 at 10:36 +0530, Krishna Kumar2 wrote: I will try that. Also on the receiver, I am using unmodified 2.6.21 bits. That should be fine as long as the sender is running the patched 2.6.22-rc4 My earlier experiments showed that even small buffers were filling the E1000

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread Krishna Kumar2
Hi Jamal, J Hadi Salim [EMAIL PROTECTED] wrote on 06/08/2007 04:44:06 PM: That should be fine as long as the sender is running the patched 2.6.22-rc4 Definitely :) Thats interesting - it is possible there is transient burstiness which fills up the ring. My observation of your results

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread jamal
KK, On Fri, 2007-08-06 at 17:01 +0530, Krishna Kumar2 wrote: I thought it comes to 1.147Mpps, or did I calculate wrong (70*1024*1024/8/8) ? I assumed 8B to mean data that is on top of TCP/UDP? If so then in the case of UDP we have 8B UDP header, 20B IP and 14B ethernet 64B minimal allowed

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread jamal
On Fri, 2007-08-06 at 12:38 +0400, Evgeniy Polyakov wrote: On Thu, Jun 07, 2007 at 06:23:16PM -0400, jamal ([EMAIL PROTECTED]) wrote: I believe both are called with no lock. The idea is to avoid the lock entirely when unneeded. That code may end up finding that the packet [..] +

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread Evgeniy Polyakov
On Fri, Jun 08, 2007 at 07:31:07AM -0400, jamal ([EMAIL PROTECTED]) wrote: On Fri, 2007-08-06 at 12:38 +0400, Evgeniy Polyakov wrote: On Thu, Jun 07, 2007 at 06:23:16PM -0400, jamal ([EMAIL PROTECTED]) wrote: I believe both are called with no lock. The idea is to avoid the lock entirely

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread jamal
On Fri, 2007-08-06 at 16:09 +0400, Evgeniy Polyakov wrote: On Fri, Jun 08, 2007 at 07:31:07AM -0400, jamal ([EMAIL PROTECTED]) wrote: But lock is still being hold - or there was no intention to reduce lock usage? As far as I read Krishna's mail, lock usage was not an issue, so that hunk

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread Rick Jones
These results are based on the test script that I sent earlier today. I removed the results for UDP 32 procs 512 and 4096 buffer cases since the BW was coming line speed (infact it was showing 1500Mb/s and 4900Mb/s respectively for both the ORG and these bits). I expect UDP to overwhelm the

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread Rick Jones
Also note something else strange that it is kind of strange that something like UDP which doesnt backoff will send out less packets/second ;- Cannot explain that either :) Perhaps delays in restarting after the intra-stack flow control is asserted. One possible thing to do to try to deal

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread Evgeniy Polyakov
On Fri, Jun 08, 2007 at 09:07:47AM -0400, jamal ([EMAIL PROTECTED]) wrote: Something, that anyone can understand :) For example /proc stats, although it is not very accurate, but it is really usable parameter from userspace point ov view. which /proc stats? /proc/$pid/stat, for pktgen it

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread jamal
On Fri, 2007-08-06 at 10:27 -0700, Rick Jones wrote: [..] you cannot take the netperf service demand directly - each netperf is calculating assuming that it is the only thing running on the system. It then ass-u-me-s that the CPU util it measured was all for its work. This means the

Re: [WIP][PATCHES] Network xmit batching

2007-06-08 Thread Rick Jones
jamal wrote: On Fri, 2007-08-06 at 10:27 -0700, Rick Jones wrote: [..] you cannot take the netperf service demand directly - each netperf is calculating assuming that it is the only thing running on the system. It then ass-u-me-s that the CPU util it measured was all for its work. This

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread Krishna Kumar2
My somewhat confusing netperf script (to run on client) is attached below. Server just requires to run netserver. Client is not completely accurate since I am not using netperf4 (moving to that after some initial hiccups). thanks, - KK (See attached file: netperf.scp) J Hadi Salim [EMAIL

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread Krishna Kumar2
Hi Jamal, I ran these bits today and the results are included. For comparison, I am running 2.6.22-rc3 original bits. The systems are both 2.8Ghz, 2 cpu, P4, 2GB RAM, one E1000 82547GI card connected using a crossover cable. The test runs for 3 mins for each case. I have run only once instead of

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread jamal
On Thu, 2007-07-06 at 11:46 +0530, Krishna Kumar2 wrote: My somewhat confusing netperf script (to run on client) is attached below. Server just requires to run netserver. Client is not completely accurate since I am not using netperf4 (moving to that after some initial hiccups). Thanks KK.

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread Evgeniy Polyakov
On Thu, Jun 07, 2007 at 07:43:49AM -0400, jamal ([EMAIL PROTECTED]) wrote: Folks, we need help. Please run this on different hardware. Evgeniy, i thought this kind of stuff excites you, no? ;- (wink, wink). Only the sender needs the patch but the receiver must be a more powerful machine (so

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread jamal
On Thu, 2007-07-06 at 20:13 +0400, Evgeniy Polyakov wrote: Actually I wonder where the devil lives, but I do not see how that patchset can improve sending situation. Let me clarify: there are two possibilities to send data: 1. via batched sending, which runs via queue of packets and performs

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread jamal
On Wed, 2007-06-06 at 09:49 -0400, jamal wrote: If you help out, when you post your results, can you please say what hardware and setup was? Ok, i have tested on new hardware with pktgen: Machine: Dual Xeon processor with EMT64 - kernel is 32 bit. Chipset: E7520 Memory Controller Hub (MCH).

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread Krishna Kumar2
Hi Evgeniy, Let me clarify: there are two possibilities to send data: 1. via batched sending, which runs via queue of packets and performs prepare call (which only setups some private flags, no work with hardware) and then sending call. 2. old xmit function (which seems to be unused by

Re: [WIP][PATCHES] Network xmit batching

2007-06-07 Thread Krishna Kumar2
Hi Jamal, Would be nice to run three sets - but i think even one would be sufficiently revealing. I will try multiple runs over the weekend. During the week, the systems are used for other purposes too. I expect UDP to overwhelm the receiver. So the receiver needs a lot more tuning (like