Re: [ofa-general] Re: IPoIB forwarding

2007-05-03 Thread Bryan Lawver
I have been able to install and use the 1.3.0 myricom driver and everything works as I expected and performance is pretty decent. Interesting little side tour through various drivers...The router node sees almost no load which is really encouraging. Thanks, bryan At 03:05 PM 5/1/2007, Loic

Re: [ofa-general] Re: IPoIB forwarding

2007-05-01 Thread Loic Prylli
On 5/1/2007 1:57 AM, Bill Fink wrote: On Mon, 30 Apr 2007, Rick Jones wrote: Ethtool -i on the interface reports 1.2.0 as the driver version. Perhaps it would be useful to have different version strings for the in-kernel Linux version and the Myricom externally provided version.

Re: [ofa-general] Re: IPoIB forwarding

2007-05-01 Thread Loic Prylli
On 4/30/2007 2:12 PM, Rick Jones wrote: Speaking of defaults, it would seem that the external 1.2.0 driver comes with 9000 bytes as the default MTU? At least I think that is what I am seeing now that I've started looking more closely. rick jones That's the same for the in-kernel-tree

Re: [ofa-general] Re: IPoIB forwarding

2007-05-01 Thread Rick Jones
Loic Prylli wrote: On 4/30/2007 2:12 PM, Rick Jones wrote: Speaking of defaults, it would seem that the external 1.2.0 driver comes with 9000 bytes as the default MTU? At least I think that is what I am seeing now that I've started looking more closely. rick jones That's the same for

Re: [ofa-general] Re: IPoIB forwarding

2007-04-30 Thread Rick Jones
What version of the myri10ge driver is this? With the 1.2.0 version that comes with the 2.6.20.7 kernel, there is no myri10ge_lro module parameter. [EMAIL PROTECTED] ~]# modinfo myri10ge | grep -i lro [EMAIL PROTECTED] ~]# And I've been testing IP forwarding using two Myricom 10-GigE NICs

Re: [ofa-general] Re: IPoIB forwarding

2007-04-30 Thread Rick Jones
David Miller wrote: From: Rick Jones [EMAIL PROTECTED] Date: Fri, 27 Apr 2007 16:48:00 -0700 No problem - just to play whatif/devil's advocate for a bit though... is there any way to tie that in with the setting of net.ipv4.ip_forward (and/or its IPv6 counterpart)? Even ignoring that,

Re: [ofa-general] Re: IPoIB forwarding

2007-04-30 Thread Rick Jones
Only the 1.2.0 version of the external driver makes LRO incompatible with forwarding. The problem should be fixed in version 1.3.0 released a few weeks ago (forwarding with myri10ge_lro enabled should then work), let us know otherwise. Anyway, following David Miller remark about netfilter,

Re: [ofa-general] Re: IPoIB forwarding

2007-04-30 Thread Bill Fink
On Mon, 30 Apr 2007, Rick Jones wrote: What version of the myri10ge driver is this? With the 1.2.0 version that comes with the 2.6.20.7 kernel, there is no myri10ge_lro module parameter. [EMAIL PROTECTED] ~]# modinfo myri10ge | grep -i lro [EMAIL PROTECTED] ~]# And I've been

Re: [ofa-general] Re: IPoIB forwarding

2007-04-29 Thread Loic Prylli
On 4/28/2007 2:51 AM, Bill Fink wrote: On Fri, 27 Apr 2007, Rick Jones wrote: Bryan Lawver wrote: I had so much debugging turned on that it was not the slowing of the traffic but the non-coelescencing that was the remedy. The NIC is a MyriCom NIC and these are easy options to set.

Re: [ofa-general] Re: IPoIB forwarding

2007-04-28 Thread Bill Fink
On Fri, 27 Apr 2007, Rick Jones wrote: Bryan Lawver wrote: I had so much debugging turned on that it was not the slowing of the traffic but the non-coelescencing that was the remedy. The NIC is a MyriCom NIC and these are easy options to set. As chance would have it, I've played with

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread Rick Jones
Bryan Lawver wrote: Your right about the ipoib module not combining packets (I believed you without checking) but I did never the less. The ipoib_start_xmit routine is definitely handed a double packet which means that the IP NIC driver or the kernel is combining two packets into a single

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread Rick Jones
Bryan Lawver wrote: I hit the IP NIC over the head with a hammer and turned off all offload features and I no longer get the super jumbo packet and I have symmetric performance. This NIC supported ethtool -K ethx tso/tx/rx/sg on/off and I am not sure at this time which one I needed to whack

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread Bryan Lawver
I hit the IP NIC over the head with a hammer and turned off all offload features and I no longer get the super jumbo packet and I have symmetric performance. This NIC supported ethtool -K ethx tso/tx/rx/sg on/off and I am not sure at this time which one I needed to whack but all off solved

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread Bryan Lawver
I had so much debugging turned on that it was not the slowing of the traffic but the non-coelescencing that was the remedy. The NIC is a MyriCom NIC and these are easy options to set. At 03:32 PM 4/27/2007, Rick Jones wrote: Bryan Lawver wrote: I hit the IP NIC over the head with a hammer

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread Rick Jones
Bryan Lawver wrote: I had so much debugging turned on that it was not the slowing of the traffic but the non-coelescencing that was the remedy. The NIC is a MyriCom NIC and these are easy options to set. As chance would have it, I've played with some Myricom myri10ge NICs recently, and even

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread David Miller
From: Rick Jones [EMAIL PROTECTED] Date: Fri, 27 Apr 2007 16:37:49 -0700 Large Receive Offload (LRO) is enabled by default. This will interfere with forwarding TCP traffic. If you plan to forward TCP traffic (using the host with the Myri10GE NIC as a router or bridge), you must disable LRO.

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread Rick Jones
David Miller wrote: From: Rick Jones [EMAIL PROTECTED] Date: Fri, 27 Apr 2007 16:37:49 -0700 Large Receive Offload (LRO) is enabled by default. This will interfere with forwarding TCP traffic. If you plan to forward TCP traffic (using the host with the Myri10GE NIC as a router or bridge),

Re: [ofa-general] Re: IPoIB forwarding

2007-04-27 Thread David Miller
From: Rick Jones [EMAIL PROTECTED] Date: Fri, 27 Apr 2007 16:48:00 -0700 No problem - just to play whatif/devil's advocate for a bit though... is there any way to tie that in with the setting of net.ipv4.ip_forward (and/or its IPv6 counterpart)? Even ignoring that, consider the potential