Re: [ofa-general] [PATCH 13 of 17]: add LRO support
> Can you clarify if/how this patch is related to the "lro: Generic Large > Receive Offload for TCP traffic" RFC sent on August this year to netdev > (eg see http://lwn.net/Articles/244206) ? I referred to mtnic driver when I made this patch which referred to other code examples, possibly from this one too. > > Assuming LRO is a --pure software-- optimization, what's the rational to > put its whole implementation in the ipoib driver and not divide it to > general part implemented in the net core and per driver part implemented > per device driver that wants to support LRO (if such second part is > needed at all)? It is a pure software optimization but it relies on the HW to report whether the checksum of the packet is valid or not in order for it to be liable for aggregation. I think it would be good however if the kernel would support this and take this from the specific drivers. > > If I am wrong and their is some LRO assistance from the connectX HW, > what is it doing? > > Or. > > ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH 13 of 17]: add LRO support
Eli Cohen wrote: Since you have posted the patch, I am asking you if it has any negative influence on packet forwarding. I am not asking you to test it or whether you tested it with forwarding. The answer is yes since I do not recalculate TCP checksum as I aggregate the SKBs so the kernel might forward the TCP segment as multiple IP packets but with wrong TCP checksum (which is that of the first aggregated packet) but not of the overall aggregated segment. OK, thanks for this clarification. Can you clarify if/how this patch is related to the "lro: Generic Large Receive Offload for TCP traffic" RFC sent on August this year to netdev (eg see http://lwn.net/Articles/244206) ? Assuming LRO is a --pure software-- optimization, what's the rational to put its whole implementation in the ipoib driver and not divide it to general part implemented in the net core and per driver part implemented per device driver that wants to support LRO (if such second part is needed at all)? If I am wrong and their is some LRO assistance from the connectX HW, what is it doing? Or. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH 13 of 17]: add LRO support
> Since you have posted the patch, I am asking you if it has any negative > influence on packet forwarding. > > I am not asking you to test it or whether you tested it with forwarding. > The answer is yes since I do not recalculate TCP checksum as I aggregate the SKBs so the kernel might forward the TCP segment as multiple IP packets but with wrong TCP checksum (which is that of the first aggregated packet) but not of the overall aggregated segment. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH 13 of 17]: add LRO support
Eli Cohen wrote: My question is, does the suggested LRO code need to be disabled when the node does forwarding? Thinking about this I probably have to add the means to disable LRO for those hosts which so IP forwarding. I will send a modified patch. why, can you explain what is the problem with doing LRO with forwarding? Or. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH 13 of 17]: add LRO support
Eli Cohen wrote: My question is, does the suggested LRO code need to be disabled when the node does forwarding? I did not test such a setup with a host operating as a router between ipoib and Ethernet networks. Once I do this I will evaluate if there is a problem and possibly add facilities to disable LRO (probably via ethtool). Since you have posted the patch, I am asking you if it has any negative influence on packet forwarding. I am not asking you to test it or whether you tested it with forwarding. Or. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH 13 of 17]: add LRO support
> > > > My question is, does the suggested LRO code need to be disabled when the > > node does forwarding? > > > I did not test such a setup with a host operating as a router between > ipoib and Ethernet networks. Once I do this I will evaluate if there is > a problem and possibly add facilities to disable LRO (probably via > ethtool). > Thinking about this I probably have to add the means to disable LRO for those hosts which so IP forwarding. I will send a modified patch. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH 13 of 17]: add LRO support
> Hi Eli, > > Back on April a user having the configuration > > A --- 10g --- B --- IB --- C > > where node B acts as an IP router having one 10g interface > and one IB interface reported on a sever bandwidth problem > which was resolved to be related to the 10g driver have LRO mechanism > which is not operative under forwarding scheme, see the email/thread > http://lists.openfabrics.org/pipermail/general/2007-April/035322.html > > My question is, does the suggested LRO code need to be disabled when the > node does forwarding? > I did not test such a setup with a host operating as a router between ipoib and Ethernet networks. Once I do this I will evaluate if there is a problem and possibly add facilities to disable LRO (probably via ethtool). > Indeed you have removed the LSO, LRO patches from the stateless offload > patch set posting to the upstream kernel, but they do exist in OFED 1.3 > > Tziporet - I am quite worried from distributing with OFED 1.3 ipoib > changes (namely LSO and LRO support) which were never reviewed by the > community and that I understand are not planned for review towards 2.6.24. > > What is your thinking on the matter? > > Or. > > > -Original Message- > > From: [EMAIL PROTECTED] On Behalf Of David Miller > > Sent: Saturday, April 28, 2007 2:40 AM > > To: [EMAIL PROTECTED] > > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; > > [email protected] > > Subject: Re: [ofa-general] Re: IPoIB forwarding > > > > From: Rick Jones <[EMAIL PROTECTED]> > > Date: Fri, 27 Apr 2007 16:37:49 -0700 > > > >> Large Receive Offload (LRO) is enabled by default. This will > >> interfere with forwarding TCP traffic. If you plan to forward TCP > >> traffic (using the host with the Myri10GE NIC as a router or bridge), > >> you must disable LRO. To disable LRO, load the myri10ge driver with > >> myri10ge_lro set to 0: > > > > LRO should be disabled by default if the driver does this. This is a major > > and unacceptable bug. > > > > Thanks for pointing this out Rick. > > > ___ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH 13 of 17]: add LRO support
Eli Cohen wrote: Add Large Receive Offload support to IPOIB Reduce overhead incurred by handling many small packets by aggregating SKBs related to the same stream and passing them up. This patch is based on the work done for MTNIC by Liran Liss <[EMAIL PROTECTED]> Hi Eli, Back on April a user having the configuration A --- 10g --- B --- IB --- C where node B acts as an IP router having one 10g interface and one IB interface reported on a sever bandwidth problem which was resolved to be related to the 10g driver have LRO mechanism which is not operative under forwarding scheme, see the email/thread http://lists.openfabrics.org/pipermail/general/2007-April/035322.html My question is, does the suggested LRO code need to be disabled when the node does forwarding? Indeed you have removed the LSO, LRO patches from the stateless offload patch set posting to the upstream kernel, but they do exist in OFED 1.3 Tziporet - I am quite worried from distributing with OFED 1.3 ipoib changes (namely LSO and LRO support) which were never reviewed by the community and that I understand are not planned for review towards 2.6.24. What is your thinking on the matter? Or. -Original Message- From: [EMAIL PROTECTED] On Behalf Of David Miller Sent: Saturday, April 28, 2007 2:40 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [email protected] Subject: Re: [ofa-general] Re: IPoIB forwarding From: Rick Jones <[EMAIL PROTECTED]> Date: Fri, 27 Apr 2007 16:37:49 -0700 Large Receive Offload (LRO) is enabled by default. This will interfere with forwarding TCP traffic. If you plan to forward TCP traffic (using the host with the Myri10GE NIC as a router or bridge), you must disable LRO. To disable LRO, load the myri10ge driver with myri10ge_lro set to 0: LRO should be disabled by default if the driver does this. This is a major and unacceptable bug. Thanks for pointing this out Rick. ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
