Sunay,
Having seen you earlier e-mail and now this one. I think
an appropriate way to simplify this is have the mac layer
provide two things:
1) Provide true soft LSO in the mac layer where the H/W driver
would receive multiple packets to map/unmap and transmit.
The mac layer can be a client of the library I propose below in #2.
2) Provide a set of mac library routines that the driver could use
to do the packet chopping that soft LSO requires while the
H/W performs the single DMA mapping. Also, the H/W driver
would advertize that it is LSO capable.
The end result is that you would have the best of both worlds
in having the mac layer provide LSO for all devices and
specific drivers provide it and offering one dma mapping.
Sound reasonable?
Michael
On Sep 25, 2007, at 2:09 PM, Sunay Tripathi wrote:
> Matheos Worku wrote:
>> Garrett D'Amore wrote:
>>> Paul Durrant wrote:
>>>
>>>> On 25/09/2007, David Edmondson <dme at sun.com> wrote:
>>>>
>>>>
>>>>> Why? I can accept a large packet in a MAC layer transmit function,
>>>>> chop it up and then present an mblk chain to the driver in a
>>>>> single
>>>>> call to its' transmit routine.
>>>>>
>>>>>
>>>>>
>>>> You lose the single DMA map this way, which is a good portion of
>>>> the
>>>> advantage of LSO (given how slow the DMA map operation appears
>>>> to be
>>>> on many platforms).
>>>>
>>>>
>>>
>>> Right. If you push it too far up the stack, then it offers no real
>>> advantage over just letting IP fragment the frames normally. :-)
>>>
>>> The meaty part of this work can be in common code that is called
>>> by mac
>>> layer drivers, though.
>>>
>>
>> I understand the obvious advantages of doing soft-lso in MAC layer
>> such
>> as same code supporting all non-hw-lso NICs,
>> designing/implementing/testing it once etc ...
>> On the other hand, doing it in the NIC driver has some advantages
>> due to
>> the fact it has intimate knowledge of the HW among other things.
>> This
>> is from the experience we gained from implementing soft-lso in
>> Neptune
>> Linux driver.
>>
>> 1) Reduced DMA mapping. Assuming that the LSO packet is presented
>> as a
>> chain of large mblks (or pages), need to do DMA mapping per large
>> mblk.
>> You can then chop the the mapped data buffer. Even in Linux X86
>> where
>> a page contains 2+ MTU sized packets this would cut down the
>> number of
>> DMA mappings significantly. For SPARC, the reduction would be
>> even bigger.
>>
>> 2) Manufacture the headers in the driver. Could exploit specific HW
>> features.
>>
>> 3) Separate the operations which require holding locks from those
>> don't.
>> Manufacturing the headers, DMA mapping, calculating pseudo
>> checksum and
>> IP header checksum (for the HW which do not support complete
>> checksum),
>> twiking IP id and sequence numbers can be done lockless. Only the
>> actual posting of the descriptors would require locking.
>>
>>
>> 4) Simpler Interface. As far as the upper layer is concerned, there
>> would be no distinction between HW and SW LSO. The driver can
>> advertise
>> as LSO capable. At the driver level, the difference would be
>> how LSO
>> packet is processed/described/posted to the HW and that is HW
>> specific
>> anyway.
>
> I think you missed my earlier reply. Yes, pushing it down for driver
> that support it is good but the functionality does need to be
> offered at a common layer. Don't want to see code in TCP or
> IP dealing with LSO or not to LSO. They should just do it based
> on advertized MTU. Here is the relevant part from my earlier email
>
> "
> The MAC layer is implementing a pseudo H/W layer anyway as part of
> Crossbow. The idea is that upper layer can always assume that
> features like classificaton, Rx/Tx rings, soft-lso, etc are
> supported. The upper layer always have a pointer to a Tx
> routine in MAC layer. The driver on the other hand can choose
> to be native LSO, soft-lso aware or none. It needs to register
> its capability as per the new document Kais and Roamer are
> preparing and corresponding functions. In most cases, the MAC
> transmit routine can just be a tail call to driver routine or
> a tail call to mac_soft_lso() in case driver doesn't support
> it. Some advantages of a function pointer driven approach ....
> "
>
> Cheers,
> Sunay
>
> --
> Sunay Tripathi
> Distinguished Engineer
> Solaris Core Operating System
> Sun MicroSystems Inc.
>
> Solaris Networking: http://www.opensolaris.org/os/community/
> networking
> Project Crossbow: http://www.opensolaris.org/os/project/crossbow
>
>
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
