Min Miles Xu wrote:
> Hi Darren,
> I'm from Solaris NIC team. Some questions you raise is also what I
> concerned before.
>
> Darren Reed wrote:
>> Over time there has been an increase in the number
>> of features that NICs provide, to the point where
>> we are at now with some providing full TOEs. So far
>> there has been little in the way of knobs to turn
>> things on or off with, aside from do_hwcksum.
>>
> Currently TOEs such as HW checksum are toggled in the .conf file of
Are we talking about stateful TOEs ("TCP Offload Engines"), which
have their own TCP/IP stack, and bypass the Solaris TCP/IP stack
entirely? Or are we talking about stateless offloads, like
hardware checksum offload, TCP Segmentation Offload (LSO),
and TCP Receive Offload (LRO)? When I think "TOE" I think "stateful"
> drivers or /etc/system. /etc/system seems to be a more consistent
> interface since the name of TOEs in different drivers varies a lot. It's
> natural to think about a program to provide a unique interface of
> configurations such as dladm. And we already have Project Brussels. The
> actual barrier I notice is that GLD interface _m_getcapab() is not
> flexible. The driver can't update its TOEs capability interactively
> through _m_getcapab() at runtime (For e1000g LSO, it has practical
> usage). Only thing you can do after changing the configurations is to
> reboot the machine. So if we can have some changes in the stack,
> consolidating the configurations to _m_getprop(), it's better.
Well, you can edit the driver's .conf file and rmdrv/add_drv to
re-load the driver, assuming you have a connection into the
machine independent of the NIC in question.
But in general, I agree there should be some way to change what
stateless offloads are in use at runtime. Solaris is way behind
here. For example, the BSDs do it via ifconfig (to disable
TSO on "mxge0": ifconfig mxge0 -tso), Linux does it via the
horribly cryptic ethtool (to disable TSO on eth2:
ethtool -K eth2 tso off).
>> Something that might provide food for thought about
>> what sort of knobs we do need is how well those TOEs
>> play with other network components. For example, if
>> there is an interoperability problem with the TOE on
>> a NIC and some HBA, how do we manage that? How do we
>> disable use of the TOE on a card until a firmware
>> upgrade is available for it?
>>
>> And what sort of extra observability does that NIC
>> need in order to allow diagnosing problems with traffic
>> flow between it and the HBA?
>>
> If the issue can be isolated to the NIC only, we usually dump the NIC
> descriptors and the buffers with mdb to watch what's going on. But it
> requires knowledge about the specific NIC. Not sure what's the problem
> you meet.
>> If snoop and /dev/net are providing us with a view of
>> traffic that is between IP and the driver, how do we
>> get a look at what the NIC is really sending?
>>
> Well, without the help of special devices to observe the internal states
> of the NIC chip, you can still observe what the NIC is actually sending
> by monitoring the other end of the network. Sure, you should ensure that
> NIC on the other end is good in advance.
And make sure it is not using some offloads of its own (like LRO) which
can influence what the traffic coming through snoop looks like.
Drew
_______________________________________________
networking-discuss mailing list
[email protected]
