Peter Memishian wrote:
Again, I'm the last person to fight against code cleanup, but it's not
quite as cut-and-dried as you're making it to be. BTW, most of the points
you make are also in strong support of making GLDv3 public, which I am
quite in support of.
> 1) these drivers have a lot of cut-n-paste code... e.g. DLPI
> handling in hme, eri, probably contribute over 5K lines _each_. Most of
> that code is also duplicated in both GLDv2 and in nemo. More lines of
> code == more chances for bugs == more support headaches.
Yes, we're well aware of the swill. If you look closely, you'll also find
all sorts of one-off bugfixes in various drivers for various customer
escalations -- we need to be careful not to regress any of those -- and
when necessary to pull those fixes into the framework or into other
drivers. (As an example of these sort of oddities, check out
eri_mk_mblk_tail_space(), which has now spread to some other drivers.)
That particular hack seems to address a problem in the ndd support.
Yech. If this driver were GLDv3, then when it makes the move to
Brussels, this hack could go away!
> 2) nemo-ification automatically gets quite a bit of "free"
> performance in fewer CPU cycles used (direct function calls, etc.)
Quite possibly, but the lion's share of these are not high-performance
drivers.
Even on low performance drivers, per-packet overheads can be
significant. Not everyone has multigigahertz cpus to allocate to each
NIC. There are still a lot of ~500MHz-ish systems out there, and a lot
of them have several NICs or at least several NIC ports (e.g. a qfe, etc.)
With link aggregation, even these "low performance" NICs are still quite
useful on higher performance systems.
> 3) it gets us one step closer to being able to eliminate legacy
> APIs... possibly making the _frameworks_ simpler. (For example, some
> Consolidation Private and Contracted Private interfaces for things like
> HW checksum offload, DLPI fastpath, etc. can pretty much get eradicated
> once the last Sun consumers of them go away.)
It doesn't really matter how much closer we get unless we can deal with
the third-party driver problem (and some third parties won't even consider
a GLDv3 conversion until they don't have to support S8/S9).
How many 3rd party drivers are using some of the ugly internals like
DL_CAPABILITY_REQ (which is not a public interface), or DL_CONTROL_REQ,
for example? What about DL_IOC_HDR_INFO? Or DLIOCNATIVE? (Okay, I
know a few use DL_IOC_HDR_INFO... but they _shouldn't_.)
These Sun-private DLPI extensions can simply go away if all the
consumers of them go away. For some of these cases, we have 100%
control over that at Sun.
> 4) centralizing functionality for stuff like DLPI handling means
> reduces long term support costs for drivers like eri, hme, etc.
Yes, this is why we did GLD in the first place.
> 5) unifies administrative tasks... e.g. these drivers can adopt
> things like Brussels, will "just work" with dladm (they don't today!) etc.
It will "just work" with Clearview UV too. Actually, all of them will
just work, no porting required.
Does this mean that Clearview is going to know about all the special
case stats that each of these drivers exports? (I.e. is there going to
be a switch table entry in Clearview for hme, qfe, gem, etc.?) And, at
what level are we going to have put brains in clearview to do things
like "ndd" for these drivers? Yech.
> 6) ultimately leading us one step closer towards nixing "ndd" (see
> point #3 above, again) (Also removing duplicated kstat and ndd code in
> _these_ drivers.)
See third party driver problem again.
The NDD driver ioctl interfaces have never been published, and aren't
part of the DDI. I hope there aren't 3rd party drivers that are relying
on that particular interface. (To my knowledge, my own "afe" and "mxfe"
drivers are the only unbundled drivers that support "ndd", and that is
by "circumstance"... I provide my own tools which use an IOCTL set that
happens to be "compatible" with ndd.
Once afe is integrated, I'll make "ndd" the official way to tune,
instead of providing my own "etherdiag" command.
The point is, if its a Sun-internal-only interface, then it should be
possible for us to fix it.
> 7) paves the way for these drivers to support additional crossbow
> features like Multiple Unicast Address support, interrupt blanking
> (which may or may not be useful with 100Mb links... but imagine an older
> system like an E450 with a few dozen 100Mbit links on it...)
We get this with Clearview UV.
The only way Clearview can do multiaddress mode is by putting the
interface in promiscuous mode. This causes all kinds of performance
penalty in the underlying drivers. (From the fact that the underlying
drivers perform an extra "dupmsg" to the fact that you take and process
_every_ packet on the wire.) I sincerely hope you're not suggesting
that this is equivalent to having the native ability in the driver to
put multiple unicast addresses in the device's mac address hardware filter.
> 8) as another item on #2, nemoification gets free
> multi-data-transmit (and receive!), yielding better performance as well.
Likewise.
This doesn't get you multidata transmit and receive _at the driver
level_. It just makes the stack think that you're getting it. Sure,
its better than nothing, but its also not as good as having that all the
way down at the driver level. (E.g. the driver isn't doing multiple
putnext() calls, or multiple gld_recv() calls. Likewise for tx...)
> 9) ultimately, also eradicating special case code in places like
> SunVTS and other test suites, that have special cases for devices like
> hme, gem, etc. (e.g. using custom ioctls to set loopback modes.)
That seems orthogonal to the GLDv3 work. GLDv3 does not specify the
loopback testing model AFAIK.
Its coming with brussels, I think. That may mean its only available to
GLDv3 drivers.
> 10) making these drivers DLPI style 1, bringing us much closer to
> removing the long standing DLPI style 2 "bug".
Again, this is part of Clearview UV. All drivers will have DLPI style 1
nodes in /dev/net. No porting necessary.
Fair enough. But will there also be /dev/hme nodes? Will clearview
make tools like snoop automatically use the style 1 nodes? Will there
be a /dev/net/hme0 if I have not set up a vanity name?
For GLDv3 I get /dev/hme0 by default. This also makes it
easier/automatic for sysadmins _today_, without having to retrain myself
to use "clearview safe" names later.
> Finally, for most of these legacy drivers, the effort to convert is
> quite small. On the order of a man day. Seriously. With all these
> benefits at such a low cost, why _wouldn't_ we do it?
The cost is not just (or even primarily) in the porting effort. It's in
the regression testing, the code churn, and the possible bugs that fallout
of the work. Again, there's nothing I like better than clean code. But
given that Clearview UV will benefit all non-GLDv3 drivers without porting
them and also allow a number of framework simplifications, and that
third-party drivers will stand in the way of ripping out legacy support,
I'd rather see us focus on looking ahead, rather than rewriting history.
But, by moving such large amounts of code to the GLDv3, you actually
greatly simplify the drivers thereby making support much, much easier
for them. Ultimately this reduces long term headaches, unless you're
going to propose that the existing drivers are flawless and will never
need to be touched again.
I also contend that the "churn" is actually pretty low, since most of
what is happening is that we are going to be _removing_ code and
replacing it with common code, that should also be well tested.
One possible risk is that this process will expose new issues in the
Nemo framework itself. Well, if that happens, that is a good thing,
because we have resources dedicated to finding and fixing nemo bugs, for
the benefit of all drivers.
Frankly, the biggest risk to eri/hme in the conversion is in the
reporting of the kstats. I consider that a pretty low impact risk, and
I'm pretty sure I can get it "right" the first time.
-- Garrett
_______________________________________________
networking-discuss mailing list
[email protected]
