Erik,
Thank you very much for your comments. I will respond most of your
questions below, another few quesitons later after I do more investigation.
The legacy support seems to assume that everything that is DDI_NT_NET is
a DLPI provider. At least in the past there has been random stuff
labeled as DDI_NT_NET (I forgot what it was; but ifconfig -a plumb tried
to use some strange device in the past). It might make sense to verify
(with a DL_INFO_REQ) that the device is a DLPI provider before softmac
is applied to the device.
Sure. Issuing DL_INFO_REQ is already part of the softmac reigsteration
process, by which we get the MAC address, SDU size etc.
Vanity naming in combination with VLANs might cause some potential
confusion with snoop. Today if I snoop on bge0 I untagged packets as
well as all tagged packets. But if I snoop a VLAN (bge1000), I only see
packets with that VLAN tag. With vanity naming we'll have the ability to
name bge1000 as net1, and later change things so that bge0 is named
net1.
But this would need renaming the net1(bge1000) to another name first
because the system won't allow two links have the conflicting link
names. And this renaming operation needs to stop the snooping. "The
rename will fail if the link is currently in use."
Unless we do something, a side effect of such a change is that the
packets one can see with snoop changes.
As mentioned above, it is not possible that one can see the packets on
one link and then on another link, by using the same one snoop instance.
This might be confusing to
administrators, and a security issue if we decide it is safe for
somebody to snoop on net1 that shouldn't be allowed to see packets with
any VLAN tag. But I don't know 1) how critical this issue is, and 2) a
way to fix it without introducing a lot more complexity.
The APIs in section 7.2.5 explicitly expose that M_CTL messages are used
to carry IPsec information. I think we want to retain the flexibility to
change the way the IPsec information is carried to a NIC, so I would
recommend that the APIs be changed so that the da_ipsec_t is carried as
a separate argument to the send and receive functions. This adds a bit
of extra overhead in softmac, since it needs to convert between this
extra argument and the pre-pended M_CTL, but it means that GLDv3 drivers
with IPsec acceleration don't constrain our ability to replace the M_CTL
hack down the road. (For example, adding a m_tx_ipsec() function that
takes a dl_ipsec_t as an extra argument, and similarly on the receive
side.)
You mean on both receive-side and transmit-side, right? Sure.
A thing that isn't changed in this project, but where this project might
be a good opportunity to improve things, is the mapping from names like
ce17, bge17, and net0 to the actual NIC and port on the system.
With DLPI style2 devices it is hard to find this mapping, since /dev/ce
is just a pointer to a pseudo node. With DLPI style 1 devices there is a
bit more of a hint, because /dev/bge0 is a symlink to a /devices path
which describe the bus on which the device appears.
With Vanity naming, will any of that be easier or harder?
If I know I'm using link name net1, then I can find the device name
using dladm.
Yes.
And then can I use 'ls -l /dev/<devicename> to see the
physical path name?
No. Vanity naming compoment will not change anything under the /dev
namespace, therefore, for those devices only create DLPI style 2 device
nodes, there will be no /dev/<devicename> node. "ls -l
/dev/net/linkname" will not help either, as it only displays the "dev_t"
of the device with which the application can access the specified link.
For exmaple, ls /dev/net nodes of legacy devices will only displays the
corresponding softmac node.
Page 38 talks pf disabling the send-side loopback mechanism when using
softmac. An other approach which has been used for similar purposes in
the past, is to define a new b_flag value (MSGLOOP) which is set by
softmac on transmit and checked by softmac on receipt. Since both
copymsg and dupmsg preserve b_flag this would be in indication of a
message looped back by software in the DLPI driver. The scheme suggested
in the document (filtering based on the source MAC address) is both a
performance issue and disables some aspects of IPv6 Duplicate Address
Detection.
Great suggestion, thank you!
Minor:
------
In section 3.1.4, does the order between the cfgadm -c configure PCI8,
and dladm rename-link ce0 net0, matter?
It doesn't matter. Both order will work.
Is there an assumption that when the ce0 replaces bge0, that the system
automatically propagate information to the new one? By this I mean the
fact that the users of net0 might have joined some set of multicast
groups, others might have the device in promiscuous mode, and we might
have IPsec state in the driver.
But by the time when ce0 replaces bge0, the hardware of net0 ((bge0
card) is already disconnected from the system. Therefore, the problem
you mentioned is not a issue.
Table 1 shows N/A for Generic VLAN support on top of VLAN. Is this an
implementation constraint or an architectural constraint? The reason I'm
asking is that there are proposal for Q-in-Q encapsulations (Ethernet
packets with multiple VLAN headers in them). I don't know if Solaris
would ever be directly connected to such a thing, but it might make
sense to not have an architectural limitations preventing us to add
support for this should it become needed in the future.
Meem already answered this. Yes, architecturally, VLAN over VLAN will
fit into this proposed model.
Section 4.1.1. Is there some option to show-link -s that, akin to
'netstat -i 10', would show delta values for the counters?
Although it was not planned, I think it is a reasonable suggestion.
Section 4.3.4 autopush. It isn't clear from the document whether this is
"just" a way to specify autopush using a link name, but that it
otherwise has all the limitations of autopush. For instance, a user
could reasonably ask whether autopush would apply to a currently used
link name, or what it would take for the new autopush configuration to
be applied. I'm assuming one would need to do an ifconfig unplumb and
ifconfig plumb for the new information to be applied.
That is correct.
If that's the
case, would it make sense for autopush -m to fail if the link name is in
use?
Actually, the dladm autopush subcommand is intentionally designed to
work the same way as autopush(1M) as I think administrators might want a
similiar administrative experience. That is, the autopush -m subcommand
will succeed if the link is in use, or even the specified link doesn't
exist.
Page 20 footnote. Will CR 6292043 be fixed as part of this project?
Is it a requirement that it be fixed before this project integrates?
Yes.
Page 22 kstats for softmac. If softmac is going to prefix "net/" to the
kstat names, wouldn't it be simpler to apply this to the GLD kstat names
overall? With the specified approach any application that wants to look
at the standard kstats would need to be aware of softmac so that the
application can prefix "net/". Or is the intent that applications see
the kstats from the legacy driver?
Yes. The kstat names will be prefixed with "net" for all links. I will
update the document to make it clear.
Page 23 autopush backward compatibility.
The section ends with an assumption that autopushed modules not walk
q_next upstream to look for IP. Do we know if this is true for common
modules like vpnmod and fw-1?
Because we neither have the vpnmod code, nor know their contact, we
don't really know. But we did check the Checkpoint person and their
implementation doesn't have such behavior.
Section 6.2.2. In the new devfsadmd support, is there an implication
that there is a new persistent devfs repository for <link name, linkid>,
or will this use some existing repository (like path_to_inst)?
In the current design, this <link name, linkid> mapping information will
be in the /etc/datelink.conf file.
Section 6.2.4 last paragraph says that "administrators will run dladm
create-vlan -l net0". But is this the case even if a VLAN was explicitly
created before the rename? For example, if the administrator does
dladm create-vlan -l bge0 -v 23 net12
dladm rename-link -l bge0 foo3
does the administrator have to do something to (re)create net12, or does
net12's existence survive the rename?
net12 will survive the rename. It is stated in page 31:
"Note that VLANs and aggregations associated with the renamed link are
created over the link's corresponding MAC in the kernel, and their
persistent link configuration will only refer to the linkid, which will
never change. Therefore, the rename operation will not affect the
traffic over the relevant VLANs or aggregations, nor their persistent
link configuration."
Section 6.3.1 talks about how softmac generates devices. But it doesn't
fully specify how different legacy devices are handled. In terms of
presence in /dev, we have at least 3 different types of legacy drivers:
1. Those that only do style 2, hence have only a /dev/foo which is a
like to a pseudo entry in /devices
2. Those that support both style 1 and style 2, with a /dev/foo as well
as /dev/foo<N>.
3. Those that only support style 1. (I haven't see any such devices, but
I haven't looked for them either).
In particular, for #1, I don't see how softmac would know what instances
exist, since there is only a pseudo node. How would softmac know to
create /dev/net/ce0 and /dev/net/ce1 when two ce cards are in the system?
One of the input parameters of the device's postattach function is
dacf_infohdl_t, which carries the device's "dip" information. So it
doesn't matter whether the device only has style-1 or style 2, or both
node.
Section 6.3.2 talks about the need to support DL_ADDR_REQ etc. I didn't
think Sun Trunking was using that set of primitives. (They were added to
dlpi.h with the intent of folding IPMP and link aggregation together,
but that folding was never completed.) It makes sense to check the Sun
Trunking code whether it uses these primitives, or drop the paragraph
from the document.
It is DL_AGGR_REQ. It is in the Sun Trunking code.
Section 7.1.4 talks about transforming an M_MULTIDATA message into an
M_DATA message. It isn't clear to me that this provides much
performance benefits if any. Wouldn't it make more sense to add some
m_multitx() function by which multiple packets can be passed to the
mac driver in one call? That way the mac driver can use a single or a
few ddi_dma calls for the multiple packets.
Good suggestion. But for legacy drivers which cannot support
M_MULTIDATA, softmac still needs to transform M_MULTIDATA messages into
M_DATA messages, and there are already several Nemo drivers who don't
understand M_MULTIDATA message format. What we can do is that the GLDv3
framework can first check whether m_multitx() function is non-NULL for
the MAC being registered, if yes, calls the m_multitx() function,
otherwise, does the transforming and calls m_tx().
Editorial:
---------
Page 3 says "all link configuration and kernel data structures will be
updated to refer to linkids". I assuming you are not updating IP, ipf,
etc, but that this change is limited to the GLD layers. Same issue in
first paragraph in section 6.2.2.
Will fix.
In section 3.1.7, which is the first time you talk about /dev and
/dev/net, it would be quite useful if you could explain what NICs appear
where (legacy using softmac, vanity names, VLANs). I found that
information much later in the document.
Also in section 3.1.7, it would be helpful if you stated the persistence
of the VLAN PPA hack names. Does ls show them in the directory? Do the
disappear after an ifconfig unplumb? (I think I found that information
later in the document as well.)
Section 6.1.3. When I read this it wasn't clear whether softmac would
replace the /dev/ce, or whether it was left in place. (The question was
answered as I read deeper into the document.)
Will see what I can do.
Section 6.2.2 link configuration file.
I think it makes sense to state in the design document that this file
are not for editing, and the administrator must use dladm to manipulate
the content.
Will do.
Section 6.2.3 starts with "In the future". When do we expect this future
to arrive? If it is part of this project it makes sense to explicitly
state so. Same wording appears in section 6.2.5, 6.2.6, 6.2.7,
Accepted.
Thanks again for your comments. And again, I will address several
questions left out in this mail later.
Thanks
- Cathy
_______________________________________________
networking-discuss mailing list
[email protected]
