On (06/24/07 20:14), Sebastien Roy wrote:
>>> Section 3:
>>>
>>> * Some details are needed regarding when it is valid to set a property's
>>> value. We've had discussions before about potential problem with
>>> modifying some properties after a driver has attached. Have these issues
>>> been worked out? If so, is there nothing to note regarding that in the
>>> design document?
>> I thought I had captured some of the prior discussions in Section 3 (page
>> 9 of Rev 0.2) - is there some specific discussion that you
>> had in mind that I had omitted to document?
>
> I don't see it. Maybe I'm not looking hard enough. I'm looking for a
> discussion detailing the implications of setting properties when there are
> active DLS clients utilizing a link. Is it okay to set all kinds of
> properties at any time, and are there no implications of doing so? Can
> some properties only be set when there are no DLS clients? Can other be
> set at anytime but only take effect when MAC drivers re-registered?
I was referring to the first 2 paragraphs at the top of page 10, but I agree
that it can be a little less terse. I will add more explanation. Not all
properties can be set when there are active DLS clients, when the
clients themselves need to be notified about change to the property,
and the notification mechanism does not exist. An example of this would
be changes to the tx/rx rings: changes to these properties on a plumbed
interface would require adjustments like flow/squeue reassignment
in IP, and the mechanism to do this does not exist today. In some
cases, (e.g., updating the mtu implies updating the sdu in the
mac registration), additional notification may be required (more on
this further below).
>> One issue that we are still working out, is property handling in
>> a diskless boot environment. The issue here is that there's no daemon
>> available for loading properties into the kernel,
>
> I'm not at all familiar with how diskless boot works, but why would there
> be no daemon in that case?
The daemon can only be started after init has started the daemons.
As I understand it, the driver attach(9E) operation happens much
before init, so if there are driver properties (e.g., chip related
properties) that need to be modified from their defaults in the xxx_attach
routine of the driver, the driver would have to rely on ddi_prop lookups
in the diskless boot case.
> I'm less interested in asynchronous notification mechanisms than in a
> MAC-layer interface to get or set a property. I'm going to need that for
> the Clearview IP tunneling device driver.
>
> Right now, I implement a mac_sdu_get() interface to do what I want. The
> iptun driver notifies the mac module of max SDU changes by using a
> mac_maxsdu_update() function, which caches the result in mac_impl_t (so
> that mac_sdu_get() can have access to the appropriate value), and also
> triggers i_mac_notify(mip, MAC_NOTE_SDU_SIZE), which in turn results in
> DL_NOTE_SDU_SIZE for all interested upper layers.
>
In the above scenario, are you interested in getting at a value maintained
in the mac layer itself, or is it more of a "pass setprop/getprop
through to driver; if driver does not have it, then see if mac can
answer the request"?
In the former case, one thought is that we could have the mac module
itself provide
mac_propf_t mac_set_prop, mac_get_prop;
functions. Then, when dld intercepts the setprop/getprop request,
if the targetted "pr_ifname" is "mac" we can invoke the mac_{set, get}_prop.
We could extend the same idea to support set/get prop requests for
other properties such as those related to squeues, where the property
is managed in ip itself, as opposed to one per network driver.
However, your subsequent comment:
> The driver tells the framework what its max SDU is when it registers in
> mac_register(), so the framework knows what the "default_mtu" is regardless
> of whether that driver implements Brussels APIs. My (perhaps useless)
> thought was that if a driver is incapable of responding to
> mc_getprop(deafult_mtu) calls, the framework could respond on its behalf
> given that it knows what the answer must be... Maybe this isn't worth
> implementing.
suggests that this is the latter case: a method for doing this would be
to pass in a flag in the dld_ioc_prop_val_t (e.g., add a pr_flags
field with the bit "DLD_GET_MAC_VALUE" set, so that if the setprop
function does not return a 0 error, mac will return its cached value
for the property).
>>> * All of these (except for MTU) are very Ethernet specific. How can the
>>> MAC-type plugin architecture be leveraged to provide a separate set of
>>> "public" properties for different link types?
>> Interesting thought.. but it strikes me that the problem areas are
>> slightly different. The MAC-type plugin architecture tries to provide
>> the same set of entry points for different mactypes: that principle
>> has been extracted into the mc_setprop/mc_getprop callback points in the
>> mac_impl_t. The actual implementation differences are embedded in the
>> setprop/getprop function itself. One related idea is to build on the
>> methods used for kstat management
>> via the mactype plugins.. if we had each mac provide a list of {property
>> number, property name, property type} with mactype_register, the framework
>> could use that to do the string <-> number mapping.
>
> Yes, this is what I was thinking. Each MAC-Type plugin tells the
> framework, "MACs for this type should support this set of properties".
>
>> It has the advantage that there's no reliance on hard-coded mapping via
>> sys/dld.h, and allows dladm to quickly filter out invalid setprop/getprop
>> attempts (e.g., setting tunnel hop limit on ethernet), but this would
>> require that libdladm upload the registered properties for
>> each mac that starts up, which requires additional system-calls for (at
>> least) the first setprop on the link, and complicates the initial
>> door_upcall hand-shake with dladmd to load up properties when the link
>> attaches.
>
> Not trivial. Maybe this points out an hole in the Nemo architecture, in
> that we have a generic framework with pluggable MAC-types in the kernel,
> but no such equivalent for dladm/libdladm.
>
>> Also, since the pd_check/pd_set/pd_get functions are already
>> a part of libdladm, the registration is primarily useful for mapping
>> name <-> number, and assesing if the property is supported. Or did you
>> have something else in mind?
>
> No, you addressed what I was curious about. The issues here are not
> trivial to solve indeed.
I'll make sure I track this as "Future Work" in the document..
> This is required by the Clearview iptun driver, and have this functionality
> working for max SDU (using DL_NOTE_SDU_SIZE upon max SDU changes). I
> basically detailed what I needed to do to get this working in the Clearview
> gate. The issues were not really that difficult to resolve for MTU.
> I could either help you make the few changes necessary to get a dynamic MTU
> property working for Brussels, or I will need to modify what you've done
> with Brussels to get this working for Clearview. Either way, a static MTU
> is a show-stopper for IP tunnel, as they adjust their MTU on the fly in
> response to path MTU changes to the tunnel destination.
Thanks! I'll check into what's needed for sending DL_NOTE_SDU_SIZE
notification from every ethernet driver, and get back to you... it seems
like it would make more sense to extract something like
tun_sendsdusize into common functionality that can be invoked for all
drivers.
>> If, otoh, the driver has not chosen to plug into the Brussels framework,
>> no
>> mc_{s, g}etprop would be registered, and this would be recognized as
>> the ENOTSUP by dladm. I realize that this causes some conflict because
>> the same error code (ENOTSUP) is also being used for the case when
>> the property has no semantics for the driver (e.g., trying to set
>> adv_autoneg_cap on tunnels). Should we use some other error code
>> for the latter case? If we use EINVAL to indicate a malformed set/get
>> property request (i.e., bad input for valid property) would it
>> make sense to use ENXIO for attempts to set/get a property in a context
>> where it does not apply?
The error code itself concerned me because it is a system-call interface,
and could conceivably be returned to a user-app that links into libdladm,
or makes the system call by itself..
> I'm not really as concerned about which error codes are returned than about
> what the administrator sees in show-link or show-linkprop output.
If the driver has not plugged into the framework yet, dladm will make
alternate system calls (if any) to find the value, e.g., use SIOCG*MTU
if that's the closest approximation of the property. If the value has
no meaning for the driver (the "ENXIO" case discussed above) dladm
would report that with the symbol used for "Not Applicable" (e.g., "--").
>
> Yes, but I think there are inter-project dependencies implied here which
> need to be made explicit. In other words, the IP tunnel properties will
> only be of use with the Clearview GLDv3 iptun driver. Likewise, VNIC
> properties will only be of use when Crossbow delivers VNICs.
>
Ok, will do.
--sowmini
