Sowmini.Varadhan at Sun.COM wrote:
> On (06/24/07 20:14), Sebastien Roy wrote:
>> I don't see it. Maybe I'm not looking hard enough. I'm looking for a
>> discussion detailing the implications of setting properties when there are
>> active DLS clients utilizing a link. Is it okay to set all kinds of
>> properties at any time, and are there no implications of doing so? Can
>> some properties only be set when there are no DLS clients? Can other be
>> set at anytime but only take effect when MAC drivers re-registered?
>
> I was referring to the first 2 paragraphs at the top of page 10, but I agree
> that it can be a little less terse. I will add more explanation. Not all
> properties can be set when there are active DLS clients, when the
> clients themselves need to be notified about change to the property,
> and the notification mechanism does not exist. An example of this would
> be changes to the tx/rx rings: changes to these properties on a plumbed
> interface would require adjustments like flow/squeue reassignment
> in IP, and the mechanism to do this does not exist today. In some
> cases, (e.g., updating the mtu implies updating the sdu in the
> mac registration), additional notification may be required (more on
> this further below).
Okay. While I understand what you've written at the top of page 10,
there are still questions in my mind. How does the software know whether
or not a given property is currently in effect, or won't be in effect
until something else is done at a later time? How will the administrator
know what to do to cause the properties to take effect if it's not
immediate? Are the conditions to cause a given property to take effect
the same for every driver which supports that property?
>>> One issue that we are still working out, is property handling in
>>> a diskless boot environment. The issue here is that there's no daemon
>>> available for loading properties into the kernel,
>> I'm not at all familiar with how diskless boot works, but why would there
>> be no daemon in that case?
>
> The daemon can only be started after init has started the daemons.
> As I understand it, the driver attach(9E) operation happens much
> before init, so if there are driver properties (e.g., chip related
> properties) that need to be modified from their defaults in the xxx_attach
> routine of the driver, the driver would have to rely on ddi_prop lookups
> in the diskless boot case.
Yuck.
>> I'm less interested in asynchronous notification mechanisms than in a
>> MAC-layer interface to get or set a property. I'm going to need that for
>> the Clearview IP tunneling device driver.
>>
>> Right now, I implement a mac_sdu_get() interface to do what I want. The
>> iptun driver notifies the mac module of max SDU changes by using a
>> mac_maxsdu_update() function, which caches the result in mac_impl_t (so
>> that mac_sdu_get() can have access to the appropriate value), and also
>> triggers i_mac_notify(mip, MAC_NOTE_SDU_SIZE), which in turn results in
>> DL_NOTE_SDU_SIZE for all interested upper layers.
>>
>
> In the above scenario, are you interested in getting at a value maintained
> in the mac layer itself, or is it more of a "pass setprop/getprop
> through to driver; if driver does not have it, then see if mac can
> answer the request"?
I'm not sure I can answer that question, as it could easily be either.
Maybe I can talk with you in private to resolve the issues here, as it
doesn't make sense to talk about the details without having a handle on
the big picture for MTU.
> In the former case, one thought is that we could have the mac module
> itself provide
> mac_propf_t mac_set_prop, mac_get_prop;
> functions. Then, when dld intercepts the setprop/getprop request,
> if the targetted "pr_ifname" is "mac" we can invoke the mac_{set, get}_prop.
I don't think this is where I wanted to go with this. My perceived need
for a kernel get/set interface wasn't to facilitate the implementation of
setprop/getprop in dld, but more to allow kernel modules to get or set
values of link properties described here.
> We could extend the same idea to support set/get prop requests for
> other properties such as those related to squeues, where the property
> is managed in ip itself, as opposed to one per network driver.
This seems unrelated to my query, I think.
> However, your subsequent comment:
>
>> The driver tells the framework what its max SDU is when it registers in
>> mac_register(), so the framework knows what the "default_mtu" is regardless
>> of whether that driver implements Brussels APIs. My (perhaps useless)
>> thought was that if a driver is incapable of responding to
>> mc_getprop(deafult_mtu) calls, the framework could respond on its behalf
>> given that it knows what the answer must be... Maybe this isn't worth
>> implementing.
>
> suggests that this is the latter case:
No, my two comments (one regarding a kernel interface, and the other
regarding where the MTU property comes from) were not related. In any case:
> a method for doing this would be
> to pass in a flag in the dld_ioc_prop_val_t (e.g., add a pr_flags
> field with the bit "DLD_GET_MAC_VALUE" set, so that if the setprop
> function does not return a 0 error, mac will return its cached value
> for the property).
It might be possible to do that, but maybe it's too much complexity for
little benefit. If every GLDv3 driver is going to be implementing the
"default_mtu" property in practice, then maybe there's little value in
implementing something like that.
>> This is required by the Clearview iptun driver, and have this functionality
>> working for max SDU (using DL_NOTE_SDU_SIZE upon max SDU changes). I
>> basically detailed what I needed to do to get this working in the Clearview
>> gate. The issues were not really that difficult to resolve for MTU.
>
>
>> I could either help you make the few changes necessary to get a dynamic MTU
>> property working for Brussels, or I will need to modify what you've done
>> with Brussels to get this working for Clearview. Either way, a static MTU
>> is a show-stopper for IP tunnel, as they adjust their MTU on the fly in
>> response to path MTU changes to the tunnel destination.
>
> Thanks! I'll check into what's needed for sending DL_NOTE_SDU_SIZE
> notification from every ethernet driver, and get back to you... it seems
> like it would make more sense to extract something like
> tun_sendsdusize into common functionality that can be invoked for all
> drivers.
The tun module in onnv-gate isn't a GLDv3 driver, so looking there will
be fruitless. The iptun driver in the Clearview iptun gate is. I can
give you an overview of what I've done there, as it is essentially 90% of
what would need to be done for Brussels. Instead of getting back to me
later after your investigation, maybe I can help you by providing you
with the information you need before you start your investigation... :-)
> If the driver has not plugged into the framework yet, dladm will make
> alternate system calls (if any) to find the value, e.g., use SIOCG*MTU
> if that's the closest approximation of the property.
SIOCG*MTU isn't at the right layer, it's an IP thing. dladm show-link
already displays the link MTU properly without a Brussels property, so
why not just do what its already doing? (it currently uses the
dlpi_info() libdlpi function, which seems like the correct thing to do)
> If the value has
> no meaning for the driver (the "ENXIO" case discussed above) dladm
> would report that with the symbol used for "Not Applicable" (e.g., "--").
Okay.
-Seb
