Thanks very much for taking the time to do the review! We really appreciate
the feedback.
On (04/01/09 13:21), Prasanna Kunisetty wrote:
> General Questions:
> ==============
>
> - What's the current release model for this feature opensolaris and Nevada?
> - Will this feature be available in the S10 updates?
> - What's the timeline for this feature availability?
We're working towards integrating this in the onnv_127 time-frame.
There are no plans to make this available in S10 update at this time and will
be only available in Opensolaris and Nevada.
> - Will the backward compatibility be provided?
> - Any plan of dropping any ifconfig flags or ndd properties as part of
> this project?
We will provide backward compatibility in that ifconfig, ndd etc will
continue to remain to be available, though they will actually use the
libipadm library where possible. But the downside to using these tools
is that they will not have support for the enhancements like property
persistence etc. Also ipadm will never support known "legacy"
configuration methods for some features like ipsec and ipmp.
Some of the legacy ndd properties may be dropped, usually because we
can come up with better ways to set these parameters (either auto-tune
them in the kernel, or provide some better alternative). When we drop
an ndd tunable, there will be an RFE filed with the "brussels-ndd-cleanup"
keyword in Bugster, to allow any users sufficient time to react to the
proposed change.
> IPMP Interaction:
> ==============
> - ipmp option is only possible with create-option, so how do we
> bring an existing interface "under" ipmp group.
there will be set of ipmp sub-commands (create-ipmp, delete-ipmp,
show-ipmp, add-ipmp, remove-ipmp) that will take care of what you
are asking. The comphrensive set of ipmp sub commands will be
available through phased delivery. We will update the design doc
shortly with those sub commands, but here is a synopsis:
- Create an ipmp group with an optional groupname and add interfaces optionally
#ipadm create-ipmp [-t] [-g <groupname>]\
[-i <under-interface>...] <ipmp-interface>
Example:
# ipadm create-ipmp -g testgrp0 grp0
- Delete an ipmp group. Returns error if the group is not empty.
#ipadm delete-ipmp <ipmp-interface>
Example:
# ipadm delete-ipmp grp0
- Add interfaces to an ipmp group
#ipadm add-ipmp -i <under-interface>\
[-i <under-interface>...] <ipmp-interface>
Example:
# ipadm add-ipmp -i net0 -i net1 grp0
- Remove interfaces from an ipmp group
#ipadm remove-ipmp -i <under-interface>\
[-i <under-interface>...] <ipmp-interface>
Example:
# ipadm remove-ipmp -i net1 grp0
# ipadm remove-ipmp -i net0 grp0
- Show ipmp group details.
#ipadm show-ipmp <ipmp-interface>
INTF UNDERINTF STATE ACTIVE STANDBY TESTADDR
ipmp0 net0 ok yes no 11.1.2.3
net1 failed no yes 11.1.2.4
- Data/Test addresses can be added to the ipmp group using ipadm add-address
command just like an address is added to any other interface.
- An underlying interface is marked standby using ipadm set-intfprop
# ipadm set-intfprop -p standby=true <under-interface>
> - If the /etc/hostname.adp mechanism is being replaced, that means going
> further the only mechanism
> to create the ipmp groups is with ipadm command. If I understand correctly,
> whether we want the changes to be persistent or not, it will be just
> ipadm command. Right?
Eventually, after ipadm becomes fully compatible with ifconfig, it will
be the preferred method for configuration. But, as mentioned earlier,
ifconfig will be available for those who want to use it.
A longer discusision of the /etc/hostname.intf files is available
further below with the questions about persistence
> - Does it mean ipadm command will be used to create the ipmp groups and
> ipmpstat is used to get the status or can we use the ipadm command
> to get the ipmp group status as well?
At least for the initial delivery, ipadm will not have the full feature
set available via ipmpstat. The eventual goal is to extend the feature-set of
'show-ipmp' to be fully compatible with ipmpstat.
> - May be this is clearview specific, but with the new ipadm command
> how do we customize the name of psuedo interface? With Clearview we
> do that by creating the file with the /etc/hostname.<psuedo_interface>
Not sure I understand the question? dladm rename-link should be
used to rename physical links. For ipmp meta interfaces, the command
"ifconfig grp0 ipmp"
is the same as
ipadm create-ipmp [-g mygroup] grp0
(the second command allow you to specify a custom group name, which
the first does not).
> In Sun Cluster, we do auto creation of ipmp groups by updating
> the /etc/hostname.<adp> file and running the ifconfig command to
> create groups for an existing interface, so answers to above questions
> would be useful.
>
> It would be good to clarify this in the design doc for reference.
>
> ipadm library:
> ==============
> - As I understand the ipadm command can be used in place of current
> ifconfig command.
yes, though the full feature set may not be available at the first
delivery.
> - Can the ipadm library be used just in place of ndd to set/get the
> protocol properties
we will have equivalent support with ndd features, with some
differences; whether these differences impact you will depend on what
properties you use today, so it would be very useful if you could tell us
what properties are critical to Sun Cluster? here are the differences
between ndd and ipadm:
- you can make function calls to set/get properties with libipadm, as
opposed to spawning /sbin/ndd. Also the function call will be able to
make the property change with persistence (i.e. will be applied on
next reboot/restart of interface).
- many properties will be settable per interface (similar to ip_forwarding
in ndd today)
- a list of properties that will be supported by ipadm will be tracked in
http://cr.opensolaris.org/~girishmg/ipadm_prop.htm
So you can always check this list to check if the property you want is
available in ipadm (existing ndd stuff will be left alone).
> or can this also be used in place of current ioctls for interface
> specific operations
> like plumb, unplumb etc? In other words, are you expecting the
> applications currently using
> the ioctls to plumb, unplumb, addif etc operations migrating to the new
> ipadm libarary calls?
The eventual goal is to have all the applications that currently
use plumb/unplumb to go through libipadm instead of cutting/pasting
the PUSH/PLINK logic for plumbing ip. Thus we are trying to provide
both the /sbin/ipadm and libipadm fucntion call interfaces for this.
The model is that there will be a 1-1 mapping between the /sbin/ipadm
arguments and the function call (i.e., if you have
/sbin/ipadm create-link <args>, the corresponding ipadm_create_link()
invocation would be straight-forward, and would also support all the
<args>).
I understand clusters have some code that do interface plumbing. We would
appreciate pointers to your source code, so that we can verify
(as we develop the code) that the conversion will not will leave out
something that is needed by clusters.
> - Expecting that this project will not have any impact on kstr_ioctls.
correct.
> ifconfig vs. ipadm:
> -===============
> - ipadm command description in the design doc did not include any flags
> related to zones. How do we attach a zone to an interface? Today using
> ifconfig we can do something like below:
>
> ifconfig <interface> <address> zone <zonename>
>
> How do we achieve this with the new ipadm command?
Good point. Meem also pointed out that it may be better to have a separate
"set-addrprop" command, rather than overloading the modify-addr command.
We are working out the details of the *-addr command taking into account
your/Meem's input.. stay tuned for details which will be posted shortly.
> - Did not see any flag for the netmask? How do we set the netmask using
> the ipadm command?
the netmask command is initially set as part of the address. But, in
order to change the existing netmask on an address, you would have to
use the set-addrprop command.
> - seems like ipadm semantics are different compare to ifconfig with respect
> to plumbing the v4 and v6 as default instead of just v4 as the default.
> Is this right? If yes, then if user is interested in only v4 version
> then need
> to restrict with -f flag. What's the motivation for change? Do we have more
> customers now who are using the v6? Otherwise, it looks unneccesary to
> use the extra flag.
We've had several discussions about this, see
http://mail.opensolaris.org/pipermail/brussels-dev/2009-March/001415.html
plus there has been a long term desire to have IPv6 on by default,
which is why we converged on this solution, for those administrators
who wish to turn off one or the other, the -f flag is made available.
> - Question related to Modify/Add address section:
> Is specifying the -l num will be same as
> ifconfig ce0:1 <address>
yes.
> Also is not specifying the -l is same as
> ifconfig ce0 addif <address>
yes.
> So how do I change the existing IP address for an interface? In other
> words, if my ce0 is plumbed
> currently with let's say x.x.x.x and want to change that to a new
> IP address let's say y.y.y.y?
The design doc proposes to do this via the modify-addr command:
# ipadm modify-addr <newaddr> -l <tag> -i <intf>
(note that meem had some input on how to name the interface object itself,
and we will post the result of these discussions in a bit).
> - For the address options, type = static. Don't remember seeing this type
> in ifconfig context, is this a new word that we are using in ipadm context?
yes, but after you asked, it occurred to us that it may be better to
make static the default, so that it does not have to be specified every time.
> Property persistent Mechanism
> =============================
>
> - In section, 2.6.1, it was mentioned that by default, the set-prop
> options are stored
> in the persistent storage:
>
> >The setting will
> >be persistent by default and applied when the interface is recreated
> with ipadm create-interface
>
> But section 2.6.3 mentions that the properties can be restored via
> init-prop:
>
> ># ipadm init-prop
> >which will consult the persistent store and re-apply global properties
>
> Does end-user ever required to use this init-prop option? Or is this only
> useful for the new process "netstart" during the reboot?
End-user should not have to use init-prop- this was only intended for
restoring persistence during reboot. We shall clarify in the document.
> - Section 4.1 mentions applications or the ipadm requires the file_dac_write
> privilege? By default is this privilege available to root?
yes.
> - Is the /etc/ipadm/ipadm.conf a directory under which there will be
> interface specific files? Otherwise how does the interface specic config
> be stored?
The /etc/ipadm/ configuration files are all hidden, Private interfaces,
just like the dladm configuaration files. So the way the information
is stored is subject to change, and the libipadm users should not make
any assumptions about this.
> - Let me reiterate the property restoration process as I understood and
> correct me if there is anything incorrect:
>
> During the reboot, the new process "netstart" will be started which will
> run the ipadm init-prop command to initialize all the global properties.
> After that when the net-physical script which I believe is an SMF service
> gets executed, it runs the ipadm show-interface to get the list of
> interfaces.
All correct so far, but the order of restoration is reversed, i.e., instead
of doing
> For each interface from this list, it refers to the
> /etc/ipadm/ipadm.conf/<interface> file to read the config and creates the
> intefaces with this information. If the /etc/ipadm/ipadm.conf/<interface>
> does not exist, then it looks at the /etc/hosntame.<inteface> and creates
> the new <interface> specific file under ipadm.conf directory to be used for
> later reboots.
What we would do is
- lookup and restore information from /etc/hostname.* files.
Since ifconfig itself will be using libipadm underneath, the attempt
to plumb the interface will invoke ipadm_create_interface() which will
actually pull down props from ipadm.conf
- for each intf in 'ipadm show-prop -p' but not in /etc/hostname/$intf
do
ipadm create-interface $intf # will pull down props from ipadm.conf
done
> Based on the above understanding have following questions:
>
> 1) How does the new process "netstart" be started? Will this be part of
> an existing
> SMF service or will there be a new SMF service? Based on that, Sun
> Cluster might need to add
> a new dependency on this SMF, if it's not already covered by the
> existing Solaris dependencies.
netstart will be started *before* smf, via /etc/inittab. Thus Sun Cluster
will not need any new dependancies.
> 2) Are the global properties which will be restored by init-prop
> also stored in the /etc/ipadm/ipadm.conf directory?
yes, but the format of storing this is Private and subject to change.
> 3) In section 4.2, third bullet mentions that if the /etc/hostname.adp
> files are
> detected then the subsequent SMF service will update this configuration
> information to input
> in the ipadm store for the next reboot. What exactly does it mean? Does
> it store the config
> information from the /etc/hosntame.<adp> file to /etc/ipadm/ipadm.conf
> directory to be used by next reboot?
> Then what happens if user has changed the /etc/hostname.<adp> file in
> between? In other words,
> how does the /etc/ipadm/ipadm.conf will be in sync with the
> /etc/hostname.adp files?
what this means is that if there are updates to the /etc/hostname.intf
file, then on reboot, we will run a process that updates ipadm.conf to
reflect the same information as the /etc/hostname.intf. Thus if the user
changes /etc/hostname.intf during some booted session, the ipadm.conf
will be updated on the next reboot.
Note that, during the transition period, we will still respect all the
changes in /etc/hostname.intf because of the network-physical sequence
I listed earlier.
> It would be good to summarize this flow or better add a flow diagram for
> the above in the design doc.
>
good idea, we'll clarify all these points in the doc and send out the updated
design doc pretty soon.
Please keep the comments coming- they are very valuable for us!
--Sowmini
