Thanks for putting this out. Some comments in-line.
Dave
On 03/31/10 02:21 AM, [email protected] wrote:
Hi all,
I've been thinking over the approach to take for supporting a
multihomed AI server, as now, one faces the bugs below. Please let me know
by Wednesday April 7th, 2010 if you see any issues.
Thank you,
Clay
Problem:
--------
Today, the OpenSolaris Automated Installer makes many assumptions
which prevent widespread or simple use of a server running multihomed.
Bugs:
6182 - installadm tools doesn't support install servers which have
mutiple subnets
6922 - installadm create-service should have option to specify ip address
passed to dhcp config
7115 - Custom wanboot.conf files are ignored in AI servers with multiple
NIC cards
7148 - installadm should give correct dhcp macro for the subnet being
configured on the server
7149 - installadm should allow user to choose which subnets to use
The primary problem in 7115 (making client-specific configuration
independent of the network it's attached to) is being addressed in the
image management work. I have a modified wanboot-cgi that implements it
which you should probably use in your work.
Requirements:
-------------
There are a number of desires for multihomed support in the Automated
Installer, however, the requirements picked are based on perceived technical
capabilities and customer site needs:
* The solution must be automatic
* The solution must support one to all subnets served by the server
* The solution must not depend on routing between subnets (i.e. subnets
are otherwise isolated)
Areas of impact:
---------------
* DHCP
Unable to automatically determine the router for an
arbitrary interface for initial DHCP setup (i.e. installadm -i and
-c options will be unsupported on a multihomed system):
SPARC:
Will provide an AI macro for each subnet
with differentiated BootFile location to provide the
correct IP for that subnet.
X86:
Will provide an AI macro for each service with a BootFile
location, however, it will be desired that an
administrator configure the BootSvrA location on a
per-subnet basis (i.e. in the network macro or some other
macro which is included for the clients' use)
(We will provide a template of Solaris DHCP commands as we do now,
however, IP addresses will need to be filled in by the admin as
it doesn't makes sense for us to become a multi-subnet DHCP
manager)
This seems to fall short of your first requirement ("must be
automatic"). I'm not clear why we'd fill in the correct IP in the SPARC
case and not x86, though.
* mDNS
Will have a separate mDNS responder listening on each interface to
respond with the correct IP address to advertise for the AI
webserver and interface.
Possible solutions for this are to use the PyZeroConf[1] library
or writing a Python CTypes bridge to libdns_sd.so.
* GRUB
Will extend the install_media and install_svc_address fallback
mechanisms in slim_source's
usr/src/cmd/auto-install/svc/manifest-locator as delivered by GRUB
to flag for using dhcpinfo(1) instead to locate which machine
provided the boot server via DHCP and use that IP address.
Alternatives:
Ideas which were thrown out to resolve providing the correct
install_media and install_svc_address IPs:
a. One TFTP server per subnet so that each one gets proper GRUB
menus for the network and all menus are written for that
subnet only
(Requires providing a TFTP server such as TFTPy[2] which has a
configurable bind address)
b. One TFTP server for all possible bind addresses, as we
have now, but then require subnet specific BootFile DHCP
macros which reference explicit GRUB menus
(Requires an n*m duplication of GRUB menus where n is clients and
services and m is machine interfaces -- or a TFTP server to on
the fly generate the GRUB menus)
We are likely to replace the boot arguments implementation in
manifest-locator with one based on receiving these parameters as part of
a wanbootfs from the server on x86, though I haven't had time to
prototype that yet. How would that affect your thinking here (I suspect
it then collapses into the below)?
* Wanboot
Wanboot in its wanboot.conf provides a root_server value which has
an IP address in it. The best option would be to modify ON's
usr/src/common/net/wanboot/bootconf.c valid_root_server() function
to populate a variable substitution similar to GRUB's handling of
variables (e.g. kernel$ and $ISADIR) for providing a subnet
specific IP to clients.
There are choices here which don't have a clear winner:
1) We can do hop detection or some other metric to try and
return the closest interface to the client
2) We can simply return a subnet local interface if not
returning a default interface.
(Both of these proposals require wanboot.cgi to know what
interfaces are serving AI data which is talked about in
the UI section.)
Alternatives:
a. Ideally (and the most parallel solution to how
GRUB determines its boot server), would be to have the OBP
Wanboot implementation continue to boot from where it had
already contacted the wanboot.cgi.
(However, this solution is impractical due to the
difficulty of deploying a new OBP across all potential AI
clients.)
b. Using the network directory structure in use by Wanboot
today we can simply create a wanboot.conf for every
client or service and every network.
(However, this again hits the n*m duplication which GRUB
alternative b would hit.)
Use cases:
---------
The cases, as an admin. sees them are:
1. They want install service to be on one subnet
2. They want install service to be on some subnets
3. They want install service to be on all subnets
All three use cases would be supportable under this proposal. I would like to
by default do option 3 and then publish how to prune back (to achieve option 1
or 2) as a blog entry or blueprint opposed to a full UI knob in installadm due
I presume you meant to say "in the product documentation"...
to the likely rarity of necessity for the larger AI using population*.
*I see demanding multiple subnets to be towards the 20% of all AI users,
and especially demanding option 1 or 2 being in the 20% of that 20% (or
~5% of all AI users).
This proposal envisions multiple subnet management to be a server wide issue
and not something at the service granularity.
UI Impact:
---------
By default, providing for use case option 3 (all subnets are configured at
create-service/create-client invocations) would cause little UI impact.
However, to allow one to achieve use cases 1 and 2 would have some SMF impact.
The SMF service would need to provide a way for masking out or explicitly
adding in interfaces. This way an administrator can choose to ensure that a
particular interface will always be excluded from providing installs (or
exposing AI webserver data - such as manifests - on a particular interface);
conversely, an administrator could set an SMF property to make the interfaces
property group a set of interfaces to only provide service on excluding all
others ever configured on the system.
If the interfaces a machine provides change, through a svcadm refresh and
restart of the install-server service, the new interfaces could be brought
online and old ones pruned. (Further, if desired, a hook to NWAM's interface
bring-up and tear-down events could automate this reconfiguration.) This would
control which interfaces are provided on a by-server basis opposed to a
by-service basis.
Would it make more sense to advise use of IPfilter or tcp wrappers (or
possibly even provide configuration of such) rather than inventing an
AI-specific set of controls for this?
Dave
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
