Hello all,
I believe this to be the final version of the Multihomed AI proposal
but please respond with any concenrns regardless to ensure the product is the
best it can be.
Problem:
--------
Today, the OpenSolaris Automated Installer makes many assumptions which
prevent widespread or simple use of a server running multihomed. Current issues
revolve around the use of multicast DNS, accidental disclosure of information;
automatic DHCP configuration and booting systems without needlessly sending
traffic across subnets.
Currently, AI uses multicast DNS to publish a text record on all
interfaces. The record contains an AI webserver IP address, which may or may
not be reachable on all attached subnets, however. Further, the AI webserver
which does bind to all interfaces has no access controls and thus may leak
information if the AI server is also on an untrusted interface.
At this time there is no guarantee of subnet consistency with the AI
webserver IP address published and the DHCP material configured when using the
create-service -i and -c options. Further there is no coordination to ensure
the GRUB menu's install_svr_address fall back mechanisim or the wanboot.conf's
RootPath share an IP used elsewhere in the AI configuration.
Bugs:
6182 - installadm tools doesn't support install servers which have
mutiple subnets
6922 - installadm create-service should have option to specify ip address
passed to dhcp config
7148 - installadm should give correct dhcp macro for the subnet being
configured on the server
7149 - installadm should allow user to choose which subnets to use
Requirements:
-------------
There are a number of desires for multihomed support in the Automated
Installer, however, the requirements picked are based on perceived technical
capabilities and customer site needs:
* The solution must be automatic to the administrator -- requiring no
extra action
* The solution must support one to all subnets served by the server
(default to be all subnets)
* The solution must not require routing between subnets (i.e. subnets
are otherwise isolated)
Use cases:
---------
The cases, as an admin. sees them are:
1. They want install service to be on one subnet
2. They want install service to be on some subnets
3. They want install service to be on all subnets
All three use cases would be supportable under this proposal. I would like to,
by default, do option 3 and then publish how to prune back (to achieve option 1
or 2) in documentation opposed to a full UI knob in installadm due to the
likely rarity of necessity for the larger AI using population* and to follow
the original AI goal of "make it work by default!"
*I see demanding multiple subnets to be towards the 20% of all AI users,
and especially demanding option 1 or 2 being in the 20% of that 20% (or
~5% of all AI users).
One use case not covered here, is that of a SPARC wanboot server which has many
subnets contacting it for AI service but for which the server does not reside
on any client subnet and for which subnet default services are desired. This --
is out of scope -- but still within the realm of administrator imagination
requiring by-hand modification and management of the /etc/netboot/<subnet>
directories.
UI Impact:
---------
This proposal assumes to control which interfaces are provided on a by-server
basis opposed to a by-service basis, simplifying coding, testing and UI.
By default, providing for use case option 3 (all subnets are configured at
create-service/create-client invocations) would cause little UI impact.
However, to allow one to achieve use cases 1 and 2 would have some
configuration impact.
It is envisioned that the configuration store would be the install/server SMF
service. The service would need to provide a way for masking out or explicitly
adding in interfaces. In this way, an administrator can choose to ensure that a
particular interface will always be excluded from providing installs (or
exposing AI webserver data - such as manifests) on a particular interface;
conversely, an administrator could set an SMF property to make the interfaces
property group a set of interfaces to only provide service on -- excluding all
others ever configured on the system. (It is possible the underlying
implementation to achieve this may be through a unified control such as TCP
wrappers but the SMF service would still control and only pass its
configuration down.)
If the interfaces a machine provides change, through a svcadm refresh and
restart of the install-server service, the new interfaces should be brought
online and old ones pruned as per the configuration set. (Further, if desired,
a hook to NWAM's interface bring-up and tear-down events could automate this
reconfiguration.) This change would be required for mDNS and DHCP macros which
can be put in place. Perhaps generating a SMF log message that various DHCP
server commands should be run to finalize bringing a new subnet online would
help as well.
Areas of impact:
---------------
* DHCP
Though technically feasible, automatic DHCP server and subnet
setup via the -i and -c create-service arguments will not be
supported. It is hoped more full implementation of DHCP
management will support multihomed servers through a UI interface
more acceptable to scaling AI DHCP support. Regardless, minimal
setup will always be to provide a macro or information on how
to create such a macro:
(As the SPARC BootFile URL has a hard coded IP address,
SPARC and X86 will differ in their DHCP requirements.
SPARC will require only a per subnet macro nothing more,
to the per-subnet BootSvrA data and per-service BootFile
data for an X86.)
SPARC:
Provide an AI macro per subnet which will provide a
specific BootFile URL specifying the subnet correct
webserver location to provide the wanboot CGI.
X86:
Will provide an AI macro for each service with a BootFile
location, however, it will be desired that a separate
macro configure the BootSvrA location on a per-subnet
basis. For example, this data could be in the network
macro or some other macro which is included for the
clients' use in a network macro.
* mDNS
Will have a separate mDNS responder listening on each interface to
respond with the correct IP address to advertise for the AI
webserver and interface.
Possible solutions for this are to use the PyBonjour library
which is a Python CTypes bridge to libdns.
* GRUB
Will extend the install_media and install_svc_address fallback
mechanisms in slim_source's
usr/src/cmd/auto-install/svc/manifest-locator as
currently delivered by GRUB to flag for using dhcpinfo(1) instead
to locate which machine provided the boot server via DHCP and use
that IP address in downloading ZLIBs.
* Wanboot
Wanboot in its wanboot.conf provides a root_server value which has
an IP address in it. The best option would be to modify ON's
usr/src/common/net/wanboot/bootconf.c valid_root_server()
function to populate a variable substitution similar to GRUB's
handling of variables (e.g. kernel$ and $ISADIR) for providing a
subnet specific IP to clients.
There are choices here which don't have a clear winner:
1) We can do hop detection or some other metric to try and
return the closest interface to the client
2) We can simply return a subnet local interface if not
returning a default interface.
(Both of these proposals require wanboot.cgi to know what
interfaces are serving AI data which is talked about in
the UI section.)
Alternatives:
a. Ideally (and the most parallel solution to how
GRUB determines its boot server), would be to have the OBP
Wanboot implementation continue to boot from where it had
already contacted the wanboot.cgi.
(However, this solution is impractical due to the
difficulty of deploying a new OBP across all potential AI
clients.)
b. Using the network directory structure in use by Wanboot
today we can simply create a wanboot.conf for every
client or service and every network.
(However, this hits n*m duplication which is unpalatable)
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
