Jan and Jerry,
I've just noticed a rather nasty issue with upgrade for Clearview IPMP
that I'd like your input on. Specifically, if a machine running Nevada
IPMP is upgraded to Clearview IPMP, upon first reboot, all IP interfaces
using IPMP will fail to configure. Subsequent boots will work fine.
The problem is due to a conflict between the new way interfaces using IPMP
are plumbed and the SMF dependency graph for the device/local service.
Specifically, each IP interface using IPMP is now also associated with
IPMP group IP interface (e.g., ipmp0). While we recommend that these IPMP
group IP interfaces be explicitly created before placing an interface into
a group, for backward compatibility, ifconfig(1M) will also create the
IPMP group IP interfaces on-demand. The IPMP group IP interface is
created by opening the special /dev/ipmpstub DLPI pseudo-datalink, which
advertises itself as a special SUNW_DL_IPMP DLPI type to the IP and ARP
kernel modules.
This generally works fine, but in the case of first reboot, since
device/local indirectly depends on network/physical, devfsadm has not yet
run and thus /dev/ipmpstub does not yet exist. Thus, the attempt to
place the interfaces into an IPMP group fails. Later in boot, devfsadm
runs, creates /dev/ipmpstub, and subsequent boots work fine.
I can't see a simple *and* elegant way to fix this.
1. Simple but hideous: have libdlpi`dlpi_open() explicitly fall
back to trying to open the node in /devices, like we used to
when netbooting pre-UV. The hack would be even worse than
before since the ipmpstub /devices node doesn't use the clone
driver because the backing driver (dlpistub) actually backs
both /dev/vni and /dev/ipmpstub.
2. Elegant but complex (and maybe impossible): create the /dev
symlinks earlier in boot. In theory, it seems like it's
possible to create the symlinks as soon as the root filesystem
is mounted r/w. Right now, we wait until /usr is mounted
(since devfsadm and many of its dependencies are on /usr).
However, for reasons I don't yet understand, even mounting root
r/w seems to have dependencies on network/physical, so this may
be hopeless.
Are there other approaches I've missed?
Thanks,
--
meem
