The /boot/ has to be unencrypted and can be stored on eg. USB
pendrive which is never left unattended, unlike laptop which can be left
in eg. a hotel room, but with entire disk encrypted.
As we discussed elsewhere, there are many options of configuring full disk
encryption. Including decisions whether root filesystem should be separate from
boot filesystem, choice of filesystem type for boot fs, ways of tying various
pieces together, and many more.
I do not believe that my change is incompatible with full disk encryption in
general.
So, I see three ways of resolving the problem that my changes caused for your
configuration.
1. [the easiest] Put zpool.cache loading instructions that used to be in
defaults/loader.conf into your loader.conf. This way everything should work as
before -- zpool.cache would be loaded from your boot pool.
2. Somehow (I don't want to go into any technical details here) arrange that
your root pool has /boot/zfs/zpool.cache that describes your boot pool. This is
probably hard given that your /boot is a symlink at the moment. This probably
would be easier to achieve if zpool.cache lived in /etc/zfs.
3. [my favorite] Remove an artificial difference between your boot and root
pools, so that they are a single root+boot pool (as zfs gods intended). As far
as I understand your setup, you use GELI to protect some sensitive data.
Apparently your kernel is not sensitive data, so I wonder if your /bin/sh or
/sbin/init are really sensitive either.
So perhaps you can arrange your unencrypted pool to hold all of the base system
(boot + root) and put all your truly sensitive filesystems (like e.g. /home or
/var/data or /opt/xyz) onto your encrypted pool.
If all you care about is laptop being stolen, then that would work.
If you however want to be protected from someone replacing your /sbin/init
with something evil then you use encryption or even better integrity
verification also supported by GELI.
There are different ways to ensure that. Including storing cryptographic
checksums in a safe place or keeping init in the same place where kernel is
kept. And probably many more.
Remember, tools not policies.
I am not trying to enforce any policy on end-users here.
There is also option number 4 - backing out your commit.
That's definitely an option. I'll discuss it a few lines below.
When I saw your commit removing those entries from defaults/loader.conf,
I thought it is fine, as we now don't require zpool.cache to import the
root pool, which was, BTW, very nice and handy improvement. Now that we
know it breaks existing installations I'd prefer the commit to be backed
out.
"breaks" sounds dramatic, but let's take a step back and see what exactly is
broken.
The system in question still can boot without a problem, it is fully usable and
it is possible to change its configuration without any hassle. The only thing
that changed is that its boot pool is not imported automatically.
Let's also recall that the system was not created / configured by any of the
existing official or semi-official tools and thus it does not represent any
recommended way of setting up such systems. Glen configured it this way, but it
doesn't mean that that is the way.
I think that there are many of ways of changing configuration of that system to
make behave as before again.
Three I mentioned already. Another is to add rc script to import the boot pool,
given that it is a special, designated pool. Yet another is to place
zpool.cache onto the root pool and use nullfs (instead of a symlink) to make
/boot be from the boot pool but /boot/zfs be from the root pool.
This is because apart from breaking some existing installations it
doesn't gain us anything.
I think I addressed the "breaking" part, as to the gains - a few lines below.
So I understand that my change causes a problem for a setup like yours, but I
believe that the change is correct.
The change is clearly incorrect or incomplete as it breaks existing
installations and doesn't allow for full disk encryption configuration
on ZFS-only systems.
I think I addressed the breaking part and also addressed your overly general
statement about full disk encryption. So I don't think that my change is
"clearly incorrect", otherwise that would be clear even to me.
BTW. If moving zpool.cache to /etc/zfs/ will work for both cases that's
fine by me, although the migration might be tricky.
Yes, that's migration that's scary to me too.
Now, about the postponed points.
I will reproduce a section from my email that you've snipped.
P.S.
ZFS/FreeBSD boot process is extremely flexible. For example zfsboot can take
zfsloader from pool1/fsA, zfsloader can boot kernel from pool2/fsB and kernel
can mount / from pool3/fsC. Of these 3 filesystems from where should
zpool.cache be taken?
My firm opinion is that it should be taken from / (pool3/fsC in the example
above). Because it is the root filesystem that defines what a system is going
to do ultimately: what daemons are started, with what configurations, etc.
And thus it should also determine what pools to auto-import.
We can say that zpool.cache is analogous to /etc/fstab in this respect.
So do you or do you not agree with my reasoning about from where zpool.cache
should be taken?
If you do not, then please explain why.
If you do, then please explain how this would be compatible with the old way of
loading zpool.cache.
I think that ensuring that zpool.cache is always loaded from a root filesystem
is the gain from my change.
