I'd like to report a tricky situation and a workaround
I've found useful - hope this helps someone in similar
To cut the long story short, I could not properly mount
some datasets from a readonly pool, which had a non-"legacy"
mountpoint attribute value set, but the mountpoint was not
available (directory absent or not empty). In this case I
could neither create/cleanup the mountpoints, nor change
the dataset properties to mountpoint=legacy.
After a while I managed to override a higher-level
point in the FS tree by mounting tmpfs over it, and in
that tmpfs I could make the mountpoints needed by zfs.
I don't want to claim this adventure as a bug (because
in this case zfs actually works as documented), but rather
as an inconvenience that might need some improvement, i.e.
to allow (forced?) use of "mount -F zfs" even for datasets
with the mountpoint property defined.
Here goes the detailed version:
I was evacuating data from a corrupted rpool which I could
only import read-only while booted from a LiveUSB. As I wrote
previously, I could not use "zfs send" (bug now submitted to
Illumos tracker), so I reverted to directly mounting datasets
and copying data off them into another location (into a similar
dataset hierarchy on my data pool), i.e.:
# zpool import -R /POOL pool
# zpool import -R /RPOOL -o readonly=on rpool
My last-used root fs (rpool/ROOT/openindiana-1) had the
property mountpoint=/ set, so it got mounted into /RPOOL of
the LiveUSB environment. I copied my data off it, roughly
# cd /RPOOL && ( find . -xdev -depth -print | \
cpio -pvdm /POOL/rpool-backup/openindiana-1 ; \
rsync -avPHK ./ /POOL/rpool-backup/openindiana-1/ )
Likewise for many other filesystems, like those with legacy
mountpoints (mounted wherever I like, like /mnt/1) or those
with existing valid mountpoints (like /export).
However I ran into trouble with secondary (older) root FSes
which I wanted to keep for posterity. For whatever reason,
the "mountpoint" property was set to "/mnt". This directory
was not empty, and I found no way to pass "-O" flag to mount
for ZFS automounted FSes (to mount over non-empty dirs).
On a read-only rpool I couldn't clean up the "/mnt" dir.
I could not use the "mount -F zfs" command because dataset's
mountpoint was defined and not "legacy".
I could not change it to legacy - because rpool is read-only.
If I unmounted the "rpool/ROOT/openindiana-1" dataset, there
was no "/mnt" left at all and no way to create one on a
So I thought of tmpfs - I can mount these with "-O" anywhere
and need no resources for that. First I tried mounting tmpfs
over /RPOOL/mnt with "openindiana-1" mounted, but I couldn't
mount the older root over this mountpoint directly.
So I unmounted all datasets of "rpool", keeping the pool
imported, and mounted tmpfs over the pool's alternate mount
point. Now I could do my trick:
# mount -F tmpfs -O - /RPOOL
# mkdir /RPOOL/mnt
# zfs mount rpool/ROOT/openindiana
To be complete for those who might need this walkthrough,
since I wanted to retain the benefits of root dataset
cloning, I did that with my backup as well:
# zfs snapshot pool/rpool-backup/openindiana-1@20110501
# zfs clone pool/rpool-backup/openindiana-1@20110501 \
zfs clone pool/rpool-backup/openindiana
# cd /RPOOL/mnt && rsync -avPHK --delete-after \
Thanks to rsync, I got only differing (older) files written
onto that copy, with newer files removed.
"cpio" and "rsync" both barked on some unreadable files
(I/O errors) which I believe were in the zfs blocks with
mismatching checksums, initially leading to the useless
rpool. I replaced these files with those on the LiveUSB
in the copy on "pool".
Finally I exported and recreated rpool on the same device,
and manually repopulated the zpool properties (failmode,
bootfs) as well as initial values of zfs properties that
I wanted (copies=2, dedup=off, compression=off). Then I
used installgrub to ensure that the new rpool is bootable.
I also set the zfs properties I wanted (i.e. copies=2, also
canmount, mountpoint, caiman.* and others set by OpenIndiana,
compression=on where allowed - on non-root datasets) on the
rpool hierarchy copy in pool. Even though the actual data on
"pool" was written with copies=1, the property value copies=2
will be copied and applied during "zfs send|zfs recv".
Now I could "zfs send" the hierarchical replication stream
from my copy in "pool" to the new "rpool", kind of like this:
# zfs snapshot -r pool/rpool-backup@20111119-05
# zfs send -R pool/rpool-backup@20111119-05 | zfs recv -vF rpool
Since the hardware was all the same, there was little else
to do. I revised "RPOOL/rpool/boot/grub/menu.lst" and
"RPOOL/etc/vfstab" just in case, but otherwise was ready
to reboot. Luckily for me, the system came up as expected.
That kind of joy does not always happen as planned,
especially when you're half-a-globe away from the
computer you're repairing ;)
Good luck and strong patience to those in similar situations,
zfs-discuss mailing list