Re: Install to ZFS root is using device names hence failing when device tree is changed.

2021-09-07 Thread Peter Jeremy
On 2021-Sep-06 17:45:31 +0200, Karel Gardas  wrote:
>just installed 14-current snapshot from 2.9. on uefi amd64 machine. 
>Installed from USB memstick which was detected as da0 into the ssd 
>hanging on usb3 in external enclosure which was detected as da1.
>
>ZFS root pool is then using /dev/da1p3 as swap and /dev/da1p1 as 
>/boot/efi and probably also something as root zpool.
>
>Anyway, expected thing happen. When I pulled out USB stick identified as 
>da0 on reboot, the drive on USB3 switch from da1 to da0 and result is 
>unbootable system with complains about various /dev/da1xx drives missing 
>for swap efi boot etc.

Can you give more details about exactly what the errors and when they
occur during the boot cycle.  In particular:
* Low-level boot (anything prior to the FreeBSD kernel) knows nothing
  about da0 or da1, so any problems there are associated with your
  BIOS config, not FreeBSD.
* The swap partition will, by default, appear as a hard-wired device
  name in /etc/fstab - that will definitely need updating.  This will
  prevent the "swapon" working but won't prevent the boot.
* ZFS doesn't care about device names - it looks for ZFS labels on all
  possible devices.

-- 
Peter Jeremy


signature.asc
Description: PGP signature


Re: Install to ZFS root is using device names hence failing when device tree is changed.

2021-09-07 Thread Karel Gardas



On 9/7/21 10:29 AM, Peter Jeremy wrote:

On 2021-Sep-06 17:45:31 +0200, Karel Gardas  wrote:

just installed 14-current snapshot from 2.9. on uefi amd64 machine.
Installed from USB memstick which was detected as da0 into the ssd
hanging on usb3 in external enclosure which was detected as da1.

ZFS root pool is then using /dev/da1p3 as swap and /dev/da1p1 as
/boot/efi and probably also something as root zpool.

Anyway, expected thing happen. When I pulled out USB stick identified as
da0 on reboot, the drive on USB3 switch from da1 to da0 and result is
unbootable system with complains about various /dev/da1xx drives missing
for swap efi boot etc.


Can you give more details about exactly what the errors and when they
occur during the boot cycle.  In particular:
* Low-level boot (anything prior to the FreeBSD kernel) knows nothing
   about da0 or da1, so any problems there are associated with your
   BIOS config, not FreeBSD.


Low level boot is all right since kernel is booting. What's broken is 
root/zfs mount and swap enablement.



* The swap partition will, by default, appear as a hard-wired device
   name in /etc/fstab - that will definitely need updating.  This will
   prevent the "swapon" working but won't prevent the boot.


ACK. Good to know.


* ZFS doesn't care about device names - it looks for ZFS labels on all
   possible devices.


I think you are wrong here. Let's see zpool status:


karel@freebsd:~ $ zpool status
  pool: zroot
 state: ONLINE
config:

NAMESTATE READ WRITE CKSUM
zroot   ONLINE   0 0 0
  da1p4 ONLINE   0 0 0

errors: No known data errors

Now, let's reboot and see error on serial console when install da0 is 
not attached:




Root mount waiting for: usbus0
Root mount waiting for: usbus0
ugen0.5:  at usbus0
umass0 numa-domain 0 on uhub1
umass0: addr 4> on usbus0

umass0:  SCSI over Bulk-Only; quirks = 0x0100
umass0:9:0: Attached to scbus9
da0 at umass-sim0 bus 0 scbus9 target 0 lun 0
da0:  Fixed Direct Access SPC-4 SCSI device
da0: Serial Number 201701C7
da0: 400.000MB/s transfers
da0: 238475MB (488397168 512 byte sectors)
da0: quirks=0x2
Dual Console: Serial Primary, Video Secondary
No suitable dump device was found.
Setting hostuuid: 8cdf33eb-6866-42ae-a49d-ae7ee4c0c33c.
Setting hostid: 0xdf6467d8.
no pools available to import
swapon: /dev/da1p3: No such file or directory
Starting file system checks:
Can't open `/dev/da1p1'
/dev/da1p1: UNEXPECTED INCONSISTENCY; RUN fsck_msdosfs MANUALLY.
THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY:
msdosfs: /dev/da1p1 (/boot/efi)
Automatic file system check failed; help!
ERROR: ABORTING BOOT (sending SIGTERM to parent)!
2021-09-07T11:23:23.710549+02:00 - init 1 - - /bin/sh on /etc/rc 
terminated abnormally, going to single user mode

Enter full pathname of shell or RETURN for /bin/sh:


So this is problematic /efi parition, if I remove it from the /etc/fstab 
I get this boot:




Wow! It boots well, so you were indeed right. Checking zpool status reveals:
karel@freebsd:~ $ zpool status
  pool: zroot
 state: ONLINE
config:

NAMESTATE READ WRITE CKSUM
zroot   ONLINE   0 0 0
  da0p4 ONLINE   0 0 0

errors: No known data errors
karel@freebsd:~ $

so indeed, ZFS on FreeBSD is similar to this on Solaris. On Linux I got 
different experience (e.g. /dev/sdaX hardwired and failing to boot) 
hence I've been in impression that this is also a case of FreeBSD when 
code is shared thorough OpenZFS project.


Great to know and thanks for kicking me into it. So just /efi partition 
mount is the culprit here...


Thanks!
Karel



Install to ZFS root is using device names hence failing when device tree is changed.

2021-09-06 Thread Karel Gardas



Hello,

just installed 14-current snapshot from 2.9. on uefi amd64 machine. 
Installed from USB memstick which was detected as da0 into the ssd 
hanging on usb3 in external enclosure which was detected as da1.


ZFS root pool is then using /dev/da1p3 as swap and /dev/da1p1 as 
/boot/efi and probably also something as root zpool.


Anyway, expected thing happen. When I pulled out USB stick identified as 
da0 on reboot, the drive on USB3 switch from da1 to da0 and result is 
unbootable system with complains about various /dev/da1xx drives missing 
for swap efi boot etc.


Would be great if ZFS root install was using kind of device uuids or 
serial numbers or so so such device change would not disrupt freebsd 
from running correctly.


Thanks,
Karel




Install to ZFS root is using device names hence failing when device tree is changed.

2021-09-06 Thread Karel Gardas



Hello,

just installed 14-current snapshot from 2.9. on uefi amd64 machine. 
Installed from USB memstick which was detected as da0 into the ssd 
hanging on usb3 in external enclosure which was detected as da1.


ZFS root pool is then using /dev/da1p3 as swap and /dev/da1p1 as 
/boot/efi and probably also something as root zpool.


Anyway, expected thing happen. When I pulled out USB stick identified as 
da0 on reboot, the drive on USB3 switch from da1 to da0 and result is 
unbootable system with complains about various /dev/da1xx drives missing 
for swap efi boot etc.


Would be great if ZFS root install was using kind of device uuids or 
serial numbers or so so such device change would not disrupt freebsd 
from running correctly.


Thanks,
Karel