Re: FreeBSD 13/stable and zpool upgrade
On 20/02/2021 00:08, Pete French wrote: I suspect there are many variants on this out there! :-) Well, guess what: * http://popeye.lapinbilly.eu/git/?p=zfsinstaller.git;a=blob;f=zfsinstall.sh;h=c63dfa803e1973006a752bd322e083217e97c67a;hb=HEAD by the fact, this one has to be updated for 13-branches. -- David Marec https://reuz.lapinbilly.eu/#/room/#FreeBSD:matrix.org ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On 19/02/2021 22:30, Matthew D. Fuller wrote:
e.g., on one system BIOS-booting system, /boot/rewrite-bootcode.sh:
--
#!/bin/sh -x
for i in /dev/nda0 /dev/nda1; do
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ${i}
done
--
Ah, that made me smile
FreeBSD 12.2-STABLE stable/12-c243191-g079273384c2 GENERIC
[pete@skerry ~]$ cat /root/update_boot_blocks
#!/bin/sh
for DRIVE in ada0 ada1 ada2 ada3
do
/sbin/gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ${DRIVE}
done
I suspect there are many variants on this out there! :-)
-pete.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Fri, Feb 19, 2021 at 10:26:15PM +0100 I heard the voice of
Kurt Jaeger, and lo! it spake thus:
>
> We do not need it automated. We need it to be described in enough
> detail that we can write that BOOTx64.efi to the proper place. If
> there are some steps to find out where to write it etc., fine.
> Describe those steps.
As a data point, years ago I started dropping a rewrite-bootcode.sh
script in /boot on all my systems, 'cuz I could never offhand remember
how to do it so I kept putting it off. And I updated a variant of it
for EFI.
e.g., on one system BIOS-booting system, /boot/rewrite-bootcode.sh:
--
#!/bin/sh -x
for i in /dev/nda0 /dev/nda1; do
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ${i}
done
--
And on an EFI system,
--
#!/bin/sh -ex
bd="/mnt/EFI/BOOT"
for i in gpt/efi0 gpt/efi1; do
mount -t msdosfs /dev/${i} /mnt
cp /boot/boot1.efi ${bd}/BOOTX64.EFI
echo "BOOTX64.EFI" > ${bd}/STARTUP.NSH
umount /mnt
done
--
Another variant I have on my workstation (which doesn't EFI boot 'cuz
DRM, but can, so I'm pretty sure the script works right),
-
#!/bin/sh
cd `dirname $0`
SRC="loader.efi"
for i in 0 1; do
mount /mnt/efi${i}
DSTDIR="/mnt/efi${i}/efi/boot"
mkdir -p ${DSTDIR}
DST="${DSTDIR}/BOOTx64.efi"
if(!(cmp ${SRC} ${DST})); then
echo "$DST: Updating"
cp -p ${SRC} ${DST}
else
echo "${DST}: Up to date already"
fi
umount /mnt/efi${i}
done
-
% grep efi /etc/fstab
/dev/gpt/nefi0 /mnt/efi0 msdosfs rw,noauto,-g0,-m770 0 0
/dev/gpt/nefi1 /mnt/efi1 msdosfs rw,noauto,-g0,-m770 0 0
-
(and yes, when I wrote these, just _figuring_ _out_ WTF EFI wanted and
where it wanted it was _incredibly_ frustrating...)
Of course, those all take manual setup for each system, to know the
devices or labels, and with EFI, to either manually mount or setup
fstab entries for whatever partition[s] apply. But at least I get to
do it once, when I setup the system, and presumably know what I just
did, rather than trying to remember years later while sitting at the
console of a system I need to finish the update of and get back up and
running. This way I know I can just `cd /boot ; ./rew` after
installworld and be done with it:)
--
Matthew Fuller (MF4839) | fulle...@over-yonder.net
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
On the Internet, nobody can hear you scream.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
Hi, On 19/02/2021 19:24, Kurt Jaeger wrote: > I suspect that your 'zpool upgrade' enabled things that weren't enabled > before. This caused the old boot blocks to no longer work. Correct. That' s definitely the source of the booting issue. > We should be better about upgrading boot blocks, but EFI is kinda new and > kinda different EFI is able to boot any FreeBSD box for a while. The main issue is that the legacy way to upgrade these /bootcode and partcode/ not only does not work, but do bad things. # gpart bootcode -p /boot/gptzfsboot -i 1 ada0s1 will install an old and inappropriate /partcode/. IMO, 'gptzfsboot' should be sweep off along with 'boot1.efifat' ( by calling 'make delete-old') or be built with the right 'BOOTx64.EFI', which actually is`/boot/loader.efi` . -- David Marec http://wiki.fug-fr.org/doku.php?id=start https://diaspora.lapinbilly.eu/ ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Fri, Feb 19, 2021 at 2:42 PM David Marec wrote: > On 19/02/2021 22:00, Warner Losh wrote: > > We can't do that. gptzfsboot is for something else that we can't get rid > > of: BIOS/CMS booting. > > > My bad. I mean 'gptboot.efifat'. > > > root@machine:~ # mdconfig -u 0 -f /boot/gptboot.efifat > root@machine:~ # mount -t msdosfs /dev/md0 /mnt > root@machine:~ # diff /boot/loader.efi /mnt/efi/boot/BOOTx64.efi > Binary files /boot/loader.efi and /mnt/efi/boot/BOOTx64.efi differ > Ahh if gptboot.efifatis still there after delete-old, something is missing from ObsoleteFiles.inc Warner > -- > > David Marec > http://wiki.fug-fr.org/doku.php?id=start > > ___ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Fri, Feb 19, 2021 at 2:21 PM Kurt Jaeger wrote: > Hi! > > > You can upgrade to 13 w/o hassle. The upgrade process doesn't > automatically > > upgrade the zpools. The only problem is if you also do a 'zpool upgrade' > > which will change your zpool features. The fix is easy: upgrade your boot > > loader (bootx64.efi) on the ESP at the same time before rebooting... > > In the past there was this nice warning explaining what to do: > > gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0 > Yea, pmbr and gptboot are now effectively built into the EFI boot loader... > Now, what would be the new procedure for that ? > mount -t msdos /dev/ada0sX /mnt cp /boot/loader.efi /mnt/boot/bootx64.efi umount /mnt > > There's nothing really to fix here. > > Well, it's very dangerous, and having a way to know how to fix > it would be very important. > You also need to know what you are doing when you upgrade your zfs pools. This should be called out prominently in the release notes, howeve. > > All the code is there. It all works. > > The upgrade is fail safe up until the point you do the zpool upgrade, > which > > has to be done by the system administrator. It's not done in the > installer > > or as part of 'make installworld'. > > I understand. But if the installer does not know that she/he needs to > do something with some bootx64.efi file... ? > > Can't zpool upgrade mention the necessary steps, as it did before ? > I wasn't aware it lost that in the openzfs upgrade... What did it used to say? Warner ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On 19/02/2021 22:00, Warner Losh wrote: We can't do that. gptzfsboot is for something else that we can't get rid of: BIOS/CMS booting. My bad. I mean 'gptboot.efifat'. root@machine:~ # mdconfig -u 0 -f /boot/gptboot.efifat root@machine:~ # mount -t msdosfs /dev/md0 /mnt root@machine:~ # diff /boot/loader.efi /mnt/efi/boot/BOOTx64.efi Binary files /boot/loader.efi and /mnt/efi/boot/BOOTx64.efi differ -- David Marec http://wiki.fug-fr.org/doku.php?id=start ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
Am 19.02.21 um 22:07 schrieb Warner Losh:
To avoid confusion and errors, I think a proper boot1.efifat should be put
back into the base system so that people don't have to track the above
recipe (which is likely to change).
There's no safe way to do this. The old process has been deprecated because
it is dangerous and inflexible.
For upgrading from an old-style installation, the new process is actually
fairly simple:
mount -t msdos /dev/daXsY /mnt
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
umount /mnt
Though if you're using eftbootmgr to select things, you'll need to copy to
efi/freebsd/loader.efi. Different BIOS vendors have a number of different
bugs that make this trickier than it should be, alas.
I have been using the attached script to update book blocks on EFI
or GPT ZFS MBR boot disks for quite some time.
More partition schemes and boot block types could be supported by
extending the covered cases and/or passing the partition type (GPT
vs. MBR).
Regards, STefan
#!/bin/sh
MOUNTPOINT=/.efi
BOOT1=/boot/pmbr
BOOT2=/boot/gptzfsboot
BOOTEFI=/boot/loader.efi
update_gptzfsboot ()
{
disk=$1
partname=$2
if strings /dev/$partname | grep -q ZFS; then
index=${partname##*p}
gpart bootcode -b $BOOT1 -p $BOOT2 -i $index $disk
fi
}
update_efiboot ()
{
disk=$1
partname=$2
efidir=$MOUNTPOINT/efi/boot
umount $MOUNTPOINT 2>/dev/null
if mount -t msdos "/dev/$partname" $MOUNTPOINT; then
mkdir -p $efidir
rm -f $efidir/BOOTx64o.efi
mv $efidir/BOOTx64.efi $efidir/BOOTx64o.efi
cp $BOOTEFI /.efi/efi/boot/BOOTx64.efi
umount $MOUNTPOINT
fi
}
while read line
do
set - $line
if [ "$1" = "=>" ]; then
disk=$4
partscheme=$5
else
partname=$3
parttype=$4
case $parttype in
efi)
update_efiboot $disk $partname
;;
freebsd-boot)
update_gptzfsboot $disk $partname
;;
esac
fi
done <<*EOF
$(gpart show -p)
*EOF
OpenPGP_signature
Description: OpenPGP digital signature
Re: FreeBSD 13/stable and zpool upgrade
Hi! > We can't do that. gptzfsboot is for something else that we can't get rid > of: BIOS/CMS booting. It's never been used for EFI booting at all. There's > no way to for EFI to use it, nor is there anyway for us to build it to just > work. You have to copy BOOTx64.efi to your ESP. What we could do, but don't > currently, is automate this process. We do not need it automated. We need it to be described in enough detail that we can write that BOOTx64.efi to the proper place. If there are some steps to find out where to write it etc., fine. Describe those steps. But mentioning a vague solution without more details lets far more unexperienced people try to fix complex issues, which will end in desasters. > That's been bogged down in > implementation details. It's easy if there's only one, but if you have a > USB stick installed, you don't want that to accidentally be upgraded, for > example... and I have test boxes with a dozen ESPs to test different boot > scenarios... And multiboot systems also can be screwed up by automatically > copying things into the ESP... Then, before automating it, give us enough details so that we can at least do it manually. -- p...@freebsd.org +49 171 3101372 Now what ? ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
Hi! > You can upgrade to 13 w/o hassle. The upgrade process doesn't automatically > upgrade the zpools. The only problem is if you also do a 'zpool upgrade' > which will change your zpool features. The fix is easy: upgrade your boot > loader (bootx64.efi) on the ESP at the same time before rebooting... In the past there was this nice warning explaining what to do: gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0 Now, what would be the new procedure for that ? > There's nothing really to fix here. Well, it's very dangerous, and having a way to know how to fix it would be very important. > All the code is there. It all works. > The upgrade is fail safe up until the point you do the zpool upgrade, which > has to be done by the system administrator. It's not done in the installer > or as part of 'make installworld'. I understand. But if the installer does not know that she/he needs to do something with some bootx64.efi file... ? Can't zpool upgrade mention the necessary steps, as it did before ? -- p...@freebsd.org +49 171 3101372 Now what ? ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Fri, Feb 19, 2021 at 1:49 PM Greg Rivers via freebsd-stable <
freebsd-stable@freebsd.org> wrote:
> On Friday, 19 February 2021 12:24:24 CST Kurt Jaeger wrote:
> > > > Unfortunately it contains an old version of the boot loader:
> > [...]
> > > We should be better about upgrading boot blocks, but EFI is kinda new
> and
> > > kinda different than the other out-of-root-filesystem boot blocks
> we've had
> > > in the past, so there's still some rough edges.
> >
> > We run many systems at remote sites. If we can't remotely upgrade
> > without drama from 12.x to 13.x, that would be a catastrophe.
> >
> > Any chance this can be fixed before 13.0-REL ?
> >
> Informed by reading the bsdinstall(8) scripts, I've been creating my own
> boot1.efifat thusly:
>
> #!/usr/bin/env sh
> TMPDIR=/mnt
> rm -f /boot/boot1.efifat
> newfs_msdos -F 32 -c 1 -L EFISYS -C 200m /boot/boot1.efifat
> MD=$(mdconfig -f /boot/boot1.efifat)
> mount -t msdosfs /dev/${MD} ${TMPDIR}
> mkdir -p ${TMPDIR}/efi/boot ${TMPDIR}/efi/freebsd
> cp -a /boot/loader.efi ${TMPDIR}/efi/boot/bootx64.efi
> cp -a /boot/loader.efi ${TMPDIR}/efi/freebsd/
> umount ${TMPDIR}
> mdconfig -d -u ${MD}
>
> Then I use `gpart bootcode -p /boot/boot1.efifat ...` as usual.
>
> To avoid confusion and errors, I think a proper boot1.efifat should be put
> back into the base system so that people don't have to track the above
> recipe (which is likely to change).
>
There's no safe way to do this. The old process has been deprecated because
it is dangerous and inflexible.
For upgrading from an old-style installation, the new process is actually
fairly simple:
mount -t msdos /dev/daXsY /mnt
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
umount /mnt
Though if you're using eftbootmgr to select things, you'll need to copy to
efi/freebsd/loader.efi. Different BIOS vendors have a number of different
bugs that make this trickier than it should be, alas.
Warner
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Fri, Feb 19, 2021 at 1:41 PM David Marec wrote: > Hi, > > On 19/02/2021 19:24, Kurt Jaeger wrote: > > > I suspect that your 'zpool upgrade' enabled things that weren't enabled > > before. This caused the old boot blocks to no longer work. > > Correct. That' s definitely the source of the booting issue. > > > We should be better about upgrading boot blocks, but EFI is kinda new > and > > kinda different > > EFI is able to boot any FreeBSD box for a while. > > The main issue is that the legacy way to upgrade these /bootcode and > partcode/ not only does not work, but do bad things. > > # gpart bootcode -p /boot/gptzfsboot -i 1 ada0s1 > > will install an old and inappropriate /partcode/. > Yes. it was a mistake to try to make EFI fit into that paradigm originally. All the efifat stuff should be gone, but you misunderstand what gptzfsboot is or can do. When booting with EFI, the above command will have no effect because those boot blocks will be ignored. IMO, > 'gptzfsboot' should be sweep off along with 'boot1.efifat' ( by calling > 'make delete-old') or be built with the right 'BOOTx64.EFI', which > actually is`/boot/loader.efi` . > We can't do that. gptzfsboot is for something else that we can't get rid of: BIOS/CMS booting. It's never been used for EFI booting at all. There's no way to for EFI to use it, nor is there anyway for us to build it to just work. You have to copy BOOTx64.efi to your ESP. What we could do, but don't currently, is automate this process. That's been bogged down in implementation details. It's easy if there's only one, but if you have a USB stick installed, you don't want that to accidentally be upgraded, for example... and I have test boxes with a dozen ESPs to test different boot scenarios... And multiboot systems also can be screwed up by automatically copying things into the ESP... Warner ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Fri, Feb 19, 2021 at 11:24 AM Kurt Jaeger wrote: > Hi! > > > > Unfortunately it contains an old version of the boot loader: > [...] > > We should be better about upgrading boot blocks, but EFI is kinda new and > > kinda different than the other out-of-root-filesystem boot blocks we've > had > > in the past, so there's still some rough edges. > > We run many systems at remote sites. If we can't remotely upgrade > without drama from 12.x to 13.x, that would be a catastrophe. > > Any chance this can be fixed before 13.0-REL ? > You can upgrade to 13 w/o hassle. The upgrade process doesn't automatically upgrade the zpools. The only problem is if you also do a 'zpool upgrade' which will change your zpool features. The fix is easy: upgrade your boot loader (bootx64.efi) on the ESP at the same time before rebooting... There's nothing really to fix here. All the code is there. It all works. The upgrade is fail safe up until the point you do the zpool upgrade, which has to be done by the system administrator. It's not done in the installer or as part of 'make installworld'. Warner ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Friday, 19 February 2021 12:24:24 CST Kurt Jaeger wrote:
> > > Unfortunately it contains an old version of the boot loader:
> [...]
> > We should be better about upgrading boot blocks, but EFI is kinda new and
> > kinda different than the other out-of-root-filesystem boot blocks we've had
> > in the past, so there's still some rough edges.
>
> We run many systems at remote sites. If we can't remotely upgrade
> without drama from 12.x to 13.x, that would be a catastrophe.
>
> Any chance this can be fixed before 13.0-REL ?
>
Informed by reading the bsdinstall(8) scripts, I've been creating my own
boot1.efifat thusly:
#!/usr/bin/env sh
TMPDIR=/mnt
rm -f /boot/boot1.efifat
newfs_msdos -F 32 -c 1 -L EFISYS -C 200m /boot/boot1.efifat
MD=$(mdconfig -f /boot/boot1.efifat)
mount -t msdosfs /dev/${MD} ${TMPDIR}
mkdir -p ${TMPDIR}/efi/boot ${TMPDIR}/efi/freebsd
cp -a /boot/loader.efi ${TMPDIR}/efi/boot/bootx64.efi
cp -a /boot/loader.efi ${TMPDIR}/efi/freebsd/
umount ${TMPDIR}
mdconfig -d -u ${MD}
Then I use `gpart bootcode -p /boot/boot1.efifat ...` as usual.
To avoid confusion and errors, I think a proper boot1.efifat should be put back
into the base system so that people don't have to track the above recipe (which
is likely to change).
--
Greg
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
Hi! > > Unfortunately it contains an old version of the boot loader: [...] > We should be better about upgrading boot blocks, but EFI is kinda new and > kinda different than the other out-of-root-filesystem boot blocks we've had > in the past, so there's still some rough edges. We run many systems at remote sites. If we can't remotely upgrade without drama from 12.x to 13.x, that would be a catastrophe. Any chance this can be fixed before 13.0-REL ? -- p...@opsec.eu+49 171 3101372Now what ? ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 13/stable and zpool upgrade
On Fri, Feb 19, 2021 at 8:13 AM David Marec wrote: > I have just upgrade one machine from 12-stable to 13-stable. > > Everything runs fine until the main ZFS pool was upgraded. > > Then the box stopped booting. > > Thanks to a FreeBSD-13 Beta2 usb stick, I was able to fix the issue by > copying `BOOTx64.efi` from the stick to the hard-drive. > > Looking to `/boot` to check out for a new EFI boot-loader available, I > noticed that `boot1.efifat` was removed, but `gptboot.efifat` is still > there. > > Unfortunately it contains an old version of the boot loader: > > root@machine:/boot # mdconfig -f gptboot.efifat -u 0 > root@machine:/boot # mount -t msdosfs /dev/md0 /mnt > root@machine:/boot # ll /mnt/efi/boot/ > total 385 > -rwxr-xr-x 1 root wheel 393216 Apr 16 2018 BOOTx64.efi* > -rwxr-xr-x 1 root wheel 12 Apr 16 2018 startup.nsh* > > The boot firmware I got from the 13-Beta2 installer is different: > > -rwxr-xr-x 1 root wheel 894464 Feb 18 21:07 BOOTx64.efi* > > What did I missed ? An error message would be more helpful :) However, I think I know what's going on. The old boot blocks have a filter on the enabled features of the zpool. If anything that's not in the allowed list is enabled, it won't boot off that pool. I suspect that your 'zpool upgrade' enabled things that weren't enabled before. This caused the old boot blocks to no longer work. We should be better about upgrading boot blocks, but EFI is kinda new and kinda different than the other out-of-root-filesystem boot blocks we've had in the past, so there's still some rough edges. You should be able to update bootx64.efi to the one that's in the 13-beta2 installer and the system should boot off your zpool. Warner ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
