Re: 12.1p7 no longer boots after doing zpool upgrade -a

[John Kennedy]

On Fri, Jul 10, 2020 at 02:25:02PM -0500, Kyle Evans wrote:
> >   So one recipe doesn't even seem to make the freebsd-boot partition, so is 
> > it
> > optional for a pure UEFI boot?  Should we always gpart-bootcode it if it 
> > exists
> > and upgrade BOOTx64.efi when the EFI partition exists, or is there a few 
> > more
> > wrinkles we need to worry about?
> 
> Correct, freebsd-boot is not needed for a pure UEFI boot. I wouldn't
> necessarily bother updating the freebsd-boot partition unless you
> suspect you may need to switch back to legacy boot at some point; UEFI
> is now rock-solid on all of my systems, so I've personally found no
> such need and on many of them I've removed the old freebsd-boot bits.
> 
> If you've got an ESP, you should update that manually. If you want to
> maintain the option of legacy boot, you should use gpart-bootcode for
> that but don't use it on the ESP with boot1.efifat.

  I don't know why I would.  Its pretty hard to find a non-UEFI motherboard
these days and, as you've said, it's been pretty solid.  I don't think I said
it initially, but this originally started off of 12.0 boot media (~2019/5/31),
rather than me partitioning it by hand or some more modern 12.1+ stick.

> >   In any case, is it a logical theory that my possible boot-order problem
> > is because drive order got swapped and maybe one wasn't properly updated?
> > They seem to be the same:
> >
> > # md5 /dev/nvd[01]p2
> > MD5 (/dev/nvd0p2) = 2ded438a2c97ea551446cc2d1d3b498e
> > MD5 (/dev/nvd1p2) = 2ded438a2c97ea551446cc2d1d3b498e
> >
> >   Ideally I'd like to have not boot through the UEFI boot menu every time.
> >
> >   I'm not sure why the drive order seems to matter right now.
> >
> 
> When you get booted back to the UEFI menu, is it a specific drive that
> you must select or do both equally work from that point?

  My options are basically M.2_1 and M.2_2 (not sure what labeled them, I think
that is the exact name, can confirm later).  #2 is currently the "first" drive,
and changing boot order in the BIOS to favor #1 doesn't seem to fix that.

  As I said, the motherboard died and the drives were moved to a new system
and I suspect that the order was swapped.  I never had a reason to question
my ability to boot from the 2nd disk, and hadn't done anything that merited
an upgrade that made me look at it this closely.

  Before I figured out what was going on, it would work some of the time.  I
suspect a race condition (where sometimes #1 would "win"), but I can't find
something that makes #2 a bad initial boot disk.  I don't see anything in
the boot messages that say which drive it had glommed onto.

  I can't say that I've tried to select #2.  I can try that later on today.
It's doing my weekly upgrade-build right now.

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Guido van Rooij]

On Fri, Jul 10, 2020 at 02:28:12PM -0500, Kyle Evans wrote:
> >
> > There was one question I forgot to ask:
> > Could I have known that my method of updating the ESP was not correct?
> > If so, where is this documented?
> >
> 
> It is probably unlikely, to be honest. I don't know that we do a good
> job of advising people/documenting how to update the ESP. In fact, I
> have no idea where any of it's documented except the wiki:
> https://wiki.freebsd.org/UEFI
> 

Hi Kyle,

I think that if the "new" method is stable as you say, the changes
done in r342283 based on https://reviews.freebsd.org/D17947
should be MFC'ed, don't you think?
Because then boot1.efi and boot1.efifat wil not even be created
anymore and the old documentation would be clearly out of date (in
fact it would even be better when both boot1.efi and boot1.efifat
would be mentioned in ObsoleteFiles.inc)

-Guido
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Charles Sprickman via freebsd-stable]

> On Jul 10, 2020, at 2:44 PM, Guido van Rooij  wrote:
> 
> On Fri, Jul 10, 2020 at 08:29:03PM +0200, Guido van Rooij wrote:
>> On Thu, Jul 09, 2020 at 08:24:54AM -0500, Kyle Evans wrote:
>>> On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij  wrote:
 
 I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
 After that, I did:
 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
 and:
 gpart bootcode -p /boot/boot1.efifat -i 1 ada0
 gpart bootcode -p /boot/boot1.efifat -i 1 ada1
 
 Now the system no longer boots from either disk and drops to the efi shell.
>>> 
>>> This method of updating the ESP is no longer recommended for new 12.x
>>> installations -- we now more carefully construct the ESP with an
>>> /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
>>> want to rebuild this as such, and that may fix part of your problem.
>> 
>> Hi Kyle,
>> 
>> Thnaks for your asnwer. I have not got it to work with that
>> configuration. What did work was to replace the  /efi/boot/BOOTx64.efi
>> with loader.efi and and change the content of startup.nsh with
>> loader.efi. Withoyt the above answer I wouldn't have figure it out
>> that quickly so thanks!
>> 
>> I will investigate further once I have more time (early next week probably).
> 
> There was one question I forgot to ask:
> Could I have known that my method of updating the ESP was not correct?
> If so, where is this documented?

+1

I have a few new servers and I figured that to keep with the times I should be 
doing EFI instead of legacy boot, but if this is one of those features where 
you have to be subbed to a number of email lists to know what state the feature 
is in, then maybe legacy boot is the right direction.

I don’t see any upgrade tips here:

https://www.freebsd.org/cgi/man.cgi?query=uefi=8=freebsd-release-ports

This handbook seems to have no upgrade info:

https://www.freebsd.org/doc/handbook/updating-upgrading.html
https://www.freebsd.org/doc/handbook/boot-introduction.html
https://www.freebsd.org/doc/handbook/book.html

And I’m still not sure if the wiki is considered a canonical source for this 
info or if it’s more of a developer’s notebook.

Thanks,

Charles

> 
> -Guido
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Kyle Evans]

On Fri, Jul 10, 2020 at 1:44 PM Guido van Rooij  wrote:
>
> On Fri, Jul 10, 2020 at 08:29:03PM +0200, Guido van Rooij wrote:
> > On Thu, Jul 09, 2020 at 08:24:54AM -0500, Kyle Evans wrote:
> > > On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij  wrote:
> > > >
> > > > I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
> > > > After that, I did:
> > > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
> > > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
> > > > and:
> > > > gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> > > > gpart bootcode -p /boot/boot1.efifat -i 1 ada1
> > > >
> > > > Now the system no longer boots from either disk and drops to the efi 
> > > > shell.
> > >
> > > This method of updating the ESP is no longer recommended for new 12.x
> > > installations -- we now more carefully construct the ESP with an
> > > /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
> > > want to rebuild this as such, and that may fix part of your problem.
> >
> > Hi Kyle,
> >
> > Thnaks for your asnwer. I have not got it to work with that
> > configuration. What did work was to replace the  /efi/boot/BOOTx64.efi
> > with loader.efi and and change the content of startup.nsh with
> > loader.efi. Withoyt the above answer I wouldn't have figure it out
> > that quickly so thanks!
> >
> > I will investigate further once I have more time (early next week probably).
>
> There was one question I forgot to ask:
> Could I have known that my method of updating the ESP was not correct?
> If so, where is this documented?
>

It is probably unlikely, to be honest. I don't know that we do a good
job of advising people/documenting how to update the ESP. In fact, I
have no idea where any of it's documented except the wiki:
https://wiki.freebsd.org/UEFI

>From your previous e-mail:

> I did not set this sysctl so most likely that was the cause. As the system
> is up and running again I didn't test it.
> Let me know if you want me to.

It's working now, so I wouldn't bother- this is probably the most
likely cause, the pool's demonstrably intact based on your latest
report.

Thanks,

Kyel EVans
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Kyle Evans]

On Thu, Jul 9, 2020 at 4:20 PM John Kennedy  wrote:
>
> On Thu, Jul 09, 2020 at 08:24:54AM -0500, Kyle Evans wrote:
> > On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij  wrote:
> > >
> > > I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
> > > After that, I did:
> > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
> > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
> > > and:
> > > gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> > > gpart bootcode -p /boot/boot1.efifat -i 1 ada1
> > >
> > > Now the system no longer boots from either disk and drops to the efi 
> > > shell.
> >
> > This method of updating the ESP is no longer recommended for new 12.x
> > installations -- we now more carefully construct the ESP with an
> > /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
> > want to rebuild this as such, and that may fix part of your problem.
>
>   So, I've got a stable box:
>
> FreeBSD 12.1-STABLE #156 r362983+cd7a9efd329c(stable/12): Tue Jul  7 
> 07:42:17 PDT 2020
>
>   I've got almost the same situation, except I haven't shot myself in the 
> foot yet.
> My partitions:
>
> # gpart show nvd0 nvd1
> =>   40  976773088  nvd0  GPT  (466G)
>  40 409600 1  efi  (200M)
>  409640   1024 2  freebsd-boot  (512K)
>  410664984- free -  (492K)
>  411648   16777216 3  freebsd-swap  (8.0G)
>17188864  959584256 4  freebsd-zfs  (458G)
>   976773120  8- free -  (4.0K)
>
> =>   40  976773088  nvd1  GPT  (466G)
>  40 409600 1  efi  (200M)
>  409640   1024 2  freebsd-boot  (512K)
>  410664984- free -  (492K)
>  411648   16777216 3  freebsd-swap  (8.0G)
>17188864  959584256 4  freebsd-zfs  (458G)
>   976773120  8- free -  (4.0K)
>
> [... snip ...]
>
>   If this was a Pre-UEFI, I would be expecting to do (only) something like:
>
> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 nvd0
> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 nvd1
>
>   IE, might hit either drive, update the freebsd-boot partition by # on both,
> and special note that the # is now one higher because of the EFI partition.
>
>   The FreeBSD wiki still has the "old" way of doing it here...
>
> https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot
>
>   ... but your newer way of doing it here:
>
> https://wiki.freebsd.org/UEFI
>
>   So the newer way of doing it would be something like this, except I'd by 
> updating
> the file rather than looking at it's checksum/version?
>
> strings -a < /boot/loader.efi | grep -A1 'EFI loader'
> FreeBSD/amd64 EFI loader, Revision 1.1
> (Tue Jul  7 07:33:24 PDT 2020 warl...@ouroboros.phouka.net)
>
> mount -t msdosfs /dev/nvd0p1 /mnt
> md5 -r /boot/loader.efi /mnt/efi/boot/BOOTx64.efi
> 9f64a96f3170e3327ca7dad1d574d852 /boot/loader.efi
> cbff58cc8ee79ce7342f5be23ad14ba3 /mnt/efi/boot/BOOTx64.efi
> strings -a < /mnt/efi/boot/BOOTx64.efi | grep -A1 'EFI loader'
> FreeBSD/amd64 EFI loader, Revision 1.1
> |/-\
> umount -v /mnt
>
> mount -t msdosfs /dev/nvd1p1 /mnt
> md5 -r /mnt/efi/boot/BOOTx64.efi
> cbff58cc8ee79ce7342f5be23ad14ba3 /mnt/efi/boot/BOOTx64.efi
> umount -v /mnt
>
>   At least some of the checksum difference is because 
> WITHOUT_REPRODUCIBLE_BUILD=YES.
>

Yes, this looks correct to me.

>   (Sorry for extra details, but I know when I was researching this, I ran
>across a *lot* of examples where we didn't have a GPT dump and people
>were just using partition #s and if you stomp on the wrong partition
>you're going to have a bad day, so leaving some specific examples here.)
>
>   So one recipe doesn't even seem to make the freebsd-boot partition, so is it
> optional for a pure UEFI boot?  Should we always gpart-bootcode it if it 
> exists
> and upgrade BOOTx64.efi when the EFI partition exists, or is there a few more
> wrinkles we need to worry about?
>

Correct, freebsd-boot is not needed for a pure UEFI boot. I wouldn't
necessarily bother updating the freebsd-boot partition unless you
suspect you may need to switch back to legacy boot at some point; UEFI
is now rock-solid on all of my systems, so I've personally found no
such need and on many of them I've removed the old freebsd-boot bits.

If you've got an ESP, you should update that manually. If you want to
maintain the option of legacy boot, you should use gpart-bootcode for
that but don't use it on the ESP with boot1.efifat.

>   In any case, is it a logical theory that my possible boot-order problem
> is because drive order got swapped and maybe one wasn't properly updated?
> They seem to 

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Guido van Rooij]

On Fri, Jul 10, 2020 at 08:29:03PM +0200, Guido van Rooij wrote:
> On Thu, Jul 09, 2020 at 08:24:54AM -0500, Kyle Evans wrote:
> > On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij  wrote:
> > >
> > > I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
> > > After that, I did:
> > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
> > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
> > > and:
> > > gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> > > gpart bootcode -p /boot/boot1.efifat -i 1 ada1
> > >
> > > Now the system no longer boots from either disk and drops to the efi 
> > > shell.
> > 
> > This method of updating the ESP is no longer recommended for new 12.x
> > installations -- we now more carefully construct the ESP with an
> > /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
> > want to rebuild this as such, and that may fix part of your problem.
> 
> Hi Kyle,
> 
> Thnaks for your asnwer. I have not got it to work with that
> configuration. What did work was to replace the  /efi/boot/BOOTx64.efi
> with loader.efi and and change the content of startup.nsh with
> loader.efi. Withoyt the above answer I wouldn't have figure it out
> that quickly so thanks!
> 
> I will investigate further once I have more time (early next week probably).

There was one question I forgot to ask:
Could I have known that my method of updating the ESP was not correct?
If so, where is this documented?

-Guido
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Guido van Rooij]

On Thu, Jul 09, 2020 at 08:24:54AM -0500, Kyle Evans wrote:
> On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij  wrote:
> >
> > I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
> > After that, I did:
> > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
> > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
> > and:
> > gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> > gpart bootcode -p /boot/boot1.efifat -i 1 ada1
> >
> > Now the system no longer boots from either disk and drops to the efi shell.
> 
> This method of updating the ESP is no longer recommended for new 12.x
> installations -- we now more carefully construct the ESP with an
> /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
> want to rebuild this as such, and that may fix part of your problem.

Hi Kyle,

Thnaks for your asnwer. I have not got it to work with that
configuration. What did work was to replace the  /efi/boot/BOOTx64.efi
with loader.efi and and change the content of startup.nsh with
loader.efi. Withoyt the above answer I wouldn't have figure it out
that quickly so thanks!

I will investigate further once I have more time (early next week probably).

> 
> > From memory, the partitions on ada0 and ada1 are as folows:
> >
> > 1 efi
> > 2 freebsd-boot
> > 3 freebsd-swap
> > 4 freebsd
> > p4 is a geli partition with zfs in it.
> > ada0p4.eli and ada1p4.eli form a mirrored zpool (called zroot)..
> >
> > The boot process no longer asks for a GELi password. It just isues
> > a no zfs pools found.
> >
> > When I boot from a stick, it does ask for the GELi password. When
> > I set currdev correctly in the loader, I am able to load the kernel,
> > opensolaris.ko and zfs.ko. However booting failed because I get an
> > Error: Solaris: NOTICE: Cannot find the pool label for 'zroot'
> >
> 
> Was vfs.root.mountfrom set correctly? My completely uneducated guess
> is that it likely wasn't and we couldn't locate the zpool.cache -- the
> value of vfs.root.mountfrom should basically match the currdev format
> here, except without a trailing colon.

I did not set this sysctl so most likely that was the cause. As the system 
is up and running again I didn't test it.
Let me know if you want me to.

-Guido
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[John Kennedy]

On Thu, Jul 09, 2020 at 08:24:54AM -0500, Kyle Evans wrote:
> On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij  wrote:
> >
> > I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
> > After that, I did:
> > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
> > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
> > and:
> > gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> > gpart bootcode -p /boot/boot1.efifat -i 1 ada1
> >
> > Now the system no longer boots from either disk and drops to the efi shell.
> 
> This method of updating the ESP is no longer recommended for new 12.x
> installations -- we now more carefully construct the ESP with an
> /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
> want to rebuild this as such, and that may fix part of your problem.

  So, I've got a stable box:

FreeBSD 12.1-STABLE #156 r362983+cd7a9efd329c(stable/12): Tue Jul  7 
07:42:17 PDT 2020

  I've got almost the same situation, except I haven't shot myself in the foot 
yet.
My partitions:

# gpart show nvd0 nvd1
=>   40  976773088  nvd0  GPT  (466G)
 40 409600 1  efi  (200M)
 409640   1024 2  freebsd-boot  (512K)
 410664984- free -  (492K)
 411648   16777216 3  freebsd-swap  (8.0G)
   17188864  959584256 4  freebsd-zfs  (458G)
  976773120  8- free -  (4.0K)

=>   40  976773088  nvd1  GPT  (466G)
 40 409600 1  efi  (200M)
 409640   1024 2  freebsd-boot  (512K)
 410664984- free -  (492K)
 411648   16777216 3  freebsd-swap  (8.0G)
   17188864  959584256 4  freebsd-zfs  (458G)
  976773120  8- free -  (4.0K)

  So this is my first UEFI system that's hit an upgrade point:

# zpool status
  pool: zroot
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool 
can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not 
support
the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0 days 00:02:04 with 0 errors on Wed Jul  1 
13:21:57 2020
config:

NAME  STATE READ WRITE CKSUM
zroot ONLINE   0 0 0
  nvd1p4.eli  ONLINE   0 0 0
  nvd0p4.eli  ONLINE   0 0 0

  I usually seed these things off of stick and they evolve from there, but this
one is slightly interesting because the motherboard died repeatedly and I'm
guessing that the M.2 drives got reinstalled in the "wrong" order so now if I
don't manually select one of the M.2 drives, it won't reboot (acts like the
encryption password is bad, fails out into a shell where I reboot).

  In any case, I know if I upgrade, it is going to tell me to upgrade the boot
code and I'm wondering what that is these days.

  If this was a Pre-UEFI, I would be expecting to do (only) something like:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 nvd0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 nvd1

  IE, might hit either drive, update the freebsd-boot partition by # on both,
and special note that the # is now one higher because of the EFI partition.

  The FreeBSD wiki still has the "old" way of doing it here...

https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot

  ... but your newer way of doing it here:

https://wiki.freebsd.org/UEFI

  So the newer way of doing it would be something like this, except I'd by 
updating
the file rather than looking at it's checksum/version?

strings -a < /boot/loader.efi | grep -A1 'EFI loader'
FreeBSD/amd64 EFI loader, Revision 1.1
(Tue Jul  7 07:33:24 PDT 2020 warl...@ouroboros.phouka.net)

mount -t msdosfs /dev/nvd0p1 /mnt
md5 -r /boot/loader.efi /mnt/efi/boot/BOOTx64.efi
9f64a96f3170e3327ca7dad1d574d852 /boot/loader.efi
cbff58cc8ee79ce7342f5be23ad14ba3 /mnt/efi/boot/BOOTx64.efi
strings -a < /mnt/efi/boot/BOOTx64.efi | grep -A1 'EFI loader'
FreeBSD/amd64 EFI loader, Revision 1.1
|/-\
umount -v /mnt

mount -t msdosfs /dev/nvd1p1 /mnt
md5 -r /mnt/efi/boot/BOOTx64.efi
cbff58cc8ee79ce7342f5be23ad14ba3 /mnt/efi/boot/BOOTx64.efi
umount -v /mnt

  At least some of the checksum difference is because 
WITHOUT_REPRODUCIBLE_BUILD=YES.

  (Sorry for extra details, but I know when I was researching this, I ran
   across a *lot* of examples where we didn't have a GPT dump and people
   were just using partition #s and if you 

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Kyle Evans]

On Thu, Jul 9, 2020 at 8:50 AM Pete French  wrote:
>
>
>
> On 09/07/2020 14:36, Kyle Evans wrote:
> > We haven't quite standardized on a good process yet, IMO, but for
> > right now the correct process is to just mount the ESP and replace
> > loader.efi with your system's updated /boot/loader.efi. At some point
> > we'll standardize a mountpoint for the ESP and mount it by default as
> > is done on at least some other OS.
>
> Ok, that makes sense. Still curious as to why the other method might
> fail though - as Karl points out in the other email, its just doing a
> direct copy of the image file, so if the iamge is built correct then I
> am surprised it wont work under all circumstances.
>

Yeah, so there's two problems here:

1.) That image is not built the way >= 12.x installations operate by
default. It will install an ESP with an \EFI\BOOT\BOOTx64.EFI (or
something like this) while we now, by default, setup
\EFI\FreeBSD\loader.efi and the appropriate efibootmgr to point at it.
In the former setup, that's actually /boot/boot1.efi which may or may
not do GELI correctly in the new world order -- I don't recall, it's
kind of special.

2.) That invocation clobbers anything else that might've been trying
to reside on your ESP. If you had any kind of dual-boot setup going
(perhaps rEFInd + grub + freebsdloader) then you've immediately hosed
it by overwriting the partition completely.

Thanks,

Kyle Evans
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Pete French]

On 09/07/2020 14:36, Kyle Evans wrote:

We haven't quite standardized on a good process yet, IMO, but for
right now the correct process is to just mount the ESP and replace
loader.efi with your system's updated /boot/loader.efi. At some point
we'll standardize a mountpoint for the ESP and mount it by default as
is done on at least some other OS.


Ok, that makes sense. Still curious as to why the other method might 
fail though - as Karl points out in the other email, its just doing a 
direct copy of the image file, so if the iamge is built correct then I 
am surprised it wont work under all circumstances.


But thanks for the info - will file this away for when I have to install 
an EFI machine. cheers,


-pete.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Karl Denninger]

On 7/9/2020 09:32, Pete French wrote:
>
> On 09/07/2020 14:24, Kyle Evans wrote:
>
>>> gpart bootcode -p /boot/boot1.efifat -i 1 ada0
>>> gpart bootcode -p /boot/boot1.efifat -i 1 ada1
>>
>>
>> This method of updating the ESP is no longer recommended for new 12.x
>> installations -- we now more carefully construct the ESP with an
>> /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
>> want to rebuild this as such, and that may fix part of your problem.
>
> Out of interest, how should the ESP partition be upgraded then ? I
> dont have any EFI machines...yet. But one day I will, and I was
> assuming that an upgrade would be done using the above lines too.
>
Nope.  An EFI partition is just a "funky" MSDOS (FAT) one, really.  Thus
the upgrade of the loader on one would be just a copy onto it as with
any other file on a filesystem (e.g. mount the partition, copy the file
to the correct place, unmount it); the gpart command does a byte-copy
onto what is, for all intents and purposes, an unformatted (no
directory, etc) reserved part of the disk.

My laptop dual boot (Windows 10 / FreeBSD) is EFI and I've yet to have
to screw with the loader, but if you do then it's just a copy over. 
Windows has several times blown up my Refind install -- all the
"Feature" upgrades from Windows have a habit of resetting the BIOS boot
order which makes the machine "Windows boots immediately and only", so I
have to go back and reset it whenever Microslug looses one of those on
me.  If I had cause to update the loader for FreeBSD then I'd just mount
the partition and copy it over.

-- 
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Kyle Evans]

On Thu, Jul 9, 2020 at 8:32 AM Pete French  wrote:
>
>
>
> On 09/07/2020 14:24, Kyle Evans wrote:
>
> >> gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> >> gpart bootcode -p /boot/boot1.efifat -i 1 ada1
> >
> >
> > This method of updating the ESP is no longer recommended for new 12.x
> > installations -- we now more carefully construct the ESP with an
> > /EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
> > want to rebuild this as such, and that may fix part of your problem.
>
> Out of interest, how should the ESP partition be upgraded then ? I dont
> have any EFI machines...yet. But one day I will, and I was assuming that
> an upgrade would be done using the above lines too.
>

We haven't quite standardized on a good process yet, IMO, but for
right now the correct process is to just mount the ESP and replace
loader.efi with your system's updated /boot/loader.efi. At some point
we'll standardize a mountpoint for the ESP and mount it by default as
is done on at least some other OS.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Pete French]

On 09/07/2020 14:24, Kyle Evans wrote:


gpart bootcode -p /boot/boot1.efifat -i 1 ada0
gpart bootcode -p /boot/boot1.efifat -i 1 ada1



This method of updating the ESP is no longer recommended for new 12.x
installations -- we now more carefully construct the ESP with an
/EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
want to rebuild this as such, and that may fix part of your problem.


Out of interest, how should the ESP partition be upgraded then ? I dont 
have any EFI machines...yet. But one day I will, and I was assuming that 
an upgrade would be done using the above lines too.


-pete.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 12.1p7 no longer boots after doing zpool upgrade -a

[Kyle Evans]

On Thu, Jul 9, 2020 at 8:12 AM Guido van Rooij  wrote:
>
> I did a zpool upgrade -a to enable large_dnode and spacemap_v2.
> After that, I did:
> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0
> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1
> and:
> gpart bootcode -p /boot/boot1.efifat -i 1 ada0
> gpart bootcode -p /boot/boot1.efifat -i 1 ada1
>
> Now the system no longer boots from either disk and drops to the efi shell.

This method of updating the ESP is no longer recommended for new 12.x
installations -- we now more carefully construct the ESP with an
/EFI/FreeBSD/loader.efi where loader.efi is /boot/loader.efi. You will
want to rebuild this as such, and that may fix part of your problem.

> From memory, the partitions on ada0 and ada1 are as folows:
>
> 1 efi
> 2 freebsd-boot
> 3 freebsd-swap
> 4 freebsd
> p4 is a geli partition with zfs in it.
> ada0p4.eli and ada1p4.eli form a mirrored zpool (called zroot)..
>
> The boot process no longer asks for a GELi password. It just isues
> a no zfs pools found.
>
> When I boot from a stick, it does ask for the GELi password. When
> I set currdev correctly in the loader, I am able to load the kernel,
> opensolaris.ko and zfs.ko. However booting failed because I get an
> Error: Solaris: NOTICE: Cannot find the pool label for 'zroot'
>

Was vfs.root.mountfrom set correctly? My completely uneducated guess
is that it likely wasn't and we couldn't locate the zpool.cache -- the
value of vfs.root.mountfrom should basically match the currdev format
here, except without a trailing colon.

Thanks,

Kyle Evans
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"