Re: UEFI booting survey

2017-12-19 Thread Warner Losh
On Tue, Dec 19, 2017 at 9:06 PM, Mark Millard  wrote:

> [The usdcard in my rpi2 example does not contain any kernel files,
> nor any dtb files. Only the USB SSD stick does. The kernel and dtb
> do not come from the usdcard. This is what I'm not sure if it is
> generally supported or not.]
>
> On 2017-Dec-19, at 7:26 PM, Warner Losh  wrote:
>
> > On Dec 19, 2017 4:26 PM, "Mark Millard"  wrote:
> >
> >> . . .
> >> It sounds different then the results I get with ubldr.bin
> >> on a rpi2 V1.1 . With the usdcard having a UFS / with
> >> basically only:
> >>
> >> /etc/fstab (redirecting to a USB SSD stick)
> >> /boot/* (with /boot/kernel/ empty and /boot/dtb/ empty)
> >>
> >> the result is that all 3 of the following come from the
> >> USB SSD stick based on the "/" line from the /etc/fstab
> >> from the usdcard:
> >>
> >> /boot/kernel
> >> /boot/dtb/bcm2836-rpi-2-b.dtb
> >> / (mounted root file system)
> >>
> >> In other words: it appears that for ubldr.bin on
> >> a rpi2 V1.1 /etc/fstab is read and used before
> >> finding the kernel that is to be loaded. (It or
> >> another /etc/fstab may be read again later.)
> >>
> >> It is read. It is literally only used to set the root for userland.
> That is all. Nothing else.
> >>
> >> /usr/src/stand/common/boot.c does show an explicit
> >> attempt to find a /etc/fstab:
> >>
> >> # grep -r /etc/fstab /usr/src/stand/
> >> . . .
> >> /usr/src/stand/common/boot.c: * Try to find the /etc/fstab file on the
> filesystem (rootdev),
> >> /usr/src/stand/common/boot.c:sprintf(lbuf, "%s/etc/fstab", rootdev);
> >> . . .
> >>
> >> That is from getrootmount(char *rootdev):
> >>
> >> int
> >> getrootmount(char *rootdev)
> >> {
> >> charlbuf[128], *cp, *ep, *dev, *fstyp, *options;
> >> int fd, error;
> >>
> >> if (getenv("vfs.root.mountfrom") != NULL)
> >> return(0);
> >>
> >> So if you set vfs.root.mountfrom in /boot/loader.conf, we don't read
> rootdev:/etc/fstab for the value to set vfs.root.mountfrom to.
> >>
> >> error = 1;
> >> sprintf(lbuf, "%s/etc/fstab", rootdev);
> >> if ((fd = open(lbuf, O_RDONLY)) < 0)
> >> goto notfound;
> >> . . .
> >>
> >> Supporting detail for the example rpi2
> >> boot context:
> >>
> >> With /mnt being the / from the usdcard:
> >>
> >> # find /mnt -print | more
> >> /mnt
> >> /mnt/.snap
> >> /mnt/boot
> >> /mnt/boot/defaults
> >> /mnt/boot/defaults/loader.conf
> >> /mnt/boot/dtb
> >> /mnt/boot/firmware
> >> /mnt/boot/kernel
> >> /mnt/boot/modules
> >> /mnt/boot/zfs
> >> /mnt/boot/msdos
> >> /mnt/boot/entropy
> >> /mnt/boot/menu.rc.sample
> >> /mnt/boot/ubldr
> >> /mnt/boot/ubldr.bin
> >> /mnt/boot/brand-fbsd.4th
> >> /mnt/boot/logo-beastie.4th
> >> /mnt/boot/logo-beastiebw.4th
> >> /mnt/boot/logo-fbsdbw.4th
> >> /mnt/boot/logo-orb.4th
> >> /mnt/boot/logo-orbbw.4th
> >> /mnt/boot/loader.conf
> >> /mnt/boot/loader.efi
> >> /mnt/boot/boot1.efi
> >> /mnt/boot/boot1.efifat
> >> /mnt/boot/beastie.4th
> >> /mnt/boot/brand.4th
> >> /mnt/boot/color.4th
> >> /mnt/boot/check-password.4th
> >> /mnt/boot/delay.4th
> >> /mnt/boot/frames.4th
> >> /mnt/boot/loader.4th
> >> /mnt/boot/loader.help
> >> /mnt/boot/menu.4th
> >> /mnt/boot/menu-commands.4th
> >> /mnt/boot/menusets.4th
> >> /mnt/boot/screen.4th
> >> /mnt/boot/shortcuts.4th
> >> /mnt/boot/support.4th
> >> /mnt/boot/version.4th
> >> /mnt/boot/loader.rc
> >> /mnt/boot/efi.4th
> >> /mnt/boot/pcibios.4th
> >> /mnt/boot/menu.rc
> >> /mnt/etc
> >> /mnt/etc/fstab
> >> /mnt/COPYRIGHT
> >> /mnt/lost+found
> >>
> >> # more /mnt/etc/fstab
> >> /dev/da0p1  /   ufs rw,noatime  1 1
> >> /dev/da0p2  noneswapsw  0 0
> >>
> >> May be this is somehow special to the rpi2 or to
> >> ubldr.bin operation. (I've never managed to identify
> >> accidents from deliberately working status in this
> >> area.)
> >>
> > So you load /boot/loader and the kernel from the sdcard.
>
> No: There are no kernel files on the usdcard. See the complete
> file list of its only UFS partition above. No dtb files either.
>
> [On a rpi2 ubldr.bin is copied to the msdosfs, where it
> is actually put to use.]


OK. Looks like the uboot code has the same bogus 'let's search everything'
code that I'm removing from the UEFI case. Still doesn't get anything from
/etc/fstab. It can't. There's no translation code in the boot loader to try
to guess. It just does a bruit force plow through all the devices and hopes
for the best.


> > /etc/fstab on the card points to the usb drive for /.
>
> True. And that is were the kernel and dtb come from,
> not the usdcard.
>

Right, but that's not how ubldr finds them.


> For reference loader.config has:
>
> # more /mnt/boot/loader.conf
> geom_label_load="YES"   # File system labels (see glabel(8))
> #
> kern.cam.boot_delay="1"
> vfs.mountroot.timeout="10"
> dumpdev="/dev/da0p2"
>
> (So no vfs.root.mountfrom .)


What does config.txt have?

> This is all standard loader behavior.
>
> I'm not sure 

Re: UEFI booting survey

2017-12-19 Thread Mark Millard
[The usdcard in my rpi2 example does not contain any kernel files,
nor any dtb files. Only the USB SSD stick does. The kernel and dtb
do not come from the usdcard. This is what I'm not sure if it is
generally supported or not.]

On 2017-Dec-19, at 7:26 PM, Warner Losh  wrote:

> On Dec 19, 2017 4:26 PM, "Mark Millard"  wrote:
> 
>> . . .
>> It sounds different then the results I get with ubldr.bin
>> on a rpi2 V1.1 . With the usdcard having a UFS / with
>> basically only:
>> 
>> /etc/fstab (redirecting to a USB SSD stick)
>> /boot/* (with /boot/kernel/ empty and /boot/dtb/ empty)
>> 
>> the result is that all 3 of the following come from the
>> USB SSD stick based on the "/" line from the /etc/fstab
>> from the usdcard:
>> 
>> /boot/kernel
>> /boot/dtb/bcm2836-rpi-2-b.dtb
>> / (mounted root file system)
>> 
>> In other words: it appears that for ubldr.bin on
>> a rpi2 V1.1 /etc/fstab is read and used before
>> finding the kernel that is to be loaded. (It or
>> another /etc/fstab may be read again later.)
>> 
>> It is read. It is literally only used to set the root for userland. That is 
>> all. Nothing else. 
>> 
>> /usr/src/stand/common/boot.c does show an explicit
>> attempt to find a /etc/fstab:
>> 
>> # grep -r /etc/fstab /usr/src/stand/
>> . . .
>> /usr/src/stand/common/boot.c: * Try to find the /etc/fstab file on the 
>> filesystem (rootdev),
>> /usr/src/stand/common/boot.c:sprintf(lbuf, "%s/etc/fstab", rootdev);
>> . . .
>> 
>> That is from getrootmount(char *rootdev):
>> 
>> int
>> getrootmount(char *rootdev)
>> {
>> charlbuf[128], *cp, *ep, *dev, *fstyp, *options;
>> int fd, error;
>> 
>> if (getenv("vfs.root.mountfrom") != NULL)
>> return(0);
>> 
>> So if you set vfs.root.mountfrom in /boot/loader.conf, we don't read 
>> rootdev:/etc/fstab for the value to set vfs.root.mountfrom to.
>> 
>> error = 1;
>> sprintf(lbuf, "%s/etc/fstab", rootdev);
>> if ((fd = open(lbuf, O_RDONLY)) < 0)
>> goto notfound;
>> . . .
>> 
>> Supporting detail for the example rpi2
>> boot context:
>> 
>> With /mnt being the / from the usdcard:
>> 
>> # find /mnt -print | more
>> /mnt
>> /mnt/.snap
>> /mnt/boot
>> /mnt/boot/defaults
>> /mnt/boot/defaults/loader.conf
>> /mnt/boot/dtb
>> /mnt/boot/firmware
>> /mnt/boot/kernel
>> /mnt/boot/modules
>> /mnt/boot/zfs
>> /mnt/boot/msdos
>> /mnt/boot/entropy
>> /mnt/boot/menu.rc.sample
>> /mnt/boot/ubldr
>> /mnt/boot/ubldr.bin
>> /mnt/boot/brand-fbsd.4th
>> /mnt/boot/logo-beastie.4th
>> /mnt/boot/logo-beastiebw.4th
>> /mnt/boot/logo-fbsdbw.4th
>> /mnt/boot/logo-orb.4th
>> /mnt/boot/logo-orbbw.4th
>> /mnt/boot/loader.conf
>> /mnt/boot/loader.efi
>> /mnt/boot/boot1.efi
>> /mnt/boot/boot1.efifat
>> /mnt/boot/beastie.4th
>> /mnt/boot/brand.4th
>> /mnt/boot/color.4th
>> /mnt/boot/check-password.4th
>> /mnt/boot/delay.4th
>> /mnt/boot/frames.4th
>> /mnt/boot/loader.4th
>> /mnt/boot/loader.help
>> /mnt/boot/menu.4th
>> /mnt/boot/menu-commands.4th
>> /mnt/boot/menusets.4th
>> /mnt/boot/screen.4th
>> /mnt/boot/shortcuts.4th
>> /mnt/boot/support.4th
>> /mnt/boot/version.4th
>> /mnt/boot/loader.rc
>> /mnt/boot/efi.4th
>> /mnt/boot/pcibios.4th
>> /mnt/boot/menu.rc
>> /mnt/etc
>> /mnt/etc/fstab
>> /mnt/COPYRIGHT
>> /mnt/lost+found
>> 
>> # more /mnt/etc/fstab
>> /dev/da0p1  /   ufs rw,noatime  1 1
>> /dev/da0p2  noneswapsw  0 0
>> 
>> May be this is somehow special to the rpi2 or to
>> ubldr.bin operation. (I've never managed to identify
>> accidents from deliberately working status in this
>> area.)
>> 
> So you load /boot/loader and the kernel from the sdcard.

No: There are no kernel files on the usdcard. See the complete
file list of its only UFS partition above. No dtb files either.

[On a rpi2 ubldr.bin is copied to the msdosfs, where it
is actually put to use.]

> /etc/fstab on the card points to the usb drive for /.

True. And that is were the kernel and dtb come from,
not the usdcard.

For reference loader.config has:

# more /mnt/boot/loader.conf
geom_label_load="YES"   # File system labels (see glabel(8))
#
kern.cam.boot_delay="1"
vfs.mountroot.timeout="10"
dumpdev="/dev/da0p2"

(So no vfs.root.mountfrom .)

> This is all standard loader behavior.

I'm not sure avoiding the kernel and dtb being on the usdcard
is standard-supported behavior. It might be a lucky,
limited-context accident rather than a general property as a
technique.

> It won't change. In your case, you aren't even using UEFI, so it doubly won't 
> change.

I also have access to an rpi3 and a pine64+ 2GB, which are
UEFI based. But I'd not yet tried a similar configuration
to what I'd recently done on the rpi2 V1.1 .

The list notices made me unsure if I should even try. It
is this part that I'm trying to figure out, both for now
and for after the changes.

> UEFI has more direct ways of doing this, but this setup would work there 
> because it is explicit. It doesn't depend on the current boot

Re: UEFI booting survey

2017-12-19 Thread Warner Losh
On Dec 19, 2017 4:26 PM, "Mark Millard"  wrote:


On 2017-Dec-19, at 1:49 PM, Warner Losh  wrote:

>
>
> On Dec 19, 2017 10:58 AM, "Mark Millard"  wrote:
>> On 2017-Dec-18, at 2:37 PM, Warner Losh  wrote:
>>
>> > . . .
>> >
>> > Or the following pseudo-code with all the weird special cases removed
for clarity
>> >
>> > load loader.efi from ESP
>> > if Boot uefi variable holds a second path, use that for root/kernel
>> > otherwise if an override variable holds a kernel/root path, use that
>> > otherwise scan for a usable ZFS pool, use that if it exists
>> > otherwise use the same partition loader.efi was booted from for
root/kernel if it's usable
>> > otherwise use the first UFS partition on the ESP that's usable.
>> >
>> > A partition is usable if /boot/loader.rc exists on that path.
>>
>> What will be the role of /etc/fstab in establishing
>> were the kernel is loaded from? Where world is
>> loaded from? Where/how does use of /etc/fstab for
>> specifying the root file system mount fit in the
>> above pseudo-code?
>
> Same as today: it is what the boot loader passes to the kernel as the
Unix name of /. I have no plans to change that. It's of almost no use to
the boot loader, since it can't know what BIOS device da3 is, for example,
if that's in fstab. Or even more complex examples like /dev/mirror/primary.
Efibootmgr can take Unix devices and paths and turn them into UEFI paths so
we know what devices to use for what. In the absence of those, or an
equivalent fallback, we are quite limited in what we can do since we don't
have the context needed to translate.
>
> Warner

Okay.

It sounds different then the results I get with ubldr.bin
on a rpi2 V1.1 . With the usdcard having a UFS / with
basically only:

/etc/fstab (redirecting to a USB SSD stick)
/boot/* (with /boot/kernel/ empty and /boot/dtb/ empty)

the result is that all 3 of the following come from the
USB SSD stick based on the "/" line from the /etc/fstab
from the usdcard:

/boot/kernel
/boot/dtb/bcm2836-rpi-2-b.dtb
/ (mounted root file system)

In other words: it appears that for ubldr.bin on
a rpi2 V1.1 /etc/fstab is read and used before
finding the kernel that is to be loaded. (It or
another /etc/fstab may be read again later.)


It is read. It is literally only used to set the root for userland. That is
all. Nothing else.

/usr/src/stand/common/boot.c does show an explicit
attempt to find a /etc/fstab:

# grep -r /etc/fstab /usr/src/stand/
. . .
/usr/src/stand/common/boot.c: * Try to find the /etc/fstab file on the
filesystem (rootdev),
/usr/src/stand/common/boot.c:sprintf(lbuf, "%s/etc/fstab", rootdev);
. . .

That is from getrootmount(char *rootdev):

int
getrootmount(char *rootdev)
{
charlbuf[128], *cp, *ep, *dev, *fstyp, *options;
int fd, error;

if (getenv("vfs.root.mountfrom") != NULL)
return(0);


So if you set vfs.root.mountfrom in /boot/loader.conf, we don't read
rootdev:/etc/fstab for the value to set vfs.root.mountfrom to.

error = 1;
sprintf(lbuf, "%s/etc/fstab", rootdev);
if ((fd = open(lbuf, O_RDONLY)) < 0)
goto notfound;
. . .

Supporting detail for the example rpi2
boot context:

With /mnt being the / from the usdcard:

# find /mnt -print | more
/mnt
/mnt/.snap
/mnt/boot
/mnt/boot/defaults
/mnt/boot/defaults/loader.conf
/mnt/boot/dtb
/mnt/boot/firmware
/mnt/boot/kernel
/mnt/boot/modules
/mnt/boot/zfs
/mnt/boot/msdos
/mnt/boot/entropy
/mnt/boot/menu.rc.sample
/mnt/boot/ubldr
/mnt/boot/ubldr.bin
/mnt/boot/brand-fbsd.4th
/mnt/boot/logo-beastie.4th
/mnt/boot/logo-beastiebw.4th
/mnt/boot/logo-fbsdbw.4th
/mnt/boot/logo-orb.4th
/mnt/boot/logo-orbbw.4th
/mnt/boot/loader.conf
/mnt/boot/loader.efi
/mnt/boot/boot1.efi
/mnt/boot/boot1.efifat
/mnt/boot/beastie.4th
/mnt/boot/brand.4th
/mnt/boot/color.4th
/mnt/boot/check-password.4th
/mnt/boot/delay.4th
/mnt/boot/frames.4th
/mnt/boot/loader.4th
/mnt/boot/loader.help
/mnt/boot/menu.4th
/mnt/boot/menu-commands.4th
/mnt/boot/menusets.4th
/mnt/boot/screen.4th
/mnt/boot/shortcuts.4th
/mnt/boot/support.4th
/mnt/boot/version.4th
/mnt/boot/loader.rc
/mnt/boot/efi.4th
/mnt/boot/pcibios.4th
/mnt/boot/menu.rc
/mnt/etc
/mnt/etc/fstab
/mnt/COPYRIGHT
/mnt/lost+found

# more /mnt/etc/fstab
/dev/da0p1  /   ufs rw,noatime  1 1
/dev/da0p2  noneswapsw  0 0

May be this is somehow special to the rpi2 or to
ubldr.bin operation. (I've never managed to identify
accidents from deliberately working status in this
area.)


So you load /boot/loader and the kernel from the sdcard.  /etc/fstab on the
card points to the usb drive for /. This is all standard loader behavior.
It won't change. In your case, you aren't even using UEFI, so it doubly
won't change. UEFI has more direct ways of doing this, but this setup would
work there because it is explicit. It doesn't depend on the current
boot1.efi behavior...

Warner

>> (For my particular interest the context uses UFS, not
>> ZFS.)
>>
>> > What is being de

Re: UEFI booting survey

2017-12-19 Thread Warner Losh
On Dec 19, 2017 3:38 PM, "Oliver Pinter" 
wrote:



On Tuesday, December 19, 2017, Warner Losh  wrote:

> On Dec 18, 2017 6:06 PM, "Rodney W. Grimes" <
> freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
>
> > On Mon, Dec 18, 2017 at 3:12 PM, Mark Millard 
> wrote:
> >
> > > Warner Losh imp at bsdimp.com wrote on
> > > Mon Dec 18 20:29:45 UTC 2017 :
> > >
> > > > The specific thing we will stop doing is that in the absence of
> > > > instructions to the contrary, we will no longer search for root on a
> > > device
> > > > other than the one the loader.efi came from.
> > >
> > > Warner Losh imp at bsdimp.com wrote on
> > > Sun Dec 17 19:52:07 UTC 2017 :
> > >
> > > > In the coming months, we're looking at dropping boot1.efi and instead
> > > > installing /boot/loader.efi onto the ESP (most likely as
> > > > \efi\freebsd\loader.efi).
> > >
> > >
> > > Combining the two statements would appear to have consequences
> > > not obvious from the separate statements in isolation. Rewording
> > > the first to substitute where loader.efi comes from based on
> > > the second (if I interpret right):
> > >
> > > MISQUOTE
> > > The specific thing we will stop doing is that in the absence of
> > > instructions to the contrary, we will no longer search for root
> > > on other than the device for the ESP used (which will hold
> > > loader.efi).
> > > END MISQUOTE
> > >
> >
> > The specific thing we will stop doing is that in the absence of
> > instructions to the contrary, we will no longer search for root on other
> > than the device for the ESP used (which will hold now loader.efi as
> > boot1.efi will shortly be eliminated).
>
> Yes please, that is the correct behavior, our searching can lead to
> problems, and as you have pointed out, often more problems than it
> ever really fixed.
>
> >
> > Or the following pseudo-code with all the weird special cases removed for
> > clarity
> >
> > load loader.efi from ESP
> > if Boot uefi variable holds a second path, use that for root/kernel
> > otherwise if an override variable holds a kernel/root path, use that
> > otherwise scan for a usable ZFS pool, use that if it exists
> > otherwise use the same partition loader.efi was booted from for
> root/kernel
> > if it's usable
> > otherwise use the first UFS partition on the ESP that's usable.
>
> use the ACTIVE ufs partition, not the first, I can have more than 1 slice,
> only 1 of them can be set active.  Do not use any ufs partitions if they
> are not in active slices, it is possible to have 0 partitions set active.
>
>
> Active is not a GPT concept. UEFI makes it hard to implement since there is
> no good API to get and set the flags FreeBSD's gptboot uses to hack this
> concept in. Active is done via BootOrder UEFI variable. Loader.efi and
> boot.efi completely ignore this today. I have no plans on changing that.


And what's about the bootme and bootonce flags in gpart?  They are
freebsdism? Or they are the equivalent of active in the UEFI standard?


They are a FreeBSD hack. Not part of the standard. All the bootnext /
bootme stuff lives in UEFI variables as documented in the standard for UEFI
boot manager, not in the partitions. FreeBSD on UEFI has never honored them.

Warner


> >
> > A partition is usable if /boot/loader.rc exists on that path.
>
> A partition is usable if it is in an active slice, and ^above
>
>
> Active isn't a got thong. So no.
>
> Is there any fallback to skip loader and go direct to
> /boot/kernel/kernel, back to /kernel any more?
>
>
> You are thinking about this wrong. We are loader.efi, not boot2. This is
> one of the big advantages of loading directly. We don't have the space
> limitations that forced that design, so we should simplify.
>
> Warner
>
> > What is being deleted is one final step: "otherwise use the first UFS
> > partition on any drive in a random order that's usable." which used to be
> > at the end of the boot1.efi psuedo code. It's my belief that no such
> > installations actually use this due to the random factor today (plug in a
> > new USB drive and it might take over). If my belief is wrong, it's my
> > belief that efibootmgr will solve it, and failing that, the fallback
> > mechanism (for platforms that use u-boot + EFI where UEFI variables don't
> > work) will allow the two or three people that are doing this today.
> >
> > Warner
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscribe@
> freebsd.org"
>
> --
> Rod Grimes
> rgri...@freebsd.org
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-curr

Re: UEFI booting survey

2017-12-19 Thread Mark Millard

On 2017-Dec-19, at 1:49 PM, Warner Losh  wrote:

> 
> 
> On Dec 19, 2017 10:58 AM, "Mark Millard"  wrote:
>> On 2017-Dec-18, at 2:37 PM, Warner Losh  wrote:
>> 
>> > . . .
>> >
>> > Or the following pseudo-code with all the weird special cases removed for 
>> > clarity
>> >
>> > load loader.efi from ESP
>> > if Boot uefi variable holds a second path, use that for root/kernel
>> > otherwise if an override variable holds a kernel/root path, use that
>> > otherwise scan for a usable ZFS pool, use that if it exists
>> > otherwise use the same partition loader.efi was booted from for 
>> > root/kernel if it's usable
>> > otherwise use the first UFS partition on the ESP that's usable.
>> >
>> > A partition is usable if /boot/loader.rc exists on that path.
>> 
>> What will be the role of /etc/fstab in establishing
>> were the kernel is loaded from? Where world is
>> loaded from? Where/how does use of /etc/fstab for
>> specifying the root file system mount fit in the
>> above pseudo-code?
> 
> Same as today: it is what the boot loader passes to the kernel as the Unix 
> name of /. I have no plans to change that. It's of almost no use to the boot 
> loader, since it can't know what BIOS device da3 is, for example, if that's 
> in fstab. Or even more complex examples like /dev/mirror/primary. Efibootmgr 
> can take Unix devices and paths and turn them into UEFI paths so we know what 
> devices to use for what. In the absence of those, or an equivalent fallback, 
> we are quite limited in what we can do since we don't have the context needed 
> to translate.
> 
> Warner 

Okay.

It sounds different then the results I get with ubldr.bin
on a rpi2 V1.1 . With the usdcard having a UFS / with
basically only:

/etc/fstab (redirecting to a USB SSD stick)
/boot/* (with /boot/kernel/ empty and /boot/dtb/ empty)

the result is that all 3 of the following come from the
USB SSD stick based on the "/" line from the /etc/fstab
from the usdcard:

/boot/kernel
/boot/dtb/bcm2836-rpi-2-b.dtb
/ (mounted root file system)

In other words: it appears that for ubldr.bin on
a rpi2 V1.1 /etc/fstab is read and used before
finding the kernel that is to be loaded. (It or
another /etc/fstab may be read again later.)

/usr/src/stand/common/boot.c does show an explicit
attempt to find a /etc/fstab:

# grep -r /etc/fstab /usr/src/stand/
. . .
/usr/src/stand/common/boot.c: * Try to find the /etc/fstab file on the 
filesystem (rootdev),
/usr/src/stand/common/boot.c:sprintf(lbuf, "%s/etc/fstab", rootdev);
. . .

That is from getrootmount(char *rootdev):

int
getrootmount(char *rootdev)
{
charlbuf[128], *cp, *ep, *dev, *fstyp, *options;
int fd, error;

if (getenv("vfs.root.mountfrom") != NULL)
return(0);

error = 1;
sprintf(lbuf, "%s/etc/fstab", rootdev);
if ((fd = open(lbuf, O_RDONLY)) < 0)
goto notfound;
. . .

Supporting detail for the example rpi2
boot context:

With /mnt being the / from the usdcard:

# find /mnt -print | more
/mnt
/mnt/.snap
/mnt/boot
/mnt/boot/defaults
/mnt/boot/defaults/loader.conf
/mnt/boot/dtb
/mnt/boot/firmware
/mnt/boot/kernel
/mnt/boot/modules
/mnt/boot/zfs
/mnt/boot/msdos
/mnt/boot/entropy
/mnt/boot/menu.rc.sample
/mnt/boot/ubldr
/mnt/boot/ubldr.bin
/mnt/boot/brand-fbsd.4th
/mnt/boot/logo-beastie.4th
/mnt/boot/logo-beastiebw.4th
/mnt/boot/logo-fbsdbw.4th
/mnt/boot/logo-orb.4th
/mnt/boot/logo-orbbw.4th
/mnt/boot/loader.conf
/mnt/boot/loader.efi
/mnt/boot/boot1.efi
/mnt/boot/boot1.efifat
/mnt/boot/beastie.4th
/mnt/boot/brand.4th
/mnt/boot/color.4th
/mnt/boot/check-password.4th
/mnt/boot/delay.4th
/mnt/boot/frames.4th
/mnt/boot/loader.4th
/mnt/boot/loader.help
/mnt/boot/menu.4th
/mnt/boot/menu-commands.4th
/mnt/boot/menusets.4th
/mnt/boot/screen.4th
/mnt/boot/shortcuts.4th
/mnt/boot/support.4th
/mnt/boot/version.4th
/mnt/boot/loader.rc
/mnt/boot/efi.4th
/mnt/boot/pcibios.4th
/mnt/boot/menu.rc
/mnt/etc
/mnt/etc/fstab
/mnt/COPYRIGHT
/mnt/lost+found

# more /mnt/etc/fstab
/dev/da0p1  /   ufs rw,noatime  1 1
/dev/da0p2  noneswapsw  0 0

May be this is somehow special to the rpi2 or to
ubldr.bin operation. (I've never managed to identify
accidents from deliberately working status in this
area.)


>> (For my particular interest the context uses UFS, not
>> ZFS.)
>> 
>> > What is being deleted is one final step: "otherwise use the first UFS 
>> > partition on any drive in a random order that's usable." which used to be 
>> > at the end of the boot1.efi psuedo code. It's my belief that no such 
>> > installations actually use this due to the random factor today (plug in a 
>> > new USB drive and it might take over). If my belief is wrong, it's my 
>> > belief that efibootmgr will solve it, and failing that, the fallback 
>> > mechanism (for platforms that use u-boot + EFI where UEFI variables don't 
>> > work) will allow the two or three people that are doing this today.
>> 
> 

===
Mark Millard
mark

Re: UEFI booting survey

2017-12-19 Thread Nathan Whitehorn



On 12/19/17 14:38, Oliver Pinter wrote:

On Tuesday, December 19, 2017, Warner Losh  wrote:




[snip]

Or the following pseudo-code with all the weird special cases removed for
clarity

load loader.efi from ESP
if Boot uefi variable holds a second path, use that for root/kernel
otherwise if an override variable holds a kernel/root path, use that
otherwise scan for a usable ZFS pool, use that if it exists
otherwise use the same partition loader.efi was booted from for

root/kernel

if it's usable
otherwise use the first UFS partition on the ESP that's usable.

use the ACTIVE ufs partition, not the first, I can have more than 1 slice,
only 1 of them can be set active.  Do not use any ufs partitions if they
are not in active slices, it is possible to have 0 partitions set active.


Active is not a GPT concept. UEFI makes it hard to implement since there is
no good API to get and set the flags FreeBSD's gptboot uses to hack this
concept in. Active is done via BootOrder UEFI variable. Loader.efi and
boot.efi completely ignore this today. I have no plans on changing that.


And what's about the bootme and bootonce flags in gpart?  They are
freebsdism? Or they are the equivalent of active in the UEFI standard?



They are a FreeBSD-ism. Because UEFI handles parsing the GPT tables 
internally, it is not even possible to read them from loader.efi.

-Nathan
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: UEFI booting survey

2017-12-19 Thread Oliver Pinter
On Tuesday, December 19, 2017, Warner Losh  wrote:

> On Dec 18, 2017 6:06 PM, "Rodney W. Grimes" <
> freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
>
> > On Mon, Dec 18, 2017 at 3:12 PM, Mark Millard 
> wrote:
> >
> > > Warner Losh imp at bsdimp.com wrote on
> > > Mon Dec 18 20:29:45 UTC 2017 :
> > >
> > > > The specific thing we will stop doing is that in the absence of
> > > > instructions to the contrary, we will no longer search for root on a
> > > device
> > > > other than the one the loader.efi came from.
> > >
> > > Warner Losh imp at bsdimp.com wrote on
> > > Sun Dec 17 19:52:07 UTC 2017 :
> > >
> > > > In the coming months, we're looking at dropping boot1.efi and instead
> > > > installing /boot/loader.efi onto the ESP (most likely as
> > > > \efi\freebsd\loader.efi).
> > >
> > >
> > > Combining the two statements would appear to have consequences
> > > not obvious from the separate statements in isolation. Rewording
> > > the first to substitute where loader.efi comes from based on
> > > the second (if I interpret right):
> > >
> > > MISQUOTE
> > > The specific thing we will stop doing is that in the absence of
> > > instructions to the contrary, we will no longer search for root
> > > on other than the device for the ESP used (which will hold
> > > loader.efi).
> > > END MISQUOTE
> > >
> >
> > The specific thing we will stop doing is that in the absence of
> > instructions to the contrary, we will no longer search for root on other
> > than the device for the ESP used (which will hold now loader.efi as
> > boot1.efi will shortly be eliminated).
>
> Yes please, that is the correct behavior, our searching can lead to
> problems, and as you have pointed out, often more problems than it
> ever really fixed.
>
> >
> > Or the following pseudo-code with all the weird special cases removed for
> > clarity
> >
> > load loader.efi from ESP
> > if Boot uefi variable holds a second path, use that for root/kernel
> > otherwise if an override variable holds a kernel/root path, use that
> > otherwise scan for a usable ZFS pool, use that if it exists
> > otherwise use the same partition loader.efi was booted from for
> root/kernel
> > if it's usable
> > otherwise use the first UFS partition on the ESP that's usable.
>
> use the ACTIVE ufs partition, not the first, I can have more than 1 slice,
> only 1 of them can be set active.  Do not use any ufs partitions if they
> are not in active slices, it is possible to have 0 partitions set active.
>
>
> Active is not a GPT concept. UEFI makes it hard to implement since there is
> no good API to get and set the flags FreeBSD's gptboot uses to hack this
> concept in. Active is done via BootOrder UEFI variable. Loader.efi and
> boot.efi completely ignore this today. I have no plans on changing that.


And what's about the bootme and bootonce flags in gpart?  They are
freebsdism? Or they are the equivalent of active in the UEFI standard?


>
> >
> > A partition is usable if /boot/loader.rc exists on that path.
>
> A partition is usable if it is in an active slice, and ^above
>
>
> Active isn't a got thong. So no.
>
> Is there any fallback to skip loader and go direct to
> /boot/kernel/kernel, back to /kernel any more?
>
>
> You are thinking about this wrong. We are loader.efi, not boot2. This is
> one of the big advantages of loading directly. We don't have the space
> limitations that forced that design, so we should simplify.
>
> Warner
>
> > What is being deleted is one final step: "otherwise use the first UFS
> > partition on any drive in a random order that's usable." which used to be
> > at the end of the boot1.efi psuedo code. It's my belief that no such
> > installations actually use this due to the random factor today (plug in a
> > new USB drive and it might take over). If my belief is wrong, it's my
> > belief that efibootmgr will solve it, and failing that, the fallback
> > mechanism (for platforms that use u-boot + EFI where UEFI variables don't
> > work) will allow the two or three people that are doing this today.
> >
> > Warner
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "
> freebsd-current-unsubscr...@freebsd.org"
>
> --
> Rod Grimes
> rgri...@freebsd.org
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: UEFI booting survey

2017-12-19 Thread Warner Losh
On Dec 19, 2017 12:26 PM, "Mark Millard"  wrote:

[I forgot to list the .dtb file with the
kernel and world.]

On 2017-Dec-19, at 9:58 AM, Mark Millard  wrote:

> On 2017-Dec-18, at 2:37 PM, Warner Losh  wrote:
>
>> . . .
>>
>> Or the following pseudo-code with all the weird special cases removed
for clarity
>>
>> load loader.efi from ESP
>> if Boot uefi variable holds a second path, use that for root/kernel
>> otherwise if an override variable holds a kernel/root path, use that
>> otherwise scan for a usable ZFS pool, use that if it exists
>> otherwise use the same partition loader.efi was booted from for
root/kernel if it's usable
>> otherwise use the first UFS partition on the ESP that's usable.
>>
>> A partition is usable if /boot/loader.rc exists on that path.
>
> What will be the role of /etc/fstab in establishing
> were the kernel is loaded from? Where world is
> loaded from? Where/how does use of /etc/fstab for
> specifying the root file system mount fit in the
> above pseudo-code?

Typo: should be "establishing where".

> (For my particular interest the context uses UFS, not
> ZFS.)

I forgot to list:

What will be the role of /etc/fstab in establishing
where the .dtb file will be found (when such is
involved)?


None. See my prior message for why.

Warner

>> What is being deleted is one final step: "otherwise use the first UFS
partition on any drive in a random order that's usable." which used to be
at the end of the boot1.efi psuedo code. It's my belief that no such
installations actually use this due to the random factor today (plug in a
new USB drive and it might take over). If my belief is wrong, it's my
belief that efibootmgr will solve it, and failing that, the fallback
mechanism (for platforms that use u-boot + EFI where UEFI variables don't
work) will allow the two or three people that are doing this today.


===
Mark Millard
markmi at dsl-only.net
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: UEFI booting survey

2017-12-19 Thread Warner Losh
On Dec 19, 2017 10:58 AM, "Mark Millard"  wrote:

On 2017-Dec-18, at 2:37 PM, Warner Losh  wrote:

> . . .
>
> Or the following pseudo-code with all the weird special cases removed for
clarity
>
> load loader.efi from ESP
> if Boot uefi variable holds a second path, use that for root/kernel
> otherwise if an override variable holds a kernel/root path, use that
> otherwise scan for a usable ZFS pool, use that if it exists
> otherwise use the same partition loader.efi was booted from for
root/kernel if it's usable
> otherwise use the first UFS partition on the ESP that's usable.
>
> A partition is usable if /boot/loader.rc exists on that path.

What will be the role of /etc/fstab in establishing
were the kernel is loaded from? Where world is
loaded from? Where/how does use of /etc/fstab for
specifying the root file system mount fit in the
above pseudo-code?


Same as today: it is what the boot loader passes to the kernel as the Unix
name of /. I have no plans to change that. It's of almost no use to the
boot loader, since it can't know what BIOS device da3 is, for example, if
that's in fstab. Or even more complex examples like /dev/mirror/primary.
Efibootmgr can take Unix devices and paths and turn them into UEFI paths so
we know what devices to use for what. In the absence of those, or an
equivalent fallback, we are quite limited in what we can do since we don't
have the context needed to translate.

Warner

(For my particular interest the context uses UFS, not
ZFS.)

> What is being deleted is one final step: "otherwise use the first UFS
partition on any drive in a random order that's usable." which used to be
at the end of the boot1.efi psuedo code. It's my belief that no such
installations actually use this due to the random factor today (plug in a
new USB drive and it might take over). If my belief is wrong, it's my
belief that efibootmgr will solve it, and failing that, the fallback
mechanism (for platforms that use u-boot + EFI where UEFI variables don't
work) will allow the two or three people that are doing this today.


===
Mark Millard
markmi at dsl-only.net
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: UEFI booting survey

2017-12-19 Thread Warner Losh
On Dec 18, 2017 6:06 PM, "Rodney W. Grimes" <
freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:

> On Mon, Dec 18, 2017 at 3:12 PM, Mark Millard  wrote:
>
> > Warner Losh imp at bsdimp.com wrote on
> > Mon Dec 18 20:29:45 UTC 2017 :
> >
> > > The specific thing we will stop doing is that in the absence of
> > > instructions to the contrary, we will no longer search for root on a
> > device
> > > other than the one the loader.efi came from.
> >
> > Warner Losh imp at bsdimp.com wrote on
> > Sun Dec 17 19:52:07 UTC 2017 :
> >
> > > In the coming months, we're looking at dropping boot1.efi and instead
> > > installing /boot/loader.efi onto the ESP (most likely as
> > > \efi\freebsd\loader.efi).
> >
> >
> > Combining the two statements would appear to have consequences
> > not obvious from the separate statements in isolation. Rewording
> > the first to substitute where loader.efi comes from based on
> > the second (if I interpret right):
> >
> > MISQUOTE
> > The specific thing we will stop doing is that in the absence of
> > instructions to the contrary, we will no longer search for root
> > on other than the device for the ESP used (which will hold
> > loader.efi).
> > END MISQUOTE
> >
>
> The specific thing we will stop doing is that in the absence of
> instructions to the contrary, we will no longer search for root on other
> than the device for the ESP used (which will hold now loader.efi as
> boot1.efi will shortly be eliminated).

Yes please, that is the correct behavior, our searching can lead to
problems, and as you have pointed out, often more problems than it
ever really fixed.

>
> Or the following pseudo-code with all the weird special cases removed for
> clarity
>
> load loader.efi from ESP
> if Boot uefi variable holds a second path, use that for root/kernel
> otherwise if an override variable holds a kernel/root path, use that
> otherwise scan for a usable ZFS pool, use that if it exists
> otherwise use the same partition loader.efi was booted from for
root/kernel
> if it's usable
> otherwise use the first UFS partition on the ESP that's usable.

use the ACTIVE ufs partition, not the first, I can have more than 1 slice,
only 1 of them can be set active.  Do not use any ufs partitions if they
are not in active slices, it is possible to have 0 partitions set active.


Active is not a GPT concept. UEFI makes it hard to implement since there is
no good API to get and set the flags FreeBSD's gptboot uses to hack this
concept in. Active is done via BootOrder UEFI variable. Loader.efi and
boot.efi completely ignore this today. I have no plans on changing that.

>
> A partition is usable if /boot/loader.rc exists on that path.

A partition is usable if it is in an active slice, and ^above


Active isn't a got thong. So no.

Is there any fallback to skip loader and go direct to
/boot/kernel/kernel, back to /kernel any more?


You are thinking about this wrong. We are loader.efi, not boot2. This is
one of the big advantages of loading directly. We don't have the space
limitations that forced that design, so we should simplify.

Warner

> What is being deleted is one final step: "otherwise use the first UFS
> partition on any drive in a random order that's usable." which used to be
> at the end of the boot1.efi psuedo code. It's my belief that no such
> installations actually use this due to the random factor today (plug in a
> new USB drive and it might take over). If my belief is wrong, it's my
> belief that efibootmgr will solve it, and failing that, the fallback
> mechanism (for platforms that use u-boot + EFI where UEFI variables don't
> work) will allow the two or three people that are doing this today.
>
> Warner
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

--
Rod Grimes
rgri...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: UEFI booting survey

2017-12-19 Thread Mark Millard
[I forgot to list the .dtb file with the
kernel and world.]

On 2017-Dec-19, at 9:58 AM, Mark Millard  wrote:

> On 2017-Dec-18, at 2:37 PM, Warner Losh  wrote:
> 
>> . . .
>> 
>> Or the following pseudo-code with all the weird special cases removed for 
>> clarity
>> 
>> load loader.efi from ESP
>> if Boot uefi variable holds a second path, use that for root/kernel
>> otherwise if an override variable holds a kernel/root path, use that
>> otherwise scan for a usable ZFS pool, use that if it exists
>> otherwise use the same partition loader.efi was booted from for root/kernel 
>> if it's usable
>> otherwise use the first UFS partition on the ESP that's usable.
>> 
>> A partition is usable if /boot/loader.rc exists on that path.
> 
> What will be the role of /etc/fstab in establishing
> were the kernel is loaded from? Where world is
> loaded from? Where/how does use of /etc/fstab for
> specifying the root file system mount fit in the
> above pseudo-code?

Typo: should be "establishing where".

> (For my particular interest the context uses UFS, not
> ZFS.)

I forgot to list:

What will be the role of /etc/fstab in establishing
where the .dtb file will be found (when such is
involved)?

>> What is being deleted is one final step: "otherwise use the first UFS 
>> partition on any drive in a random order that's usable." which used to be at 
>> the end of the boot1.efi psuedo code. It's my belief that no such 
>> installations actually use this due to the random factor today (plug in a 
>> new USB drive and it might take over). If my belief is wrong, it's my belief 
>> that efibootmgr will solve it, and failing that, the fallback mechanism (for 
>> platforms that use u-boot + EFI where UEFI variables don't work) will allow 
>> the two or three people that are doing this today.


===
Mark Millard
markmi at dsl-only.net






___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: UEFI booting survey

2017-12-19 Thread Mark Millard
On 2017-Dec-18, at 2:37 PM, Warner Losh  wrote:

> . . .
> 
> Or the following pseudo-code with all the weird special cases removed for 
> clarity
> 
> load loader.efi from ESP
> if Boot uefi variable holds a second path, use that for root/kernel
> otherwise if an override variable holds a kernel/root path, use that
> otherwise scan for a usable ZFS pool, use that if it exists
> otherwise use the same partition loader.efi was booted from for root/kernel 
> if it's usable
> otherwise use the first UFS partition on the ESP that's usable.
> 
> A partition is usable if /boot/loader.rc exists on that path.

What will be the role of /etc/fstab in establishing
were the kernel is loaded from? Where world is
loaded from? Where/how does use of /etc/fstab for
specifying the root file system mount fit in the
above pseudo-code?

(For my particular interest the context uses UFS, not
ZFS.)

> What is being deleted is one final step: "otherwise use the first UFS 
> partition on any drive in a random order that's usable." which used to be at 
> the end of the boot1.efi psuedo code. It's my belief that no such 
> installations actually use this due to the random factor today (plug in a new 
> USB drive and it might take over). If my belief is wrong, it's my belief that 
> efibootmgr will solve it, and failing that, the fallback mechanism (for 
> platforms that use u-boot + EFI where UEFI variables don't work) will allow 
> the two or three people that are doing this today.


===
Mark Millard
markmi at dsl-only.net





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: RFC: Removing hpt* drivers from GENERIC

2017-12-19 Thread Colin Percival
On 10/25/17 15:56, O'Connor, Daniel wrote:
>> On 26 Oct 2017, at 08:13, Colin Percival  wrote:
>> [Proposal for removing hpt* drivers since hpt27xx and hptnr take a long
>> time to in DEVICE_PROBE.]
> 
> Seems sensible to me, but also worth contacting the blob authors if
> possible and asking them what gives (and if they can fix it).

Turns out that they were indeed able to fix it, with startling rapidity.
delphij@ committed r325683 (MFCed as r32600[56]) which reduces the time
spent in these DEVICE_PROBE routines from ~150 ms down to ~37 *us* on my
laptop.  So my immediate desire for faster booting has been satisfied with
regard to these drivers.

I know some people (CCed) were enthusiastic about removing these from GENERIC
on the basis that we shouldn't have binary blobs in GENERIC; while I'm
certainly sympathetic to this, I'd suggest that it should be done by someone
who has time to look at the other binary blobs in the tree and formulate a
general policy rather than just picking on the hpt* drivers.  Unfortunately,
that person is not me; I have 12 days left to submit a talk to AsiaBSDCon
about my work on profiling the kernel boot (which is how I noticed the slow
probing originally) and then a long list of other places to speed up the
boot performance.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"