subject:"Re\: F29 System Wide Change\: Make BootLoaderSpec the default"

On Tue, Jun 26, 2018 at 1:47 PM, Lennart Poettering
 wrote:

> The logic I am working on is built as an add-on to the boot loader
> spec: I simply encode the counter in the file name of the snippet, so
> that counting down and marking an entry as good is a simple rename
> operation, which has a good chance to be implementable atomically even
> in simpler file systems (such as fat), as there should not be the need
> to allocate or release blocks.

rename() is not atomic on FAT. On VFAT in particular, it actually
stores two filenames, 8.3 and LFN - so there are multiple independent
writes that need to happen.

It's probably find most of the time. But if there's a crash, well it's
not a crash safe file system, so if your boot failure includes a
crash, either your rename() based counter, or kdump dumping crash dump
data onto /boot, might be a problem.

TFAT, KFAT, and Window's transaction logs for the BCD on the ESP are
meant to work around the limits of FAT in this regard. We're not doing
anything like that in this proposal though.

>> Let me tell you how totally non-trivial VFAT is for sharing when the
>> driver is in firmware. Digital camera vendors have had vfat drivers in
>> both consumer and professional cameras for over a decade. The one sure
>> way you can corrupt your CF/SD card file system, is transferring it
>> between cameras *even of the same make and model with different
>> firmware versions* and doing basic file operations like create and
>> delete. Boom! Fuck all your files! Hahaha! (Yes the camera maniacally
>> laughs in your face as it corrupts the file system.) The manufacturer
>> recommendation, even on professional gear? Format the card in-camera
>> before each use. Shoot. Do not ever delete files. When you're ready
>> suck the images off the card, back them up, put the card back in the
>> camera, reformat. If you switch cards to different cameras, reformat
>> the card. You can't do that? Expect data loss is possible.
>
> Dunno. We are not talking about digital cameras here. Already for
> licensing/patent reasons firmware tends to stick to the intel uefi
> fat driver. Which might bad and they might have patched around in it
> to make it worse, but I think it's certainly not as bad as you assume
> it to be.

The digital camera case should be even more reliable than our case.
They never write to high latency devices, we do. They also aren't
doing writes in the context of boot failures, including crashes.

>> I've lost count how many times I personally have experienced such data
>> loss, with all sorts of consumer and professional gear, let alone the
>> number of stories I've heard from professional photographers and from
>> camera and SD/CF card engineers.
>
> I did not assume the goal was to run fedora on a digital camera. This
> is borderline FUD...

Nope. Your argument is that FAT is somehow this simple, well
understood thing, by all kinds of UEFI firmware and the kernel. And
doing daily (Rawhide) and weekly (either updates or updates-testing)
modification of this critical file system is completely safe, no more
risky than doing it on a log based file system. I think that's an open
question.

> But given that the exclusive (or almost exclusive) write side of
> things is done from Linux (and not from the boot loader) it's under
> our control, and hence can be made as safe as we can. or to say this
> differently: it's the crappy firmware fs code that *reads* primarily,
> and our Linux code that *writes*. That's systematically different from
> the camera word, where it's the crappy device code that *writes*
> primarily and we on our Linux PC's mostly only *read* the CF cards...

Yes, true. It's more reliable in that sense. But it's less reliable in
that there could be modifications, possibly quite a lot of them, to
the $BOOT file system in the middle of a crash using a file system
known to not be crash safe.

>> > Oh, right. this approach already failed with Grub, with it's
>> > relatively large commercial support, and now you want pile on?
>>
>> Oh please stop. It's not a failed approach. Windows and macOS have
>> been doing this reliably for 20 years because, holy crap! they
>
> Well, still, the fs situation with grub is bad, see the xfs mess. This
> is not going to be any better if you pick some random fs nobody uses
> (such as udf or all the other exotic stuff that was mentioned).

I don't like it either, but at least it's a crash safe file system.
It's just not crash safe for the bootloader, and neither is VFAT.

It's hardly a mess though, it doesn't affect any Fedora default
installations. What you're proposing is a default Fedora installation.

> I understand you have some love for niche file systems, but seriously,
> a boot process is not a place where you want to try something new and
> shiny, but where you stick to the boring stuff you one can manage.

That's funny.

I haven't advocated anything. I'm merely wondering why the spec should
define something other than ext3/4

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

2018-06-26 Thread Javier Martinez Canillas

On Tue, Jun 26, 2018 at 5:36 PM, Javier Martinez Canillas
 wrote:
> On Tue, Jun 26, 2018 at 4:33 PM, Peter Jones  wrote:

[snip]

>>
>>> It attempts to sort using the id field, and if this isn't defined
>>> other fields are used as fallback in the following order: version,
>>> title, linux.
>>
>> Yeah, so maybe we want to make that:
>>
>>   version, id, , title, linux
>>
>
> Yes, although it probably should be:
>
> , id, version, title, linux
>

Ok, I noticed that said something stupid here. The filename is the
only thing expected to be unique so there's no point to use anything
as a fallback for it. So either we should only use the file name to
sort or it should be the last thing used.

I propose to just do: version, 

and use the config file name (without the .conf) as the grub2 menu
entry id and drop the id field since it won't be needed anymore.

That way it will cover ostree case (version has precedence over
filename) and all bootloaders will use the file name to sort.
Alternatively we can just use the file name and change ostree to use
the version instead of the index as its trailing number.

The advantage of the latter is that ostree will also work with
Petitboot and zipl (I don't know if that's a goal).

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/GYMHGRS5P3M52ZDHGD6VZHZXBWCOEZER/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

2018-06-26 Thread Lennart Poettering

On Di, 26.06.18 10:33, Peter Jones (pjo...@redhat.com) wrote:

> On Tue, Jun 26, 2018 at 03:46:59PM +0200, Javier Martinez Canillas wrote:
> > > That raises two questions:
> > > 1. Why isn't just the bls-snippet filename used as the key? It's
> > >necessarily unique and should be usable for the purpose of uniquely
> > >identifying the boot entry without creating a separate field.
> > 
> > I'll let Peter answer this question since he wrote the grub2
> > implementation. Doing this will be pretty trivial, but in that case we
> > should also sort using the filenames if the id field isn't defined.
> 
> I don't know that I have a specific reason in mind when I wrote that bit
> of code.  Most likely the real answer is: because when I realized we
> have to sort them, I was writing parser code not directory iterating
> code.

In the interest of minimal redundancy and compat with other
implementations of the spec, please stick to a single set of ids for
each entry, and according to the spec that's the file name. If you
have multiple id concepts for the same thing then things start
becoming ambiguous.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/MKLC7BUJESZDAIX2MGACEZBRRB4LPMTF/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-26 Thread Lennart Poettering

On Di, 26.06.18 12:51, Chris Murphy (li...@colorremedies.com) wrote:

> I'm making it about systemd-boot because it is literally the only
> bootloader that can't read any file system that the firmware can't
> already read; it's a UEFI only bootloader and it definitely sounds
> like the spec is written with the bootloader in mind rather than the
> other way around.

Let me stress that I am bringing forward technical reasons to use FAT
that have nothing to do with sd-boot, it's you who wants to make it
about that, I haven't mentioned it before.

> > boot menu after failed boots, instead of reverting back). Now, in that
> > model you need to count the attempted boots somewhere. Thus you need
> > write access somewhere (and no, EFI variables don't work for this,
> > they are not suitable for stuff changed on every boot, as their memory
> > is generally not ready to be written too that often). Which hence
> > means you need write access to some file system in some form from the
> > boot loader. And how do you think that's going to work out if already
> > read access to modern file systems is, well, a desaster?
> 
> I agree the counter use case has merit. Is it a bootloader specific
> feature, or is it supposed to work across bootloaders?

Well, I am actually working on adding boot counting support to
systemd, on all layers, i.e. in systemd-boot to count down, and in
systemd itself to assess whether a boot should be considered "good"
and to tell the boot loader about it for the next cycle. All of this
stuff is fairly generic, i.e. communication between systemd and the
boot loader, and it is intended to be, so that other boot loaders can
reuse as much or as little of this logic for whatever they want to
do. But of course, I myself will only put this together for
systemd-boot, it's up to others to hook things up with grub if there's
interest.

The logic I am working on is built as an add-on to the boot loader
spec: I simply encode the counter in the file name of the snippet, so
that counting down and marking an entry as good is a simple rename
operation, which has a good chance to be implementable atomically even
in simpler file systems (such as fat), as there should not be the need
to allocate or release blocks. It also has the benefit that clean-up
cycles are very clear as no additional data is kept and removing a
snippet implicitly also drops its counting information. Example: if a
boot spec entry would normally be called "foobar.conf" without boot
counting in use, then with boot counting it would be called
"foobar+5.conf" or so (in case it is configured to be tries 5 times
before giving up). It would then be renamed to foobar+4.conf on the
first try, foobar+3.conf, foobar+2.conf, foobar+1.conf and finally if
all attempts failed foobar+0.conf. And a tiny new systemd component
will rename the current entry's name to foobar.conf (i.e. dropping the
counter) on success.

> So are you proposing a BLS variant of the grubenv that all bootloaders
> can share? It does matter, because not all file systems support
> grubenv.

Well, I am proposing a solution based on rename()s above. But I am
fully aware that this might not be an option that convinces everbody,
which is why the stuff is generic on all levels: if you don't want to
use renames, then built your own logic instead, but you can still make
use of systemd's boot completion check logic in that case.

> Let me tell you how totally non-trivial VFAT is for sharing when the
> driver is in firmware. Digital camera vendors have had vfat drivers in
> both consumer and professional cameras for over a decade. The one sure
> way you can corrupt your CF/SD card file system, is transferring it
> between cameras *even of the same make and model with different
> firmware versions* and doing basic file operations like create and
> delete. Boom! Fuck all your files! Hahaha! (Yes the camera maniacally
> laughs in your face as it corrupts the file system.) The manufacturer
> recommendation, even on professional gear? Format the card in-camera
> before each use. Shoot. Do not ever delete files. When you're ready
> suck the images off the card, back them up, put the card back in the
> camera, reformat. If you switch cards to different cameras, reformat
> the card. You can't do that? Expect data loss is possible.

Dunno. We are not talking about digital cameras here. Already for
licensing/patent reasons firmware tends to stick to the intel uefi
fat driver. Which might bad and they might have patched around in it
to make it worse, but I think it's certainly not as bad as you assume
it to be. 

> I've lost count how many times I personally have experienced such data
> loss, with all sorts of consumer and professional gear, let alone the
> number of stories I've heard from professional photographers and from
> camera and SD/CF card engineers.

I did not assume the goal was to run fedora on a digital camera. This
is borderline FUD...

also, your comparison is very flawed, as firmware

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Tue, Jun 26, 2018 at 1:33 PM, Peter Jones  wrote:
>> > There's a lot of clouds going to uEFI now
>>
>> [citation needed]
> ...
>> I got sort of lost in Azure versus Hyper-V and gen1/gen2 - apparently 
>> Hyper-V likes
>> UEFI and supports secure boot but Azure may not or something?
>
> Ignoring the question of how many is a lot, I think you may just be
> discussing the difference between the first gen and current gen Azure
> cloud.  They definitely support UEFI, at least in some configuration.
>
>> > The other advantage of having a uEFI partition is you can boot one
>> > image on hardware and VM, I'm planning that with IoT.
>>
>> I later found out that some OpenStack installations like to do this too.
>>
>> So...I agree we should probably do this, particularly if Ubuntu already is 
>> today.
>
> The annoying thing about the hybrid strategy is it means keeping
> grub-install around for BIOS.  But that means it has to be on the thing
> making images, not in the image itself.

At one time the VM images were using extlinux for BIOS, and extlinux
is quite a lot smaller than grub-install and grub-mkconfig. The ISO
images are definitely using isolinux not GRUB.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/5BKSDS6MYOJ7HAMAMJBOOP3PPWSFRIUG/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-26 Thread Peter Jones

> > There's a lot of clouds going to uEFI now 
> 
> [citation needed]
...
> I got sort of lost in Azure versus Hyper-V and gen1/gen2 - apparently Hyper-V 
> likes
> UEFI and supports secure boot but Azure may not or something?

Ignoring the question of how many is a lot, I think you may just be
discussing the difference between the first gen and current gen Azure
cloud.  They definitely support UEFI, at least in some configuration.

> > The other advantage of having a uEFI partition is you can boot one
> > image on hardware and VM, I'm planning that with IoT.
> 
> I later found out that some OpenStack installations like to do this too.
> 
> So...I agree we should probably do this, particularly if Ubuntu already is 
> today.

The annoying thing about the hybrid strategy is it means keeping
grub-install around for BIOS.  But that means it has to be on the thing
making images, not in the image itself.

-- 
  Peter
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/NSLCXQWNAZUSHCBJ25G3BZGMFDZ6TJPO/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-26 Thread Colin Walters

On Tue, Jun 26, 2018, at 11:48 AM, Peter Robinson wrote:

> There's a lot of clouds going to uEFI now 

[citation needed]

In my brief google searches:

AWS: https://forums.aws.amazon.com/thread.jspa?threadID=155626
GCE: https://groups.google.com/forum/#!topic/gce-discussion/OD_Zd_6YVbw
DO: 
https://www.digitalocean.com/community/questions/are-compute-instances-uefi-capable

Now given the reference to Windows...I briefly played with Azure, and booted an 
Ubuntu VM there; one thing
I noticed is that while it didn't boot via EFI, it looks like Ubuntu may 
actually
already do the "bios-and-UEFI" pattern.

I got sort of lost in Azure versus Hyper-V and gen1/gen2 - apparently Hyper-V 
likes
UEFI and supports secure boot but Azure may not or something?

> The other advantage of having a uEFI partition is you can boot one
> image on hardware and VM, I'm planning that with IoT.

I later found out that some OpenStack installations like to do this too.

So...I agree we should probably do this, particularly if Ubuntu already is 
today.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/3ORSKUJ5VKOP3NYGP5OLODWI6Y3HZJKP/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mon, Jun 25, 2018 at 11:22 AM, Kyle Marek  wrote:

>
> /boot is already ext on Fedora. Why does anyone *need* to implement XFS
> for GRUB?


The use case is an XFS root filesystem, and /boot is just a directory.
There is no separate boot volume.

ext4 is default for a volume mounted at /boot; but XFS is permitted.

Our resident ext4/XFS dev likes the idea of making ext2 mandatory for /boot


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/EMZZBL5R77GAYXS2I5Q6KQWMVGVVD35N/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mon, Jun 25, 2018 at 10:09 AM, Andrew Lutomirski  wrote:

> (d) It is useful to write to $BOOT from the bootloader to indicate
> that we're trying to boot and again from the booted system to indicate
> that the boot succeeded.

> (g) The bootloader's driver for $BOOT should implement at least reads
> and preferably writes compatibly with the kernel.  (With the possible
> exception of F2FS, there are no crash-safe filesystems that meet this
> requirement, sadly.)

This isn't reliably done through either firmware or bootloader file
system drivers for reasons I state in the immediately previous email.
But also the GRUB folks aren't ever going to write to any file system
with their drivers, no matter what that file system is. It's against
their philosophy.

What is on the table is something standardized across bootloaders,
akin to the GRUB grubenv file. It could be used for saving state where
it's not possible, reliable or desired to write to NVRAM.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/BB2TWG3SDCPF5CIOY2NTBCA655KQI7DE/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mon, Jun 25, 2018 at 4:40 AM, Lennart Poettering
 wrote:

> I am not sure why you are making this about systemd-boot. Let's just
> focus on why (or why not) vfat is the best option for $BOOT.

I'm making it about systemd-boot because it is literally the only
bootloader that can't read any file system that the firmware can't
already read; it's a UEFI only bootloader and it definitely sounds
like the spec is written with the bootloader in mind rather than the
other way around.

VFAT: No links, labels, or xattr. Like I said earlier, UDF does
support them, and has almost as much crossplatform kernel and
bootloader support as FAT, and is as simple a volume format as FAT. If
there is a way to get atomic updates on vfat without symlink, maybe
vfat could be used for $BOOT. See below.

>> > Which file system do you have in mind even for this?
>>
>> Unspecified for now. i.e. no change. It would remain ext4 by default I
>> expect, but ultimately whatever anaconda allows.
>
> So think about this one bit ahead. Right now it's clear that even with
> Grub's relatively large contributor base it'shard to impossible to
> support modern Linux file systems properly — even just for
> read-only. See the the XFS debacle as one example, and that the kernel
> folks made clear they only consider their own in-kernel implementation
> to be supportable. Now, I'd assume that sooner or later features such
> as boot counting are something we want for Fedora too (i.e. that we
> can update the kernel, try to boot the new one a couple of times, and
> when it fails all the time revert back to an older version, fully
> automatically; in fact the fedora desktop have very recently started
> work on that, though they have a weaker model of simply showing the
> boot menu after failed boots, instead of reverting back). Now, in that
> model you need to count the attempted boots somewhere. Thus you need
> write access somewhere (and no, EFI variables don't work for this,
> they are not suitable for stuff changed on every boot, as their memory
> is generally not ready to be written too that often). Which hence
> means you need write access to some file system in some form from the
> boot loader. And how do you think that's going to work out if already
> read access to modern file systems is, well, a desaster?

I agree the counter use case has merit. Is it a bootloader specific
feature, or is it supposed to work across bootloaders?

GRUB folks absolutely proscribe any file system writes. There is one
state file, the grubenv. This is created through the file system
'grub-editenv create' and is used by GRUB by reading the file system
to determine the LBA's for that file. Any writes by GRUB to grubenv
happen outside the file system driver, directly to an LBA, so they're
atomic (or at least as atomic as a drive can support). This matters
because creating a new file has no guarantee any of the multiple
writes required when going through the file system will actually
happen, which could leave the file system dirty and needing repair -
in addition to failing the meet the requirement for your use case
which is a reliable way of counting boots *in the face of unknown boot
failure*.

So are you proposing a BLS variant of the grubenv that all bootloaders
can share? It does matter, because not all file systems support
grubenv.

And it also matters because this same state file could be used for
atomically switching the default boot entry. e.g. rpm-ostree depends
on a symlink to switch default boot, so perhaps this could be done by
modifying a flag in this static file, outside of the file system.

> Again, FAT is the one thing everyone can agree on. Boot loaders can
> read it *and write it*, UEFI and raspberry pi firmwares have support
> for it, and all OSes and their initrds generally too.

Bootloader absolutely do not write to any file system including FAT.
And GRUB's grubenv permits storing limited state information on file
systems other than vfat, so vfat still isn't required.

Let me tell you how totally non-trivial VFAT is for sharing when the
driver is in firmware. Digital camera vendors have had vfat drivers in
both consumer and professional cameras for over a decade. The one sure
way you can corrupt your CF/SD card file system, is transferring it
between cameras *even of the same make and model with different
firmware versions* and doing basic file operations like create and
delete. Boom! Fuck all your files! Hahaha! (Yes the camera maniacally
laughs in your face as it corrupts the file system.) The manufacturer
recommendation, even on professional gear? Format the card in-camera
before each use. Shoot. Do not ever delete files. When you're ready
suck the images off the card, back them up, put the card back in the
camera, reformat. If you switch cards to different cameras, reformat
the card. You can't do that? Expect data loss is possible.

> From the Linux side we can provide relatively safe read and write
> suppport for FAT. For example, if Fedora

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-26 Thread Peter Robinson

>> > And in my opinion, it's not simple to say: OK if you have this size
>> > ESP to start, you get this layout, and if it's bigger you get this
>> > other layout, and if it's BIOS you have this 3rd layout.
>
> Chris, I have to say I'm glad you're part of the Fedora community - your
> input on this topic has been very valuable!
>
>> Well, for fresh installs[1] there is no reason to have efi and bios use
>> different layouts.  You can just do this:
>
> When you say "install" it really matters to say install of *what* - a
> desktop system, a physical server, a VM, etc.
>
> From my perspective (Fedora CoreOS developer) that straddles
> both physical and cloud for the server case, the problem is that
> the virtualized case, and in particular public cloud, and really
> specifically EC2 - no one really cares about EFI to boot their VMs.
> Except a special case here is "disaster recovery" scenarios where
> a physical server is imaged and uploaded to the cloud as a VM,
> and the topic of UEFI does come up there.  Apparently most
> implementations of this convert back to BIOS.

There's a lot of clouds going to uEFI now due to the Windows
requirement for secure boot so it's useful to have it on generic cloud
images.

> Don't get me wrong, I agree with Lennart (indirectly) in that it is
> kind of crazy how influentual the "Windows dual boot for desktop"
> case is on everything Fedora, which also includes physical
> servers.  But the virtualized case also pushes at this from the other
> angle.
>
>>   [root@ibm-p8-kvm-03-guest-02 ~]# fdisk -l /dev/sda
>>   Disk /dev/sda: 4 GiB, 4294967296 bytes, 8388608 sectors
>>   Units: sectors of 1 * 512 = 512 bytes
>>   Sector size (logical/physical): 512 bytes / 512 bytes
>>   I/O size (minimum/optimal): 512 bytes / 512 bytes
>>   Disklabel type: gpt
>>   Disk identifier: 92035660-BEFD-45D5-9883-B2B91EC429D1
>>
>>   Device   Start End Sectors  Size Type
>>   /dev/sda1 2048   1023981924M BIOS boot
>>   /dev/sda210240 1058815 1048576  512M EFI System
>
> I wouldn't *oppose* that; in fact if you (or someone else)
> wanted to push for that with e.g. Fedora CoreOS, I'd be happy
> to discuss it.  But it's not like it has a truly compelling advantage
> over what we ship today - it'd just be *another* weird variant
> of things in the end right?

The other advantage of having a uEFI partition is you can boot one
image on hardware and VM, I'm planning that with IoT.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/J7TJFXMR6G2UKK3YYFX5GXK4X3PVUS6G/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

2018-06-26 Thread Javier Martinez Canillas

On Tue, Jun 26, 2018 at 4:33 PM, Peter Jones  wrote:
> On Tue, Jun 26, 2018 at 03:46:59PM +0200, Javier Martinez Canillas wrote:
>> > That raises two questions:
>> > 1. Why isn't just the bls-snippet filename used as the key? It's
>> >necessarily unique and should be usable for the purpose of uniquely
>> >identifying the boot entry without creating a separate field.
>>
>> I'll let Peter answer this question since he wrote the grub2
>> implementation. Doing this will be pretty trivial, but in that case we
>> should also sort using the filenames if the id field isn't defined.
>
> I don't know that I have a specific reason in mind when I wrote that bit
> of code.  Most likely the real answer is: because when I realized we
> have to sort them, I was writing parser code not directory iterating
> code.
>
> Way back when the config files were on VFAT on UEFI machines, it
> also meant I didn't have to even worry about the (entirely likely)
> situation arising where I have to re-implement this with some other API
> where I don't know if I'm going to get long filenames or not.
>
> For me, having it in the file seems more natural while I'm staring at
> the file with my eyes, since that's where I'm reading other values used
> in the same context.  But I see the point being made as well.  Maybe the
> thing to do here is prefer "id" if it's in the file, but use the
> filename if it isn't, and to disambiguate collisions.
>
>> This is already the case for Petitboot since the maintainer asked for
>> the filename to be used as the key as well. For zipl the key is the
>> version field because currently the kernel version is used as the
>> zipl.conf section name and the maintainer wanted to keep the same
>> behavior with BLS.
>
> Does this effectively mean we just have a different resolution order
> there?  Like are we comparing id and linux if version collides, or just
> not using them?
>

Sorry, I should had been more clear on this point. Both Petitboot and
zipl sort the entries using the filename and the strverscmp(3)
function. The fields are not used for sorting there since the
filenames should be unique.

The difference I mentioned was that Petitboot uses the filename as the
Petitboot boot entry ID and zipl uses the version field as the zipl
boot entry ID. For Petitboot there can't be duplicates since the
filenames are used, for zipl it will fail to parse the BLS if there
are two snippets with the same version, but that's already the case
without BLS if two zipl boot sections have the same name, so we won't
change that behavior.

>> Sorting using the filename won't work with ostree though. The BLS
>> snippets are named ostree-$ID-$VARIANT_ID-$index.conf but the entries
>> are sorted using the version field, and the index is the inverse of
>> the version.
>>
>> One option is to change ostree so the BLS snippets are named
>> ostree-$ID-$VARIANT_ID-$version.conf, but I don't know if there's a
>> reason to have the index there. Another option is to give more
>> precedence to the version field and only sort using the filenames if
>> neither the id nor the version fields are defined.
>>
>> > 2. Why is "id" supposed to be sortable? What sorting would grub2 do
>> >with it?
>>
>> It's sortable so grub2 can display the boot menu entries in the
>> correct order. This is also answered in the Changes page:
>>
>> id - provides us with a sort key, sorted using rpm's version
>> comparison function. Generally of the form:
>>  fedora-20180530145228-4.16.13-300.fc28.x86_64
>
> Put another way: if you don't have a saved_entry set in grub.cfg, or if
> it's saved_entry=0, if we don't have a sort field then you're going to
> get whatever comes out of your filesystem's directory storage algorithm
> first.  Sometimes that's a linked-list that may or may not be sorted,
> sometimes it's a b+tree, sometimes it's a hash bucket.  It's kind of
> hard to think anyone wants that behavior.
>
>> It attempts to sort using the id field, and if this isn't defined
>> other fields are used as fallback in the following order: version,
>> title, linux.
>
> Yeah, so maybe we want to make that:
>
>   version, id, , title, linux
>

Yes, although it probably should be:

, id, version, title, linux

That way all bootloaders will default to the filename to sort the
entries. That only leaves us with ostree since its BLS filenames can't
be used for sorting.

Colin, do you think that it may be feasible to change the ostree BLS
filenames so the trailing number is the version instead of the index?

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/P24M437GSWOF2EOSEMPN43CYGXLYOWP3/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

2018-06-26 Thread Peter Jones

On Tue, Jun 26, 2018 at 03:46:59PM +0200, Javier Martinez Canillas wrote:
> > That raises two questions:
> > 1. Why isn't just the bls-snippet filename used as the key? It's
> >necessarily unique and should be usable for the purpose of uniquely
> >identifying the boot entry without creating a separate field.
> 
> I'll let Peter answer this question since he wrote the grub2
> implementation. Doing this will be pretty trivial, but in that case we
> should also sort using the filenames if the id field isn't defined.

I don't know that I have a specific reason in mind when I wrote that bit
of code.  Most likely the real answer is: because when I realized we
have to sort them, I was writing parser code not directory iterating
code.

Way back when the config files were on VFAT on UEFI machines, it
also meant I didn't have to even worry about the (entirely likely)
situation arising where I have to re-implement this with some other API
where I don't know if I'm going to get long filenames or not.

For me, having it in the file seems more natural while I'm staring at
the file with my eyes, since that's where I'm reading other values used
in the same context.  But I see the point being made as well.  Maybe the
thing to do here is prefer "id" if it's in the file, but use the
filename if it isn't, and to disambiguate collisions.

> This is already the case for Petitboot since the maintainer asked for
> the filename to be used as the key as well. For zipl the key is the
> version field because currently the kernel version is used as the
> zipl.conf section name and the maintainer wanted to keep the same
> behavior with BLS.

Does this effectively mean we just have a different resolution order
there?  Like are we comparing id and linux if version collides, or just
not using them?

> Sorting using the filename won't work with ostree though. The BLS
> snippets are named ostree-$ID-$VARIANT_ID-$index.conf but the entries
> are sorted using the version field, and the index is the inverse of
> the version.
> 
> One option is to change ostree so the BLS snippets are named
> ostree-$ID-$VARIANT_ID-$version.conf, but I don't know if there's a
> reason to have the index there. Another option is to give more
> precedence to the version field and only sort using the filenames if
> neither the id nor the version fields are defined.
> 
> > 2. Why is "id" supposed to be sortable? What sorting would grub2 do
> >with it?
> 
> It's sortable so grub2 can display the boot menu entries in the
> correct order. This is also answered in the Changes page:
> 
> id - provides us with a sort key, sorted using rpm's version
> comparison function. Generally of the form:
>  fedora-20180530145228-4.16.13-300.fc28.x86_64

Put another way: if you don't have a saved_entry set in grub.cfg, or if
it's saved_entry=0, if we don't have a sort field then you're going to
get whatever comes out of your filesystem's directory storage algorithm
first.  Sometimes that's a linked-list that may or may not be sorted,
sometimes it's a b+tree, sometimes it's a hash bucket.  It's kind of
hard to think anyone wants that behavior.

> It attempts to sort using the id field, and if this isn't defined
> other fields are used as fallback in the following order: version,
> title, linux.

Yeah, so maybe we want to make that:

  version, id, , title, linux

So that it can be the same everywhere.  Aside from cases where it'll be
confusing if all the values just arbitrarily don't match each other
(just don't do that unless you want to confuse yourself), is there some
case I'm missing where that makes it worse?

-- 
  Peter
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/Z2BYMCTDTE7YIIF7CSROPT22E24ZCNYL/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

2018-06-26 Thread Javier Martinez Canillas

On Mon, Jun 25, 2018 at 6:14 PM, Zbigniew Jędrzejewski-Szmek
 wrote:
> On Mon, Jun 18, 2018 at 04:40:41PM +0200, Ondřej Lysoněk wrote:
>> On 14.6.2018 12:06, Jan Kurik wrote:
>> I noticed the official spec defines a field named "machine-id". AFAICS,
>> GRUB2 doesn't implement that option, but it supports a field named "id".
>> Are these used for the same thing? If they are, why are they named
>> differently?
>
> This questions wasn't addressed yet afaics.
>

The reason why are named differently is that the BLS isn't generated
on the machine but at kernel build time and shipped in the kernel
package. So there's a need for a machine independent id. The ostree
BLS snippets also don't define a machine-id BTW, but I don't know if
there's a reason for that.

> The partial answer is that "id" serves a different purpose to "machine-id":
> - "machine-id" is used to specify the machine for which the entry was 
> installed
> - "id" is used by grub2 as a unique identifier usable for saving entries
>

Yes, the "Differences from BootLoaderSpec" section in the Changes page [0] says:

id is also used for menuentry's --id parameter, so you can use it in saved_entry

> That raises two questions:
> 1. Why isn't just the bls-snippet filename used as the key? It's
>necessarily unique and should be usable for the purpose of uniquely
>identifying the boot entry without creating a separate field.

I'll let Peter answer this question since he wrote the grub2
implementation. Doing this will be pretty trivial, but in that case we
should also sort using the filenames if the id field isn't defined.

This is already the case for Petitboot since the maintainer asked for
the filename to be used as the key as well. For zipl the key is the
version field because currently the kernel version is used as the
zipl.conf section name and the maintainer wanted to keep the same
behavior with BLS.

Sorting using the filename won't work with ostree though. The BLS
snippets are named ostree-$ID-$VARIANT_ID-$index.conf but the entries
are sorted using the version field, and the index is the inverse of
the version.

One option is to change ostree so the BLS snippets are named
ostree-$ID-$VARIANT_ID-$version.conf, but I don't know if there's a
reason to have the index there. Another option is to give more
precedence to the version field and only sort using the filenames if
neither the id nor the version fields are defined.

> 2. Why is "id" supposed to be sortable? What sorting would grub2 do
>with it?
>

It's sortable so grub2 can display the boot menu entries in the
correct order. This is also answered in the Changes page:

id - provides us with a sort key, sorted using rpm's version
comparison function. Generally of the form:
 fedora-20180530145228-4.16.13-300.fc28.x86_64

It attempts to sort using the id field, and if this isn't defined
other fields are used as fallback in the following order: version,
title, linux.

> Zbyszek

[0]: https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/3GAGU3MV2LNLFMJ37DDW2IRKTZNE2GI7/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-26 Thread Gerd Hoffmann

  Hi,

> (f) The scheme should function without UEFI.  There are plenty of
> non-UEFI systems out there.  This means that the bootloader might need
> to live in a BIOS boot partition, or in flash, or in ROM, or in other
> strange places.

> Now let's think this through.  You're proposing that $BOOT be the ESP.
> This fails (f) entirely.

Well, no.  You can have both ESP and bios boot partitions, and configure
grub-pc to read the BLS config from the ESP ...

cheers,
  Gerd
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/OFWLR3DO6RS6T7KIYJAFO3I3FWINFVCM/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Kyle Marek

On 06/25/2018 02:49 PM, Lennart Poettering wrote:
> On Mo, 25.06.18 13:22, Kyle Marek (pspps...@gmail.com) wrote:
>
>>> So think about this one bit ahead. Right now it's clear that even with
>>> Grub's relatively large contributor base it'shard to impossible to
>>> support modern Linux file systems properly — even just for
>>> read-only. See the the XFS debacle as one example, and that the kernel
>>> folks made clear they only consider their own in-kernel implementation
>>> to be supportable. Now, I'd assume that sooner or later features such
>>> as boot counting are something we want for Fedora too (i.e. that we
>>> can update the kernel, try to boot the new one a couple of times, and
>>> when it fails all the time revert back to an older version, fully
>>> automatically; in fact the fedora desktop have very recently started
>>> work on that, though they have a weaker model of simply showing the
>>> boot menu after failed boots, instead of reverting back). Now, in that
>>> model you need to count the attempted boots somewhere. Thus you need
>>> write access somewhere (and no, EFI variables don't work for this,
>>> they are not suitable for stuff changed on every boot, as their memory
>>> is generally not ready to be written too that often). Which hence
>>> means you need write access to some file system in some form from the
>>> boot loader. And how do you think that's going to work out if already
>>> read access to modern file systems is, well, a desaster?
>> I think it is better (especially for recovery scenarios) if bootloaders
>> operate read-only. I think things like boot counting should be
>> disregarded simply to preserve read-only access.
> Well, most other big OSes actually do implement boot counting in some
> form, even Linux based ones (such as ChromeOS). It would be a pity if
> Fedora would make its choices in a way that this can never be
> implemented.
>
> boot counting is currently being worked on by the desktop folks, in
> context of the "no boot menu" feature, so that they can reenable the
> boot menu if a previous boot failed.

Note that I am also opposed to the hidden menu... I'm definitely not a
fan for implementing a hidden menu *or* boot count simply because it is
implemented elsewhere, let alone the functionality issues I've mentioned
with the hidden menu proposal.

But back on the side of *writing* to the disk from a boot loader: it's a
boot loader. While advanced things like software RAID support end up in
boot loaders, it is justified because it is necessary to read the
kernel. Reading the kernel into memory is really the defining goal of
any bootloader, so anything that will help accomplish that is good.

However, when the functionality of the bootloader depends on what the
bootloader has *written* during a previous boot, you end up with issues
where a disk failure preventing writes will prevent administrators from
adding "ro shell=/bin/sh" to their cmdline for recovery because the one
bit that determines if a user *can* edit the boot entries is never
written by the boot failures like it is supposed to be.

> But it's useful for unattended systems in general, be it servers or
> appliances: if a boot fails there generally should be a way to recover
> the system through rebooting without manual interfering. Quite
> frankly, it's quite surprising we haven't implemented anything like
> that on Fedora/RHEL at all yet, as it's a major piece in making
> unattended system updates less risky.

I'm still not a fan. I'm not convinced that an issue that is correctable
by booting an old kernel could be caused by a system being left
"unattended". Systems should never automatically reboot due to a kernel
update, and kernel updates really should be given administrator
attention simply *because* of the potential boot issues.

Not to mention that broken kernels may just panic or deadlock, in which
case the system isn't automatically rebooting, anyway.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/IFHJDLI4QXFI2BZBZCBXEBQ66FGUIGHC/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 25.06.18 13:22, Kyle Marek (pspps...@gmail.com) wrote:

> > So think about this one bit ahead. Right now it's clear that even with
> > Grub's relatively large contributor base it'shard to impossible to
> > support modern Linux file systems properly — even just for
> > read-only. See the the XFS debacle as one example, and that the kernel
> > folks made clear they only consider their own in-kernel implementation
> > to be supportable. Now, I'd assume that sooner or later features such
> > as boot counting are something we want for Fedora too (i.e. that we
> > can update the kernel, try to boot the new one a couple of times, and
> > when it fails all the time revert back to an older version, fully
> > automatically; in fact the fedora desktop have very recently started
> > work on that, though they have a weaker model of simply showing the
> > boot menu after failed boots, instead of reverting back). Now, in that
> > model you need to count the attempted boots somewhere. Thus you need
> > write access somewhere (and no, EFI variables don't work for this,
> > they are not suitable for stuff changed on every boot, as their memory
> > is generally not ready to be written too that often). Which hence
> > means you need write access to some file system in some form from the
> > boot loader. And how do you think that's going to work out if already
> > read access to modern file systems is, well, a desaster?
> 
> I think it is better (especially for recovery scenarios) if bootloaders
> operate read-only. I think things like boot counting should be
> disregarded simply to preserve read-only access.

Well, most other big OSes actually do implement boot counting in some
form, even Linux based ones (such as ChromeOS). It would be a pity if
Fedora would make its choices in a way that this can never be
implemented.

boot counting is currently being worked on by the desktop folks, in
context of the "no boot menu" feature, so that they can reenable the
boot menu if a previous boot failed.

But it's useful for unattended systems in general, be it servers or
appliances: if a boot fails there generally should be a way to recover
the system through rebooting without manual interfering. Quite
frankly, it's quite surprising we haven't implemented anything like
that on Fedora/RHEL at all yet, as it's a major piece in making
unattended system updates less risky.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/U2WZBA3F6T7NFVNHHXI6R2FFUVWRE2LL/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 25.06.18 09:09, Andrew Lutomirski (l...@mit.edu) wrote:

> Now let's think this through. You're proposing that $BOOT be the
> ESP.

Yes, I think that's wise, but the boot loader spec allows $BOOT to be
separate from the ESP, so I am not sure why you are warming this up
again.

You can merge $BOOT with the ESP and be conforming to the spec, but
you can also separate the two, and you are still conforming to the
spec. I'd propose for Fedora to use a merged setup when it works, and
a split setup if it doesn't, but it's also OK if if always splits it.

The questions whether to merge or split them is very much independent
from the question which fs to use for $BOOT, and that's actually what
has been discussed most recently in this thread. Let's hence focus on
that, thanks!

> This fails (f) entirely.

I don't think you ever had a look at the boot loader spec, did you? I
invite you to have a look:

https://github.com/systemd/systemd/blob/master/doc/BOOT_LOADER_SPECIFICATION.md

If you look there you'll find that it doesn't state what you appear to
think it states, it's fine without UEFI, it just makes some additional
recommendations on UEFI, which you however can decide to ignore.

> but vfat + mdadm 0.9 and 1.0 fails (e) catastrophically. If we

So, which file system would perform better than fat in this case and
would still be suitable for a boot loader? It's generally the
journalling file systems that can provide better data protection, but
they generally are the ones that don't work properly from boot
loaders, see xfs mess (which is mostly about r/o access, but r/w is
certainly much worse), and there's no perspective for this to change.

And again, it's not that we have complex access patterns on $BOOT. We
never change files, we just create them as one linear blob, and remove
them again. Both operations should totally be implementable in a
fail-save mode onm FAT.

Lennart

--
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/WVBGWTOVRPJJ2DKSLMYH5X7UUCIAVCP3/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Kyle Marek

On 06/25/2018 06:40 AM, Lennart Poettering wrote:
> On Fr, 22.06.18 14:17, Chris Murphy (li...@colorremedies.com) wrote:
>
>> On Fri, Jun 22, 2018 at 12:57 PM, Lennart Poettering
>>  wrote:
>>> On Fr, 22.06.18 19:01, Javier Martinez Canillas (jav...@dowhile0.org) wrote:
>>>
> Whereas constantly changing the ESP, means we need some way to
> establish a master and rsync to the extras.
 So the consensus seems to be to have the BLS fragments in
 $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
 mounted on /boot. That will give us the following advantages as
 mentioned in this thread:
>>> Uh, as one of the authors of the spec, I am not convinced using
>>> arbitrary non-FAT file systems for $BOOT. In fact the spec currently
>>> says it has to be VFAT. I wouldn't call that "consensus".
>> Lennart, I sympathize, but face it. There is a single implementation
>> of kernels and initramfs on the ESP and that's systemd-boot. Everyone
>> else wants to get as far away from vfat as fast as possible. For a
> I am not sure why you are making this about systemd-boot. Let's just
> focus on why (or why not) vfat is the best option for $BOOT.
>
>>> Which file system do you have in mind even for this?
>> Unspecified for now. i.e. no change. It would remain ext4 by default I
>> expect, but ultimately whatever anaconda allows.
> So think about this one bit ahead. Right now it's clear that even with
> Grub's relatively large contributor base it'shard to impossible to
> support modern Linux file systems properly — even just for
> read-only. See the the XFS debacle as one example, and that the kernel
> folks made clear they only consider their own in-kernel implementation
> to be supportable. Now, I'd assume that sooner or later features such
> as boot counting are something we want for Fedora too (i.e. that we
> can update the kernel, try to boot the new one a couple of times, and
> when it fails all the time revert back to an older version, fully
> automatically; in fact the fedora desktop have very recently started
> work on that, though they have a weaker model of simply showing the
> boot menu after failed boots, instead of reverting back). Now, in that
> model you need to count the attempted boots somewhere. Thus you need
> write access somewhere (and no, EFI variables don't work for this,
> they are not suitable for stuff changed on every boot, as their memory
> is generally not ready to be written too that often). Which hence
> means you need write access to some file system in some form from the
> boot loader. And how do you think that's going to work out if already
> read access to modern file systems is, well, a desaster?

I think it is better (especially for recovery scenarios) if bootloaders
operate read-only. I think things like boot counting should be
disregarded simply to preserve read-only access.

> Again, FAT is the one thing everyone can agree on. Boot loaders can
> read it *and write it*, UEFI and raspberry pi firmwares have support
> for it, and all OSes and their initrds generally too.
>
> From the Linux side we can provide relatively safe read and write
> suppport for FAT. For example, if Fedora would use the systemd
> automount logic for mounting $BOOT then the file system will generally
> be unmounted, except in a small time window around actual
> accesses. This means the chance that the file system remains in a
> clean state is maxmized.
>
> $BOOT is a place to place very few files, with very simple access
> patterns. Basically, during update cycles we just add a few files
> there and remove some others, and they are written in one linear write
> operation. For doing that we need no fancy file system features. The
> simplest, most common file system storing files ist good enough for
> that. 
>
>> This problem has many little saboteurs acting together to betray the
>> user. It isn't really any one single thing, they all have to happen to
>> capsize the ship.
> So what are you proposing? Are you going to work on the XFS driver in
> grub to make it match the kernel's current version? And for ext4 too?
> I mean, good luck with that...

/boot is already ext on Fedora. Why does anyone *need* to implement XFS
for GRUB?

>>> Why not just stick to VFAT? As mentioned, it's really the only thing
>>> generally understood by everything that has a stake in boot
>>> loading. Grub speaks it. The EFI firmware speaks it (and that also
>>> means the EFI shell, which is immensly useful). Linux speaks it in the
>>> initrd and after boot. Windows speaks it. MacOS speaks it. It's the
>>> lowest common denominator and should be entirely sufficient to store a
>>> few kernels and their initrds. I mean, we build our kernels as EFI
>>> binaries on Fedora, IIRC. Wouldn't it be a pity if EFI can't actually
>>> access them, because they are stored on an fs only Linux speaks?
>> Wouldn't it be a pity if we didn't teach UEFI to read every goddamn
>> file system ever invented just because we can?!

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

2018-06-25 Thread Zbigniew Jędrzejewski-Szmek

On Mon, Jun 18, 2018 at 04:40:41PM +0200, Ondřej Lysoněk wrote:
> On 14.6.2018 12:06, Jan Kurik wrote:
> I noticed the official spec defines a field named "machine-id". AFAICS,
> GRUB2 doesn't implement that option, but it supports a field named "id".
> Are these used for the same thing? If they are, why are they named
> differently?

This questions wasn't addressed yet afaics.

The partial answer is that "id" serves a different purpose to "machine-id":
- "machine-id" is used to specify the machine for which the entry was installed
- "id" is used by grub2 as a unique identifier usable for saving entries

That raises two questions:
1. Why isn't just the bls-snippet filename used as the key? It's
   necessarily unique and should be usable for the purpose of uniquely
   identifying the boot entry without creating a separate field.
2. Why is "id" supposed to be sortable? What sorting would grub2 do
   with it?

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YNA3GM3JPZVPZSE6CGFWZ54NMFIJJ2Y4/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Andrew Lutomirski

> On Jun 25, 2018, at 3:40 AM, Lennart Poettering  wrote:
>
>> On Fr, 22.06.18 14:17, Chris Murphy (li...@colorremedies.com) wrote:
>>
>> On Fri, Jun 22, 2018 at 12:57 PM, Lennart Poettering
>>  wrote:
>>> On Fr, 22.06.18 19:01, Javier Martinez Canillas (jav...@dowhile0.org) wrote:
>>>
> Whereas constantly changing the ESP, means we need some way to
> establish a master and rsync to the extras.

 So the consensus seems to be to have the BLS fragments in
 $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
 mounted on /boot. That will give us the following advantages as
 mentioned in this thread:
>>>
>>> Uh, as one of the authors of the spec, I am not convinced using
>>> arbitrary non-FAT file systems for $BOOT. In fact the spec currently
>>> says it has to be VFAT. I wouldn't call that "consensus".
>>
>> Lennart, I sympathize, but face it. There is a single implementation
>> of kernels and initramfs on the ESP and that's systemd-boot. Everyone
>> else wants to get as far away from vfat as fast as possible. For a
>
> I am not sure why you are making this about systemd-boot. Let's just
> focus on why (or why not) vfat is the best option for $BOOT.
>
>>> Which file system do you have in mind even for this?
>>
>> Unspecified for now. i.e. no change. It would remain ext4 by default I
>> expect, but ultimately whatever anaconda allows.
>
> So think about this one bit ahead. Right now it's clear that even with
> Grub's relatively large contributor base it'shard to impossible to
> support modern Linux file systems properly — even just for
> read-only. See the the XFS debacle as one example, and that the kernel
> folks made clear they only consider their own in-kernel implementation
> to be supportable. Now, I'd assume that sooner or later features such
> as boot counting are something we want for Fedora too (i.e. that we
> can update the kernel, try to boot the new one a couple of times, and
> when it fails all the time revert back to an older version, fully
> automatically; in fact the fedora desktop have very recently started
> work on that, though they have a weaker model of simply showing the
> boot menu after failed boots, instead of reverting back). Now, in that
> model you need to count the attempted boots somewhere. Thus you need
> write access somewhere (and no, EFI variables don't work for this,
> they are not suitable for stuff changed on every boot, as their memory
> is generally not ready to be written too that often). Which hence
> means you need write access to some file system in some form from the
> boot loader. And how do you think that's going to work out if already
> read access to modern file systems is, well, a desaster?
>
> Again, FAT is the one thing everyone can agree on. Boot loaders can
> read it *and write it*, UEFI and raspberry pi firmwares have support
> for it, and all OSes and their initrds generally too.
>
> From the Linux side we can provide relatively safe read and write
> suppport for FAT. For example, if Fedora would use the systemd
> automount logic for mounting $BOOT then the file system will generally
> be unmounted, except in a small time window around actual
> accesses. This means the chance that the file system remains in a
> clean state is maxmized.
>
> $BOOT is a place to place very few files, with very simple access
> patterns. Basically, during update cycles we just add a few files
> there and remove some others, and they are written in one linear write
> operation. For doing that we need no fancy file system features. The
> simplest, most common file system storing files ist good enough for
> that.

You’ve done a great job stating requirements, but I think you’ve drawn
the wrong conclusion based on the requirements. I’ll summarize:

(a) The bootloader needs to be able to read $BOOT.

(b) It would be nice if UEFI's filesystem layer supported $BOOT. (You
would prefer if it were baked into firmware.  Others might debate this
requirement.)

(c) $BOOT needs to be written when new kernels are installed.

(d) It is useful to write to $BOOT from the bootloader to indicate
that we're trying to boot and again from the booted system to indicate
that the boot succeeded.

(e) Writing $BOOT should be safe.  (You said "relatively safe".  I
would argue that "as safe as possible against power failure and kernel
panics" is better.)

Let's also add:

(f) The scheme should function without UEFI.  There are plenty of
non-UEFI systems out there.  This means that the bootloader might need
to live in a BIOS boot partition, or in flash, or in ROM, or in other
strange places.

(g) The bootloader's driver for $BOOT should implement at least reads
and preferably writes compatibly with the kernel.  (With the possible
exception of F2FS, there are no crash-safe filesystems that meet this
requirement, sadly.)

(h) Putting $BOOT on RAID1 is extremely nice to have.


Now let's think this through.  You're proposing that $BOOT be the ESP.
This fails (f)

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Andrew Lutomirski

> On Jun 25, 2018, at 3:54 AM, Daniel P. Berrangé  wrote:
>
>> On Mon, Jun 25, 2018 at 12:47:35PM +0200, Lennart Poettering wrote:
>>> On Mo, 25.06.18 11:23, Daniel P. Berrangé (berra...@redhat.com) wrote:
>>>
>>> That would break applications like libguestfs which run as non-root and
>>> have valid need to access /boot/vmlinuz*
>>
>> Hmm, can you elaborate on that? What precisely do they need there?
>
> libguestfs boots a KVM guest to do its work inside and uses the installed
> kernel image from /boot/vmlinuz-$UNAME_R for this purpose, together with
> a custom initrd image with specific modules + specialized init binary.

As does virtme.  I just added support for /usr/lib/modules to virtme.
Thanks for the pointer.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QIXTUMKAJW437VO65BCV65ESTHTO7V6L/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Zbigniew Jędrzejewski-Szmek

On Mon, Jun 25, 2018 at 01:48:12PM +0200, Gerd Hoffmann wrote:
> On Mon, Jun 25, 2018 at 12:14:36PM +0200, Lennart Poettering wrote:
> > On Fr, 22.06.18 13:35, Chris Murphy (li...@colorremedies.com) wrote:
> > 
> > > $BOOT being non-vfat is a fairly substantial departure from either
> > > BootLoaderSpec, the original requires $BOOT be vfat, the mjg59 version
> > > require $BOOT be firmware readable. That is not a complaint, I'm just
> > > making an observation of the consequences. I'm personally on the fence
> > > when it comes to the merit of a shared $BOOT. It sounds like a good
> > > idea, but maybe it's specious?
> > 
> > BTW, I think we should actually relax the wording in the spec, and
> > move towards matthew's version on this: instead of saying "must be
> > vfat" to say "must be firmware readable" essentially means the same,
> > but is friendlier towards MacOS of course.

FTR, this got added in https://github.com/systemd/systemd/commit/7f1fc7c6d4.

> Would also allow "we drop a ext2.efi driver to BSP to access $BOOT"
> I guess?

That'd be pretty cool. Is there a widely accepted driver?
Google yields this:
https://efi.akeo.ie/
https://github.com/tianocore/tianocore.github.io/wiki/Tasks-ext2-file-system-driver

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/CXOM7UD6RXGVY5SG3LJS4KXOQTWSTBTZ/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Zbigniew Jędrzejewski-Szmek

On Tue, Jun 19, 2018 at 11:42:33AM +0200, Lennart Poettering wrote:
> On Mo, 18.06.18 16:50, Ondřej Lysoněk (olyso...@redhat.com) wrote:
> 
> > Hi,
> > 
> > On 18.6.2018 15:27, Lennart Poettering wrote:
> > > On Do, 14.06.18 14:20, Chris Murphy (li...@colorremedies.com) wrote:
> > > 
> > >> The cited BLS spec is the original one, not the more thoroughly
> > >> discussed and thought through variant by Matthew Garrett [1] some
> > >> years ago.
> > > 
> > > Quite frankly, as one of the authors of the original BLS spec, I can'd
> > > say Matthew's version was much discussed with me...
> > > 
> > > I mean, I am open to extending the spec, but we should do this bit by
> > > bit.
> > > 
> > > Zbigniew suggested to move the spec into docbook or markdown format,
> > > and then accept changes via usual github PRs. If there's interest
> > > still in extending the spec with some of Matthew's ideas we can
> > > certainly look into that, but in general I'd actually prefer to reduce
> > > the size of the spec if possible, and drop as many bits of it as we
> > > can, i.e. the stuff noone implements anyway.
> > 
> > It would be great if we could extend the spec with optional support for
> > multiple initrd images (the Tuned daemon depends on that). Fedora's
> > GRUB2 already supports multiple initrd images (it allows specifying
> > multiple lines with the "initrd" key), but I'd like to make sure that
> > whoever implements BLS in the future and decides to support multiple
> > initrds will not choose a different syntax for it. Would you be open to
> > extending the spec with that?
> 
> Sure, allowing multiple initrd keys in the snippets makes a ton of sense.

FTR, this got added in https://github.com/systemd/systemd/commit/1d8a48e8e3.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/HVZ756I5RH3ZZHPACTTAAWUNRTWQ4REU/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 25.06.18 13:46, Gerd Hoffmann (kra...@redhat.com) wrote:

>   Hi,
> 
> > > > Which file system do you have in mind even for this?
> > > 
> > > Unspecified for now. i.e. no change. It would remain ext4 by default I
> > > expect, but ultimately whatever anaconda allows.
> 
> IMHO the only thing which is reasonable here would be something simple
> with posix semantics, which is unlikely to change on-disk format anytime
> soon.  So symlinks are working (which appears to be used by atomic /
> ostree), which is something vfat can't support.
> 
> ext2 looks like a good pick here.  Simple, posix, no journal replay
> issues, and drivers for non-linux systems are relatively widespread.

Well, is ext2 currently well maintained? ext4 is, sure, but ext2?  I
sounds questionnable to me to commit to some file system that is
clearly on its way out pretty much everywhere, and that since
years.

I mean, fat isn't really the epitome of file systems, but it *is*
generally very well supported, all across systems, and it's not really
on its way anywhere at all...

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/DVYG64P6CGBO7QAUD4GQRIVMJ6HLN7PN/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Gerd Hoffmann

On Mon, Jun 25, 2018 at 12:14:36PM +0200, Lennart Poettering wrote:
> On Fr, 22.06.18 13:35, Chris Murphy (li...@colorremedies.com) wrote:
> 
> > $BOOT being non-vfat is a fairly substantial departure from either
> > BootLoaderSpec, the original requires $BOOT be vfat, the mjg59 version
> > require $BOOT be firmware readable. That is not a complaint, I'm just
> > making an observation of the consequences. I'm personally on the fence
> > when it comes to the merit of a shared $BOOT. It sounds like a good
> > idea, but maybe it's specious?
> 
> BTW, I think we should actually relax the wording in the spec, and
> move towards matthew's version on this: instead of saying "must be
> vfat" to say "must be firmware readable" essentially means the same,
> but is friendlier towards MacOS of course.

Would also allow "we drop a ext2.efi driver to BSP to access $BOOT"
I guess?

cheers,
  Gerd
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QTAGOWS3GCR6FZYCELGVGHLZHT33BYAC/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Gerd Hoffmann

  Hi,

> > > Which file system do you have in mind even for this?
> > 
> > Unspecified for now. i.e. no change. It would remain ext4 by default I
> > expect, but ultimately whatever anaconda allows.

IMHO the only thing which is reasonable here would be something simple
with posix semantics, which is unlikely to change on-disk format anytime
soon.  So symlinks are working (which appears to be used by atomic /
ostree), which is something vfat can't support.

ext2 looks like a good pick here.  Simple, posix, no journal replay
issues, and drivers for non-linux systems are relatively widespread.

cheers,
  Gerd
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/E6YKIZOZ5EMNF4566AQ3MQPBG65F2XYW/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Richard W.M. Jones

On Mon, Jun 25, 2018 at 12:57:28PM +0200, Lennart Poettering wrote:
> On Mo, 25.06.18 11:54, Daniel P. Berrangé (berra...@redhat.com) wrote:
> 
> > On Mon, Jun 25, 2018 at 12:47:35PM +0200, Lennart Poettering wrote:
> > > On Mo, 25.06.18 11:23, Daniel P. Berrangé (berra...@redhat.com) wrote:
> > > 
> > > > That would break applications like libguestfs which run as non-root and
> > > > have valid need to access /boot/vmlinuz*
> > > 
> > > Hmm, can you elaborate on that? What precisely do they need there?
> > 
> > libguestfs boots a KVM guest to do its work inside and uses the installed
> > kernel image from /boot/vmlinuz-$UNAME_R for this purpose, together with
> > a custom initrd image with specific modules + specialized init binary.
> > 
> > > If it's just the kernel image itself then they shouldn't really use
> > > /boot anyway I figure, but instead the kernel in
> > > /usr/lib/modules/`uname -r`/vmlinux. It's the same thing really.
> > 
> > I wasn't aware of that duplicate vmlinuz file, I wonder how portable
> > that is across distros.
> 
> I figure it should check /usr/lib/modules/`uname -r`/vmlinux first,
> and then fall back to /boot/vmlinuz-$UNAME_R only if that didn't
> exist.

libguestfs has been preferring the /lib/modules/... path for a while
now, although it will fall back to using /boot if it can't find the
kernel in /lib/modules.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/XX56GIV3MJGPD2LS4VXAHCR4NMS74EQR/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 25.06.18 11:54, Daniel P. Berrangé (berra...@redhat.com) wrote:

> On Mon, Jun 25, 2018 at 12:47:35PM +0200, Lennart Poettering wrote:
> > On Mo, 25.06.18 11:23, Daniel P. Berrangé (berra...@redhat.com) wrote:
> > 
> > > That would break applications like libguestfs which run as non-root and
> > > have valid need to access /boot/vmlinuz*
> > 
> > Hmm, can you elaborate on that? What precisely do they need there?
> 
> libguestfs boots a KVM guest to do its work inside and uses the installed
> kernel image from /boot/vmlinuz-$UNAME_R for this purpose, together with
> a custom initrd image with specific modules + specialized init binary.
> 
> > If it's just the kernel image itself then they shouldn't really use
> > /boot anyway I figure, but instead the kernel in
> > /usr/lib/modules/`uname -r`/vmlinux. It's the same thing really.
> 
> I wasn't aware of that duplicate vmlinuz file, I wonder how portable
> that is across distros.

I figure it should check /usr/lib/modules/`uname -r`/vmlinux first,
and then fall back to /boot/vmlinuz-$UNAME_R only if that didn't
exist.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/I4FGGMW4Y7NFCNBFPILEVUYAG347FZFO/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Daniel P . Berrangé

On Mon, Jun 25, 2018 at 12:47:35PM +0200, Lennart Poettering wrote:
> On Mo, 25.06.18 11:23, Daniel P. Berrangé (berra...@redhat.com) wrote:
> 
> > That would break applications like libguestfs which run as non-root and
> > have valid need to access /boot/vmlinuz*
> 
> Hmm, can you elaborate on that? What precisely do they need there?

libguestfs boots a KVM guest to do its work inside and uses the installed
kernel image from /boot/vmlinuz-$UNAME_R for this purpose, together with
a custom initrd image with specific modules + specialized init binary.

> If it's just the kernel image itself then they shouldn't really use
> /boot anyway I figure, but instead the kernel in
> /usr/lib/modules/`uname -r`/vmlinux. It's the same thing really.

I wasn't aware of that duplicate vmlinuz file, I wonder how portable
that is across distros.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/T7AIYPBWUSAHHAUVM4YGIVRJ5DMPARZT/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 25.06.18 11:23, Daniel P. Berrangé (berra...@redhat.com) wrote:

> That would break applications like libguestfs which run as non-root and
> have valid need to access /boot/vmlinuz*

Hmm, can you elaborate on that? What precisely do they need there?

If it's just the kernel image itself then they shouldn't really use
/boot anyway I figure, but instead the kernel in
/usr/lib/modules/`uname -r`/vmlinux. It's the same thing really.

Generally I think it'd be a good idea to ensure that only the boot
loader and tools setting up the boot loader would access /boot.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/3A4BQDWXSA2SQQSNIVEJR7EA5GG3YNGI/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fr, 22.06.18 14:26, Chris Murphy (li...@colorremedies.com) wrote:

> On Fri, Jun 22, 2018 at 1:30 PM, Kyle Marek  wrote:
> 
> > Anaconda in F28 currently claims /boot cannot be vfat. However, this appears
> > to be an artificial limitation, because `grub2-install` works and makes a
> > bootable GRUB with a vfat-typed --boot-directory.
> > I'm not sure why there would be an issue with /boot being vfat. I guess two
> > good questions to ask that might offer some insight:
> >
> > What filesystem limitations make vfat unappealing? (do we need symlinks?)
> 
> Unappealing from a non-shared distro-centric point of view: no xattr,
> no POSIX permissions or owners, no links.
> 
> Some of those things are unappealing and maybe disqualifying for a
> shared boot, security labels being one.

Please be less vague. We already established that currently the
selinux databse only uses two different relevant labels on /boot. And
it's not clear to me that it's really worth maintaining those
separately. And if there's only one label for it, then fat is fine, as
it's sufficient then to specify the label to use in the mount options.

I mean, let's face it, the main stakeholder on $BOOT is not going to
honour the labels anyway, so I think it's only fair to treat the
whoile thing as a single security domain.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/66JMLS4ZYQLB3FBABQPZUIXITZCZCMH2/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fr, 22.06.18 14:17, Chris Murphy (li...@colorremedies.com) wrote:

> On Fri, Jun 22, 2018 at 12:57 PM, Lennart Poettering
>  wrote:
> > On Fr, 22.06.18 19:01, Javier Martinez Canillas (jav...@dowhile0.org) wrote:
> >
> >> > Whereas constantly changing the ESP, means we need some way to
> >> > establish a master and rsync to the extras.
> >>
> >> So the consensus seems to be to have the BLS fragments in
> >> $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
> >> mounted on /boot. That will give us the following advantages as
> >> mentioned in this thread:
> >
> > Uh, as one of the authors of the spec, I am not convinced using
> > arbitrary non-FAT file systems for $BOOT. In fact the spec currently
> > says it has to be VFAT. I wouldn't call that "consensus".
> 
> Lennart, I sympathize, but face it. There is a single implementation
> of kernels and initramfs on the ESP and that's systemd-boot. Everyone
> else wants to get as far away from vfat as fast as possible. For a

I am not sure why you are making this about systemd-boot. Let's just
focus on why (or why not) vfat is the best option for $BOOT.

> > Which file system do you have in mind even for this?
> 
> Unspecified for now. i.e. no change. It would remain ext4 by default I
> expect, but ultimately whatever anaconda allows.

So think about this one bit ahead. Right now it's clear that even with
Grub's relatively large contributor base it'shard to impossible to
support modern Linux file systems properly — even just for
read-only. See the the XFS debacle as one example, and that the kernel
folks made clear they only consider their own in-kernel implementation
to be supportable. Now, I'd assume that sooner or later features such
as boot counting are something we want for Fedora too (i.e. that we
can update the kernel, try to boot the new one a couple of times, and
when it fails all the time revert back to an older version, fully
automatically; in fact the fedora desktop have very recently started
work on that, though they have a weaker model of simply showing the
boot menu after failed boots, instead of reverting back). Now, in that
model you need to count the attempted boots somewhere. Thus you need
write access somewhere (and no, EFI variables don't work for this,
they are not suitable for stuff changed on every boot, as their memory
is generally not ready to be written too that often). Which hence
means you need write access to some file system in some form from the
boot loader. And how do you think that's going to work out if already
read access to modern file systems is, well, a desaster?

Again, FAT is the one thing everyone can agree on. Boot loaders can
read it *and write it*, UEFI and raspberry pi firmwares have support
for it, and all OSes and their initrds generally too.

From the Linux side we can provide relatively safe read and write
suppport for FAT. For example, if Fedora would use the systemd
automount logic for mounting $BOOT then the file system will generally
be unmounted, except in a small time window around actual
accesses. This means the chance that the file system remains in a
clean state is maxmized.

$BOOT is a place to place very few files, with very simple access
patterns. Basically, during update cycles we just add a few files
there and remove some others, and they are written in one linear write
operation. For doing that we need no fancy file system features. The
simplest, most common file system storing files ist good enough for
that. 

> This problem has many little saboteurs acting together to betray the
> user. It isn't really any one single thing, they all have to happen to
> capsize the ship.

So what are you proposing? Are you going to work on the XFS driver in
grub to make it match the kernel's current version? And for ext4 too?
I mean, good luck with that...

> > Why not just stick to VFAT? As mentioned, it's really the only thing
> > generally understood by everything that has a stake in boot
> > loading. Grub speaks it. The EFI firmware speaks it (and that also
> > means the EFI shell, which is immensly useful). Linux speaks it in the
> > initrd and after boot. Windows speaks it. MacOS speaks it. It's the
> > lowest common denominator and should be entirely sufficient to store a
> > few kernels and their initrds. I mean, we build our kernels as EFI
> > binaries on Fedora, IIRC. Wouldn't it be a pity if EFI can't actually
> > access them, because they are stored on an fs only Linux speaks?
> 
> Wouldn't it be a pity if we didn't teach UEFI to read every goddamn
> file system ever invented just because we can?!
> 
> http://efi.akeo.ie/downloads/efifs-latest/x64/
> https://github.com/pbatard/efifs

Oh, right. this approach already failed with Grub, with it's
relatively large commercial support, and now you want pile on?

> I mean honestly, we can teach EFI whatever the hell we want. File
> system support does not need to be baked into the bootloader on UEFI.
> Drop these guys onto your ESP and

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Daniel P . Berrangé

On Mon, Jun 25, 2018 at 06:04:54AM -0400, Simo Sorce wrote:
> On Fri, 2018-06-22 at 16:30 -0500, Chris Adams wrote:
> > Once upon a time, Kyle Marek  said:
> > > On 06/22/2018 05:15 PM, Chris Adams wrote:
> > > > And basic Unix permissions... because there can be privileged
> > > > content in
> > > > GRUB config and even initramfs.
> > > 
> > > That's interesting. I generally don't see /boot as something that
> > > normal
> > > users shouldn't be able to read. Or, in other words, I generally
> > > don't
> > > see it as a place where secret data should be stored.
> > > 
> > > Any particular examples?
> > 
> > GRUB can have passwords to protect booting, menu options, and
> > changing
> > config.  The initramfs can have network and iSCSI config for mounting
> > the root filesystem.
> 
> And /boot can be mounted (and probably should be) only readable to root

That would break applications like libguestfs which run as non-root and
have valid need to access /boot/vmlinuz*

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/2QOZP2KD6KUOT7U6ZCJFC3ZVMYLWK2DH/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fr, 22.06.18 15:54, Kyle Marek (pspps...@gmail.com) wrote:

> What is the benefit to sharing $BOOT between different operating
> systems/distros?
> 
> I'd like to point out that $BOOT doesn't have to be shared to dual-boot
> multiple distros or benefit from other details of BLS.

But it's an awful mess... Whenever one OS updates or installs its boot
loader it tends to kick out all the others, unless it contains messy
scripts that try to find the other boot loaders first, and readds them
to the menu it's mostly luck if things work. With the bootloader spec
this is cleaned up, as it's built around drop-in files in a shared
directory, instead of exclusive last-writer ownership of the boot
loader.

Ultimately it just takes inspiration how RPM-based OSes nowadays
heavily rely on drop-in dirs so that lose coupling between packages
canbe implemented, as packages generally can't and shouldn't override
each other's files.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/GIOCGO6CXSF7OW266ZSBQZNBZCPXF43W/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fr, 22.06.18 13:35, Chris Murphy (li...@colorremedies.com) wrote:

> $BOOT being non-vfat is a fairly substantial departure from either
> BootLoaderSpec, the original requires $BOOT be vfat, the mjg59 version
> require $BOOT be firmware readable. That is not a complaint, I'm just
> making an observation of the consequences. I'm personally on the fence
> when it comes to the merit of a shared $BOOT. It sounds like a good
> idea, but maybe it's specious?

BTW, I think we should actually relax the wording in the spec, and
move towards matthew's version on this: instead of saying "must be
vfat" to say "must be firmware readable" essentially means the same,
but is friendlier towards MacOS of course. So yes, we should totally
relax the language on this, but not, using completely arbitrary file
systems on this certainly doesn't make much sense.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/6B6ABM25RUSGSIFQQLZPFK57VISF7MA4/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-25 Thread Simo Sorce

On Fri, 2018-06-22 at 16:30 -0500, Chris Adams wrote:
> Once upon a time, Kyle Marek  said:
> > On 06/22/2018 05:15 PM, Chris Adams wrote:
> > > And basic Unix permissions... because there can be privileged
> > > content in
> > > GRUB config and even initramfs.
> > 
> > That's interesting. I generally don't see /boot as something that
> > normal
> > users shouldn't be able to read. Or, in other words, I generally
> > don't
> > see it as a place where secret data should be stored.
> > 
> > Any particular examples?
> 
> GRUB can have passwords to protect booting, menu options, and
> changing
> config.  The initramfs can have network and iSCSI config for mounting
> the root filesystem.

And /boot can be mounted (and probably should be) only readable to root
?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/H4QCJXT3GWTBSKWT5ZV756MMMDQ3FMCR/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Chris Adams

Once upon a time, Kyle Marek  said:
> On 06/22/2018 05:15 PM, Chris Adams wrote:
> > And basic Unix permissions... because there can be privileged content in
> > GRUB config and even initramfs.
> 
> That's interesting. I generally don't see /boot as something that normal
> users shouldn't be able to read. Or, in other words, I generally don't
> see it as a place where secret data should be stored.
> 
> Any particular examples?

GRUB can have passwords to protect booting, menu options, and changing
config.  The initramfs can have network and iSCSI config for mounting
the root filesystem.

-- 
Chris Adams 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/PPY6WBGG5ORBQMTH25WEQGPFCHI7SDCM/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Kyle Marek

On 06/22/2018 05:15 PM, Chris Adams wrote:
> Once upon a time, Matthew Miller  said:
>> On Fri, Jun 22, 2018 at 03:30:23PM -0400, Kyle Marek wrote:
>>> Anaconda in F28 currently claims /boot cannot be vfat. However, this
>>> appears to be an artificial limitation, because `grub2-install` works
>>> and makes a bootable GRUB with a vfat-typed --boot-directory.
>>> I'm not sure why there would be an issue with /boot being vfat. I guess
>>> two good questions to ask that might offer some insight:
>> Well, currently, we have things in there with different selinux
>> labels
> And basic Unix permissions... because there can be privileged content in
> GRUB config and even initramfs.

That's interesting. I generally don't see /boot as something that normal
users shouldn't be able to read. Or, in other words, I generally don't
see it as a place where secret data should be stored.

Any particular examples?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/H2E27EMY6EMCZTYPKZVO5T7ZNVG3EO3S/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Chris Adams

Once upon a time, Matthew Miller  said:
> On Fri, Jun 22, 2018 at 03:30:23PM -0400, Kyle Marek wrote:
> > Anaconda in F28 currently claims /boot cannot be vfat. However, this
> > appears to be an artificial limitation, because `grub2-install` works
> > and makes a bootable GRUB with a vfat-typed --boot-directory.
> > I'm not sure why there would be an issue with /boot being vfat. I guess
> > two good questions to ask that might offer some insight:
> 
> Well, currently, we have things in there with different selinux
> labels

And basic Unix permissions... because there can be privileged content in
GRUB config and even initramfs.
-- 
Chris Adams 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/XDI3UNOKKWRUIH6GDQURD2XJKSHN6D4K/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fri, Jun 22, 2018 at 1:54 PM, Kyle Marek  wrote:
> On 06/22/2018 03:35 PM, Chris Murphy wrote:

>
> What is the benefit to sharing $BOOT between different operating
> systems/distros?

Some of this is argued in the two BootLoaderSpecs. Mainly to avoid
stomping on each other's installations and bootloaders, and a bit less
redundancy instead of every distro having its own $BOOT.

But really, how many people multiboot 2 or more Linux distros? My
shits n giggles guess?

Most common is Windows + Linux. Next most common macOS + Linux. Next
most common Linux only. I think in some sense we're in the weeds on
multibooting. It is possible we're overvaluing shared $BOOT.

> I'd like to point out that $BOOT doesn't have to be shared to dual-boot
> multiple distros or benefit from other details of BLS.

True. Although the original BootLoaderSpec script file format only
supports paths relative to $BOOT. There's no way to reference other
volumes, even if they are readable by the firmware. You'd have to
depend on the bootloader's native configuration file format instead of
BLS format for such a feature, meaning no way to support it with one
format across bootloaders.

> Each installed OS that wants to use some derivative of BLS really *can*
> just each have their own $BOOT and even use different bootloaders to
> implement BLS. (bootloaders can be chained in BIOS, and they can exist
> independently of each other in EFI)

Sure.

> The primary benefit I see to adopting BLS here is the drop-in
> configuration and consistent syntax regardless of the implementing
> bootloaders. The benefit to sharing $BOOT between operating systems
> isn't obvious to me, and only introduces limitations such as this
> filesystem one.

I agree. And I like the consistent location and path.

The change is a win, even if it's not a warm and fuzzy embrace of the
whole BootLoaderSpec. It leaves the door open to either distros
constraining their implementations toward BootLoaderSpec, or the
broadening of the BootLoaderSpec to grow the market.

My personal assessment is that the latter is more likely.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/7WXRGNDMQ7VN73QUQN6TVRCW53VS73YJ/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fri, Jun 22, 2018 at 1:30 PM, Kyle Marek  wrote:

> Anaconda in F28 currently claims /boot cannot be vfat. However, this appears
> to be an artificial limitation, because `grub2-install` works and makes a
> bootable GRUB with a vfat-typed --boot-directory.
> I'm not sure why there would be an issue with /boot being vfat. I guess two
> good questions to ask that might offer some insight:
>
> What filesystem limitations make vfat unappealing? (do we need symlinks?)

Unappealing from a non-shared distro-centric point of view: no xattr,
no POSIX permissions or owners, no links.

Some of those things are unappealing and maybe disqualifying for a
shared boot, security labels being one.

So many things here are in possible conflict from the distro centric
way of viewing the world, that simply have to change if we're really
going to get to a shared $BOOT world.

> Does Fedora plan to support installing with bootloaders other than GRUB on
> x86?

extlinux is an option supported by anaconda, it's not supported in
that if something about it doesn't work we aren't going to block
release; but it's a preferred bootloader for some VM images I think
Cloud stuff was or is using extlinux.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/Z5GA5SWIMS2TH3FFBX2MS5WAY2GGUTDU/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fri, Jun 22, 2018 at 12:57 PM, Lennart Poettering
 wrote:
> On Fr, 22.06.18 19:01, Javier Martinez Canillas (jav...@dowhile0.org) wrote:
>
>> > Whereas constantly changing the ESP, means we need some way to
>> > establish a master and rsync to the extras.
>>
>> So the consensus seems to be to have the BLS fragments in
>> $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
>> mounted on /boot. That will give us the following advantages as
>> mentioned in this thread:
>
> Uh, as one of the authors of the spec, I am not convinced using
> arbitrary non-FAT file systems for $BOOT. In fact the spec currently
> says it has to be VFAT. I wouldn't call that "consensus".

Lennart, I sympathize, but face it. There is a single implementation
of kernels and initramfs on the ESP and that's systemd-boot. Everyone
else wants to get as far away from vfat as fast as possible. For a
long time one of the first things a bootloader does is learn how to
read a modern file system, including \EFI\Microsoft\Boot\bootmgfw.efi

I totally get the arguments that GRUB is too big, too complicated,
does too many things, each distro effectively live forks it into
something often incompatible everywhere else. But systemd-boot really
does go in the opposite extreme. I think it's too simple. But
whatever. The spec's requirement that $BOOT be vfat was simply not
going to go anywhere beyond systemd-boot and not even beyond UEFI.
From that perspective, BootLoaderSpec as currently written is too UEFI
centric and thus itself is not even trying to reach a consensus which
is why it hasn't reached consensus.

And systemd-boot could leverage other projects that wrap any of GRUB's
existing file system modules as EFI programs to teach the firmware
pre-boot environment how to read any file system you want, without
having to write in separate support for file systems in systemd-boot.

> Which file system do you have in mind even for this?

Unspecified for now. i.e. no change. It would remain ext4 by default I
expect, but ultimately whatever anaconda allows.

> As far as I know it's very clear now that boot loaders have a hard
> time implementing any of the current file systems properly. AFAIK the
> XFS folks as one case were pretty clear that any implementation of XFS
> which is not the in-kernel one is not supportable, but I am pretty
> sure for the other more modern file systems things aren't too
> different either. The fact that grub doesn't properly implement XFS is
> a real issue, as I am sure you know, since it won't replay the
> journal, and hence doesn't see changes made on previous boots when the
> file system wasn't unmounted (which is a regularly seen issue, as ply
> keeps XFS busy during shutdown), resulting in unbootable systems.

This problem has many little saboteurs acting together to betray the
user. It isn't really any one single thing, they all have to happen to
capsize the ship.

Nevertheless I pretty much find your original criticism that XFS
sync() behavior is wrong, the most convincing. I'm happy to give all
file systems a pass on fsync() dumping changes only to the log in such
a way that only kernel log replay will catch things up properly. But
in my sabotage testing, only XFS sync() behavior seems to end up with
a file system that is unreliably readable by a bootloader. And I'm not
a fan of non-atomic FIFREEZE/FITHAW as the work around, not least of
which is I've found a way to sabotage it and break an updating system
(with no estimate of how likely this is in the real world).

> Why not just stick to VFAT? As mentioned, it's really the only thing
> generally understood by everything that has a stake in boot
> loading. Grub speaks it. The EFI firmware speaks it (and that also
> means the EFI shell, which is immensly useful). Linux speaks it in the
> initrd and after boot. Windows speaks it. MacOS speaks it. It's the
> lowest common denominator and should be entirely sufficient to store a
> few kernels and their initrds. I mean, we build our kernels as EFI
> binaries on Fedora, IIRC. Wouldn't it be a pity if EFI can't actually
> access them, because they are stored on an fs only Linux speaks?

Wouldn't it be a pity if we didn't teach UEFI to read every goddamn
file system ever invented just because we can?!

http://efi.akeo.ie/downloads/efifs-latest/x64/
https://github.com/pbatard/efifs

I mean honestly, we can teach EFI whatever the hell we want. File
system support does not need to be baked into the bootloader on UEFI.
Drop these guys onto your ESP and now the firmware with any bootloader
can read any of those file systems directly. Pick one.

I have to defer to others on the value of symlinks for atomic
configuration swapout, but if you want the most widely supported file
system that also has symlink support, it's UDF. For the time being
though, the concept of a widely sharable $BOOT really doesn't have
enough momentum or backing.

I still think the change pushing us closer to BootLoaderSpec. But I
also

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Kyle Marek

On 06/22/2018 03:35 PM, Chris Murphy wrote:
> On Fri, Jun 22, 2018 at 11:01 AM, Javier Martinez Canillas
>  wrote:
>> On Thu, Jun 21, 2018 at 11:19 PM, Chris Murphy  
>> wrote:
>>
>> [snip]
>>
> OK anyway, I don't see broad BLS consensus forming yet, but I do see
> two items that aren't controversial and could move forward as part of
> this feature proposal:
>
> a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
> the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
> existing /boot ext4 volume currently used). i.e. do not put
> /loader/entries in /EFI/fedora
 By "consistent", do you mean that both EFI and BIOS boot loaders will
 reference the same entry files? I like that.
>>> Yes.
>>>
 However, I don't like the entries existing on ESP mostly because of the
 use case of md-RAID for /boot referenced in another thread. I think it
 would be best to just put the GRUB EFI file on ESP and put the rest of
 the bulk GRUB stuff in its utility partition (which may be RAID-ed).
>>> Right. The config + kernel + initramfs on traditional /boot can make
>>> use of software raid, depending on static one time proper
>>> configuration of each member drive's ESPs and now the ESPs never need
>>> to be touched and thus not sync'd.
>>>
>>> Whereas constantly changing the ESP, means we need some way to
>>> establish a master and rsync to the extras.
>>>
>> So the consensus seems to be to have the BLS fragments in
>> $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
>> mounted on /boot. That will give us the following advantages as
>> mentioned in this thread:
>>
>> 1. The BLS will not be stored in vfat, so ostree could keep using
>> symlinks to do atomic swap of its /boot/loader/entries
>> 2. The ESP won't be modified on kernels install / removal, that will
>> make it easier for software RAID.
>> 3. There will be consistency on where the BLS fragments are installed
>> regardless of the firmware interface (EFI, BIOS, OPAL on ppc64le and
>> zipl on s390x will all use /boot/loader/entries).
>>
>> I've updated the wiki page to reflect this and we will also change the
>> implementation.
>
> $BOOT being non-vfat is a fairly substantial departure from either
> BootLoaderSpec, the original requires $BOOT be vfat, the mjg59 version
> require $BOOT be firmware readable. That is not a complaint, I'm just
> making an observation of the consequences. I'm personally on the fence
> when it comes to the merit of a shared $BOOT. It sounds like a good
> idea, but maybe it's specious?
>
> Just to give some people still hanging on to this thread a visual of
> the dilemma:
>
> Not Shared $BOOT <||---> Shared $BOOT
> md raid  vfat
> lvm, lvmraid udf
> btrfsntfs
> LUKS
> F2FS
>
> By making it possible to put /boot/loader/entries on e.g. md or even
> lvm raid1 or btrfs raid1, that implies a /boot/loader/entries that's
> readable for very few bootloaders, and no operating systems other than
> Linux. So it is not a shared $BOOT design. And that is a big departure
> from the entire point of the BootLoaderSpec, which I think is a bit
> too rigid.  I think the spec would be better off presenting itself as
> a continuum from a highly sharable $BOOT, to one that has features
> that inevitably make it less sharable.
>
> e.g. Somewhere close to shared $BOOT would be udf or ntfs. Both have
> read support on all major OS's, and by GRUB. Both support symlinks and
> some other features of modern file systems that FAT lacks. But UDF
> gets the edge on Linux because we have kernel level support, whereas
> only FUSE support for NTFS.
>
> Syncing a share $BOOT without fancy Linux features (upper left list)
> isn't that hard, it just requires a big dose of political capital to
> get a service or daemon that most every distro is willing to support
> in their core components so that sharing $BOOT doesn't result in out
> of sync ESPs. That could be a supplement to BootLoaderSpec and
> possibly a feature of systemd. But to date, there has been no one
> willing to do that work.
>
> Anyway, I'm OK with all of the changes made so far. I think it does
> simplify things quite a bit for Fedora users, while not shutting the
> door on future standardization efforts. e.g. An option still available
> to Fedora users is $BOOT on ESP where ESP automounts via systemd gpt
> generator onto /boot - and you'll get /boot/loader/entries just like
> everyone else if you want to use systemd-boot.

What is the benefit to sharing $BOOT between different operating
systems/distros?

I'd like to point out that $BOOT doesn't have to be shared to dual-boot
multiple distros or benefit from other details of BLS.

Each installed OS that wants to use some derivative of BLS really *can*
just each have their own $BOOT and even use different bootloaders to
implement BLS. (bootloaders can be

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Matthew Miller

On Fri, Jun 22, 2018 at 03:30:23PM -0400, Kyle Marek wrote:
> Anaconda in F28 currently claims /boot cannot be vfat. However, this
> appears to be an artificial limitation, because `grub2-install` works
> and makes a bootable GRUB with a vfat-typed --boot-directory.
> I'm not sure why there would be an issue with /boot being vfat. I guess
> two good questions to ask that might offer some insight:

Well, currently, we have things in there with different selinux
labels


-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ITW6XHUDUENH3WB6HKOJJS2R3YRJN7U3/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Fri, Jun 22, 2018 at 11:01 AM, Javier Martinez Canillas
 wrote:
> On Thu, Jun 21, 2018 at 11:19 PM, Chris Murphy  
> wrote:
>
> [snip]
>
>>>
 OK anyway, I don't see broad BLS consensus forming yet, but I do see
 two items that aren't controversial and could move forward as part of
 this feature proposal:

 a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
 the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
 existing /boot ext4 volume currently used). i.e. do not put
 /loader/entries in /EFI/fedora
>>>
>>> By "consistent", do you mean that both EFI and BIOS boot loaders will
>>> reference the same entry files? I like that.
>>
>> Yes.
>>
>>> However, I don't like the entries existing on ESP mostly because of the
>>> use case of md-RAID for /boot referenced in another thread. I think it
>>> would be best to just put the GRUB EFI file on ESP and put the rest of
>>> the bulk GRUB stuff in its utility partition (which may be RAID-ed).
>>
>> Right. The config + kernel + initramfs on traditional /boot can make
>> use of software raid, depending on static one time proper
>> configuration of each member drive's ESPs and now the ESPs never need
>> to be touched and thus not sync'd.
>>
>> Whereas constantly changing the ESP, means we need some way to
>> establish a master and rsync to the extras.
>>
>
> So the consensus seems to be to have the BLS fragments in
> $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
> mounted on /boot. That will give us the following advantages as
> mentioned in this thread:
>
> 1. The BLS will not be stored in vfat, so ostree could keep using
> symlinks to do atomic swap of its /boot/loader/entries
> 2. The ESP won't be modified on kernels install / removal, that will
> make it easier for software RAID.
> 3. There will be consistency on where the BLS fragments are installed
> regardless of the firmware interface (EFI, BIOS, OPAL on ppc64le and
> zipl on s390x will all use /boot/loader/entries).
>
> I've updated the wiki page to reflect this and we will also change the
> implementation.

$BOOT being non-vfat is a fairly substantial departure from either
BootLoaderSpec, the original requires $BOOT be vfat, the mjg59 version
require $BOOT be firmware readable. That is not a complaint, I'm just
making an observation of the consequences. I'm personally on the fence
when it comes to the merit of a shared $BOOT. It sounds like a good
idea, but maybe it's specious?

Just to give some people still hanging on to this thread a visual of
the dilemma:

Not Shared $BOOT <||---> Shared $BOOT
md raid  vfat
lvm, lvmraid udf
btrfsntfs
LUKS
F2FS

By making it possible to put /boot/loader/entries on e.g. md or even
lvm raid1 or btrfs raid1, that implies a /boot/loader/entries that's
readable for very few bootloaders, and no operating systems other than
Linux. So it is not a shared $BOOT design. And that is a big departure
from the entire point of the BootLoaderSpec, which I think is a bit
too rigid.  I think the spec would be better off presenting itself as
a continuum from a highly sharable $BOOT, to one that has features
that inevitably make it less sharable.

e.g. Somewhere close to shared $BOOT would be udf or ntfs. Both have
read support on all major OS's, and by GRUB. Both support symlinks and
some other features of modern file systems that FAT lacks. But UDF
gets the edge on Linux because we have kernel level support, whereas
only FUSE support for NTFS.

Syncing a share $BOOT without fancy Linux features (upper left list)
isn't that hard, it just requires a big dose of political capital to
get a service or daemon that most every distro is willing to support
in their core components so that sharing $BOOT doesn't result in out
of sync ESPs. That could be a supplement to BootLoaderSpec and
possibly a feature of systemd. But to date, there has been no one
willing to do that work.

Anyway, I'm OK with all of the changes made so far. I think it does
simplify things quite a bit for Fedora users, while not shutting the
door on future standardization efforts. e.g. An option still available
to Fedora users is $BOOT on ESP where ESP automounts via systemd gpt
generator onto /boot - and you'll get /boot/loader/entries just like
everyone else if you want to use systemd-boot.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/TKFDVPRTCL737REDIQ5TZN7AQH3ZXE3M/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Kyle Marek

On 06/22/2018 02:57 PM, Lennart Poettering wrote:
> On Fr, 22.06.18 19:01, Javier Martinez Canillas (jav...@dowhile0.org) wrote:
>
>>> Whereas constantly changing the ESP, means we need some way to
>>> establish a master and rsync to the extras.
>> So the consensus seems to be to have the BLS fragments in
>> $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
>> mounted on /boot. That will give us the following advantages as
>> mentioned in this thread:
> Uh, as one of the authors of the spec, I am not convinced using
> arbitrary non-FAT file systems for $BOOT. In fact the spec currently
> says it has to be VFAT. I wouldn't call that "consensus".
>
> Which file system do you have in mind even for this?
>
> As far as I know it's very clear now that boot loaders have a hard
> time implementing any of the current file systems properly. AFAIK the
> XFS folks as one case were pretty clear that any implementation of XFS
> which is not the in-kernel one is not supportable, but I am pretty
> sure for the other more modern file systems things aren't too
> different either. The fact that grub doesn't properly implement XFS is
> a real issue, as I am sure you know, since it won't replay the
> journal, and hence doesn't see changes made on previous boots when the
> file system wasn't unmounted (which is a regularly seen issue, as ply
> keeps XFS busy during shutdown), resulting in unbootable systems.
>
> Why not just stick to VFAT? As mentioned, it's really the only thing
> generally understood by everything that has a stake in boot
> loading. Grub speaks it. The EFI firmware speaks it (and that also
> means the EFI shell, which is immensly useful). Linux speaks it in the
> initrd and after boot. Windows speaks it. MacOS speaks it. It's the
> lowest common denominator and should be entirely sufficient to store a
> few kernels and their initrds. I mean, we build our kernels as EFI
> binaries on Fedora, IIRC. Wouldn't it be a pity if EFI can't actually
> access them, because they are stored on an fs only Linux speaks?

Anaconda in F28 currently claims /boot cannot be vfat. However, this
appears to be an artificial limitation, because `grub2-install` works
and makes a bootable GRUB with a vfat-typed --boot-directory.

I'm not sure why there would be an issue with /boot being vfat. I guess
two good questions to ask that might offer some insight:

  * What filesystem limitations make vfat unappealing? (do we need
symlinks?)
  * Does Fedora plan to support installing with bootloaders other than
GRUB on x86?

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/UW7ASBMC5VPEELQXZ5GTGC6ZZ3SNNRSX/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Lennart Poettering

On Fr, 22.06.18 19:01, Javier Martinez Canillas (jav...@dowhile0.org) wrote:

> > Whereas constantly changing the ESP, means we need some way to
> > establish a master and rsync to the extras.
> 
> So the consensus seems to be to have the BLS fragments in
> $BOOT/loader/entries even on EFI, where $BOOT is the boot partition
> mounted on /boot. That will give us the following advantages as
> mentioned in this thread:

Uh, as one of the authors of the spec, I am not convinced using
arbitrary non-FAT file systems for $BOOT. In fact the spec currently
says it has to be VFAT. I wouldn't call that "consensus".

Which file system do you have in mind even for this?

As far as I know it's very clear now that boot loaders have a hard
time implementing any of the current file systems properly. AFAIK the
XFS folks as one case were pretty clear that any implementation of XFS
which is not the in-kernel one is not supportable, but I am pretty
sure for the other more modern file systems things aren't too
different either. The fact that grub doesn't properly implement XFS is
a real issue, as I am sure you know, since it won't replay the
journal, and hence doesn't see changes made on previous boots when the
file system wasn't unmounted (which is a regularly seen issue, as ply
keeps XFS busy during shutdown), resulting in unbootable systems.

Why not just stick to VFAT? As mentioned, it's really the only thing
generally understood by everything that has a stake in boot
loading. Grub speaks it. The EFI firmware speaks it (and that also
means the EFI shell, which is immensly useful). Linux speaks it in the
initrd and after boot. Windows speaks it. MacOS speaks it. It's the
lowest common denominator and should be entirely sufficient to store a
few kernels and their initrds. I mean, we build our kernels as EFI
binaries on Fedora, IIRC. Wouldn't it be a pity if EFI can't actually
access them, because they are stored on an fs only Linux speaks?

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/C4EEYIG6HER4QTSPMLCMLESZYDGSHQLI/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Peter Jones

On Mon, Jun 18, 2018 at 02:42:40PM -0700, Andrew Lutomirski wrote:
> > On Jun 18, 2018, at 10:02 AM, Javier Martinez Canillas 
> >  wrote:
> >
> >> On Thu, Jun 14, 2018 at 10:20 PM, Chris Murphy  
> >> wrote:
> >> On Thu, Jun 14, 2018 at 12:51 PM, Adam Williamson
> >>  wrote a monolithic config
> >
> 
> >
> >> The cited BLS spec requires $BOOT be VFAT, are we doing that?
> >>
> >
> > Yes for EFI systems but no otherwise. On EFI the BLS snippets are in
> > /boot/efi/EFI/fedora/loader/entries and on non-EFI systems are in
> > /boot/loader/entries.
> >
> 
> I think this is the wrong approach. I see no valid reason that the
> paths should be different on EFI.

Yeah, I think you've convinced Javier and I that we should just put the
BLS fragments in /boot/loader/entries in either case.  So that will probably
happen next week sometime.

> >> If there's no room on the EFI System partition for all of this, will
> >> we following bullets 2 and 5 of the BLS spec under "The installer
> >
> > No, $BOOT is always the ESP where GRUB 2 is installed.
> 
> I’m going to go out on a limb and make a stronger objection than
> Chris’: I think that $BOOT SHOULD NOT be the ESP. The ESP is
> problematic for any number of reasons. It’s usually vfat, so it’s
> fragile. It does not support RAID safely. And it’s often small.
> 
> Most of this can be solved by putting $BOOT on a different partition.
> Stick it on mdadm 1.1 if you want RAID (*not* 1.0 or 0.9 due to
> corruption risks [0]), and maybe even use a journaling filesystem that
> the bootloader can *correctly* read. (That means the bootloader should
> be able to parse the journal.).  And make it however big you want.

Yeah, I've never understood why some people seem to really want to use
the ESP for anything that doesn't need to be read through the firmware's
file I/O code.  The only thing we really want to be loading from the ESP
is the bootloader itself and some relatively static config data -
basically, how to find /boot.  For simplicity, I expect that means we'll
make it be a grub.cfg that's generated once, and a grubenv file
containing UUIDs and the like.

Once we have most of this working well, I do intend on shipping an
actual grub2-static-config package with a config file that isn't machine
specific at all, but loads everything from bls, small config snippets
(like grub-setpassword makes now), or grubenv, so you don't have to have
grub-mkconfig or the other bulky tools installed at all on platforms
that don't need grub-install.

> As an extra plus, upgrading a kernel doesn’t require mounting the ESP,
> which means that the bootloader installation can sync the ESP across
> multiple disks and it will remain synced.

Yeah, that's a thing you can do.

> All that being said, $BOOT should not use security context xattrs —
> getting that to work right across distros is probably impossible.

Not to agree or disagree, but I'm not sure what of the above led you to
say this part.

> [0] I use mdadm a lot, and I never use 0.9 or 1.0. It’s too fragile.

One caveat here (that's not particularly relevant to the broader
conversation) is that you *can* make /boot and the ESP both reasonably
redundant - obviously by using hardware RAID, but less obviously by
using Intel's IMSM firmware RAID, because they made mdadm support it
pretty well.  But it's present on scarce few platforms in the world.

-- 
  Peter
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/SJYJ7LAK2CWOQEJYCT47EKXLKTOKZISI/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Peter Jones

On Mon, Jun 18, 2018 at 11:55:28PM +0100, Tom Hughes wrote:
> On 18/06/18 23:46, Javier Martinez Canillas wrote:
> > On Mon, Jun 18, 2018 at 11:54 PM, Tom Hughes  wrote:
> > > On 18/06/18 18:15, Peter Jones wrote:
> > > 
> > > > That's true - though we actually shipped nearly all of the code to
> > > > implement this stuff f28, minus some parts of the upgrade story and the
> > > > anaconda bits to enable it by default.  You can go run
> > > > grub2-switch-to-blscfg today, and it will work.  I hope :)
> > > 
> > > 
> > > Unless you have /boot on your root partition like this machine
> > > seems to have for some reason... Then it breaks because the loader
> > > fragments use /vmlinuz... rather than /boot/vmlinuz etc.
> > > 
> > 
> > Yes, /boot not being a mount point was an issue (and also /boot being
> > a btrfs subvolme) but it got fixed [0] a couple of weeks ago. Now the
> > 20-grub.install kernel-install script uses grub2-mkrelpath to get the
> > relative path of the kernel and initramfs images, which is the same
> > that's done by the /etc/grub.d/10_linux script. It's just that the
> > grub2 update didn't made to F28 yet.
> 
> Does that extend to grub2-switch-to-blscfg as well? That was what
> broke my boot.

Yes, I've already got that fixed in my git repo, and it'll be fixed to
do that when I do the next f28 update.

-- 
  Peter
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QESTGIFYYKKBHCBSVYZ4EURLVIO4QTZH/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-22 Thread Javier Martinez Canillas

On Thu, Jun 21, 2018 at 11:19 PM, Chris Murphy  wrote:

[snip]

>>
>>> OK anyway, I don't see broad BLS consensus forming yet, but I do see
>>> two items that aren't controversial and could move forward as part of
>>> this feature proposal:
>>>
>>> a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
>>> the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
>>> existing /boot ext4 volume currently used). i.e. do not put
>>> /loader/entries in /EFI/fedora
>>
>> By "consistent", do you mean that both EFI and BIOS boot loaders will
>> reference the same entry files? I like that.
>
> Yes.
>
>> However, I don't like the entries existing on ESP mostly because of the
>> use case of md-RAID for /boot referenced in another thread. I think it
>> would be best to just put the GRUB EFI file on ESP and put the rest of
>> the bulk GRUB stuff in its utility partition (which may be RAID-ed).
>
> Right. The config + kernel + initramfs on traditional /boot can make
> use of software raid, depending on static one time proper
> configuration of each member drive's ESPs and now the ESPs never need
> to be touched and thus not sync'd.
>
> Whereas constantly changing the ESP, means we need some way to
> establish a master and rsync to the extras.
>

So the consensus seems to be to have the BLS fragments in
$BOOT/loader/entries even on EFI, where $BOOT is the boot partition
mounted on /boot. That will give us the following advantages as
mentioned in this thread:

1. The BLS will not be stored in vfat, so ostree could keep using
symlinks to do atomic swap of its /boot/loader/entries
2. The ESP won't be modified on kernels install / removal, that will
make it easier for software RAID.
3. There will be consistency on where the BLS fragments are installed
regardless of the firmware interface (EFI, BIOS, OPAL on ppc64le and
zipl on s390x will all use /boot/loader/entries).

I've updated the wiki page to reflect this and we will also change the
implementation.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/B3SMCSWFFEYIJIWPESY27MB6AMH3DH52/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Thu, Jun 21, 2018 at 8:53 PM, Kyle Marek  wrote:

> I've noticed that Windows 10 does MBR installs on BIOS, as well. I've
> always found that interesting because I have also found several laptops
> (I think most of them were HP) where the OEM installed a BIOS bootloader
> in addition to the EFI bootloader. My guess is that these OEMs see a
> similar value in being able to boot both BIOS and EFI.

I've seen gdisk -l output from such systems. I've never interacted
with one. The most common OEM "installer" is just an imager. Basically
it's dd'ing an image, so it it includes all the partitioning and
contents of those partitions, it's not like the Microsoft installer.
It might be that the manufacturer used the same image on a couple of
different SKUs where the only difference was whether they had legacy
BIOS enabled. I don't suppose they ever intended to have a flip from
BIOS to UEFI in the field, but maybe?

>> Are you gonna own the feature? :-) Maybe Colin Walters finds it
>> compelling enough to be co-owner?
>
> Can I? I'm a bit new to Fedora development (I don't have a FAS account,
> yet).

Sure. Jump into the deep end. Shout if you need help. Quick like a
bunny though if you want to get it in for Fedora 29.

But definitely start a new separate devel@ thread. This one's
approaching a hijacking. In the new thread you can point out the two
pronged approach, what liabilities there are, solicit liabilities you
haven't thought of. You could cull bugzilla for anaconda and grub2
bugs related to GPT and not booting for hardware models that were
affected, and include that list of models in the kickoff email. Maybe
that gets the attention of would be testers, but also at least puts
people still using such hardware some notice (even in future searches)
should they do a clean install, what problems they might run into. For
testing, it's been six years so it's plausible there have been
firmware updates fixing some or even most of these problems. Perhaps
other GRUB upstream work arounds were discovered.

One big con is the unknown factor, and how to get broad testing
coverage. A contrary position to that is, we didn't know about any
problems six years ago when the switch was flipped, and then we ran
into the problem. So how is it any worse of an idea now than it was
originally? The change is still valid as long as a bunch of people
aren't negatively impacted. Also, it's not change for the sake of
change, it has a followup feature that does incrementally simplify the
layout and makes it a bit more flexible at pretty much no cost.

> However, I do care about correcting things like this. I've found
> Fedora's way of doing boot loader configuration a little strange and
> confusing compared to other distros. If Fedora is open to contribution
> by new members, I would be happy to contribute.

Yes it is, but contribution in the context of actually changing
inertia means producing the changes: patches, commits, documentation,
tests, bug reports. Things that get much less attention are mere
feature requests without committing resources to producing a tangible
result. And also, this applies to the current spec convo
https://xkcd.com/927/

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/H5YKG4ZF65IYWR3XVGFAJLHSIAW75GDR/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On 06/21/2018 10:28 PM, Chris Murphy wrote:
> On Thu, Jun 21, 2018 at 3:59 PM, Kyle Marek  wrote:
>
>> If it helps, I've only ever used GRUB on GPT when installing to BIOS
>> systems. I haven't encountered *any* issues so far.
> It was always model specific. Maybe 1/2 dozen models were affected.
> https://bugzilla.redhat.com/show_bug.cgi?id=755226
> https://bugzilla.redhat.com/show_bug.cgi?id=754850
> https://bugzilla.redhat.com/show_bug.cgi?id=741120
>
> In one case, the firmware won't boot if it doesn't see the active bit
> set on a partition in the MBR, and yet the UEFI spec says no partition
> should have an active bit set in a PMBR. And the workaround caused
> worse problems (blast from the past):
> https://bugzilla.redhat.com/show_bug.cgi?id=754850#c8
> https://bugzilla.redhat.com/show_bug.cgi?id=754850#c9
>
>
> Anyway, there must be some other bug that ultimately caused GPT to be
> abandoned, I'm not finding it at the moment though.

Interesting.

>> There was a lot to digest with the above and the GPT-default-splatting
>> part. Just for clarification, are you "completely" opposed to installing
>> GPT on BIOS systems by default *until* there is reason to believe that
>> the issues described were now-fixed GRUB bugs?
> I'm not opposed to someone looking into it. But I'm opposed to just
> assuming it's all going to work out OK. And it's not up to me anyway.
>
> It might qualify as a system wide change, deadline for which is July
> 3. The full change is GPT on BIOS by default *and* including ESP and
> BIOSBoot partitions by default needs a conversation with
> anaconda/blivet folks if they would accept patches to make that happen
> in the Fedora 29 timeframe. They need to be asked first, there's no
> point in pushing a feature proposal by a July 3 deadline if the
> anaconda folks won't merge the change. And you'd need to own the
> feature or find someone to own it, and go through the process.
> Basically realize what you might very well be counting on, is that
> buggy BIOS computers are now too old and this has since been fixed.
> But there's no evidence for this one way or another, as far as I know.
> For example, Windows 10's installer today in 2018, will only use MBR
> on a drive when the computer boots in BIOS mode. So there's maybe not
> much external testing for this. I'm not sure what other distros do by
> default. That might be a useful data point.

I've noticed that Windows 10 does MBR installs on BIOS, as well. I've
always found that interesting because I have also found several laptops
(I think most of them were HP) where the OEM installed a BIOS bootloader
in addition to the EFI bootloader. My guess is that these OEMs see a
similar value in being able to boot both BIOS and EFI.

> It may be easier to go for just GPT on BIOS by default, for Fedora 29.
> If that doesn't blow up, then go for the partition changes in Fedora
> 30.

That sounds like a fair way to make this manageable.

>> Beyond the benefit of being able to boot EFI and GPT by default, there
>> is also the benefit of not storing GRUB in the gap before the first MBR
>> partition (I think this is *especially* eyebrow raising on the MBR side
>> of things), and the benefits of having GPT in general such as a lack of
>> a partition limit, backup partition table, and support for drives larger
>> than 2 TiB (though it needs to be noted that the partitions that need to
>> be accessed with BIOS calls need to exist within the first 2 TiB, or 8
>> GiB for compatibility with really stupid/old BIOSes).
> For what it's worth, if the BIOS system has a drive bigger than 2TB,
> then GPT is used by default. This has been true since at least Fedora
> 17.
>
>
>> Once the GPT-splatting issue is better understood, I really think that
>> if GPT-by-default can be considered at all, it should be.
>>
> Are you gonna own the feature? :-) Maybe Colin Walters finds it
> compelling enough to be co-owner?

Can I? I'm a bit new to Fedora development (I don't have a FAS account,
yet).

However, I do care about correcting things like this. I've found
Fedora's way of doing boot loader configuration a little strange and
confusing compared to other distros. If Fedora is open to contribution
by new members, I would be happy to contribute.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ZOZ6C6IZA5Q64JW6XUCREGF657XECWI6/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Thu, Jun 21, 2018 at 3:59 PM, Kyle Marek  wrote:

> If it helps, I've only ever used GRUB on GPT when installing to BIOS
> systems. I haven't encountered *any* issues so far.

It was always model specific. Maybe 1/2 dozen models were affected.
https://bugzilla.redhat.com/show_bug.cgi?id=755226
https://bugzilla.redhat.com/show_bug.cgi?id=754850
https://bugzilla.redhat.com/show_bug.cgi?id=741120

In one case, the firmware won't boot if it doesn't see the active bit
set on a partition in the MBR, and yet the UEFI spec says no partition
should have an active bit set in a PMBR. And the workaround caused
worse problems (blast from the past):
https://bugzilla.redhat.com/show_bug.cgi?id=754850#c8
https://bugzilla.redhat.com/show_bug.cgi?id=754850#c9

Anyway, there must be some other bug that ultimately caused GPT to be
abandoned, I'm not finding it at the moment though.

> There was a lot to digest with the above and the GPT-default-splatting
> part. Just for clarification, are you "completely" opposed to installing
> GPT on BIOS systems by default *until* there is reason to believe that
> the issues described were now-fixed GRUB bugs?

I'm not opposed to someone looking into it. But I'm opposed to just
assuming it's all going to work out OK. And it's not up to me anyway.

It might qualify as a system wide change, deadline for which is July
3. The full change is GPT on BIOS by default *and* including ESP and
BIOSBoot partitions by default needs a conversation with
anaconda/blivet folks if they would accept patches to make that happen
in the Fedora 29 timeframe. They need to be asked first, there's no
point in pushing a feature proposal by a July 3 deadline if the
anaconda folks won't merge the change. And you'd need to own the
feature or find someone to own it, and go through the process.
Basically realize what you might very well be counting on, is that
buggy BIOS computers are now too old and this has since been fixed.
But there's no evidence for this one way or another, as far as I know.
For example, Windows 10's installer today in 2018, will only use MBR
on a drive when the computer boots in BIOS mode. So there's maybe not
much external testing for this. I'm not sure what other distros do by
default. That might be a useful data point.

It may be easier to go for just GPT on BIOS by default, for Fedora 29.
If that doesn't blow up, then go for the partition changes in Fedora
30.

> Beyond the benefit of being able to boot EFI and GPT by default, there
> is also the benefit of not storing GRUB in the gap before the first MBR
> partition (I think this is *especially* eyebrow raising on the MBR side
> of things), and the benefits of having GPT in general such as a lack of
> a partition limit, backup partition table, and support for drives larger
> than 2 TiB (though it needs to be noted that the partitions that need to
> be accessed with BIOS calls need to exist within the first 2 TiB, or 8
> GiB for compatibility with really stupid/old BIOSes).

For what it's worth, if the BIOS system has a drive bigger than 2TB,
then GPT is used by default. This has been true since at least Fedora
17.

>
> Once the GPT-splatting issue is better understood, I really think that
> if GPT-by-default can be considered at all, it should be.
>

Are you gonna own the feature? :-) Maybe Colin Walters finds it
compelling enough to be co-owner?

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/U4TED5PGT4TWAHEP3UJHRU2CC2BXF6CO/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On 06/21/2018 05:19 PM, Chris Murphy wrote:
> On Thu, Jun 21, 2018 at 2:28 PM, Kyle Marek  wrote:
>> On 06/21/2018 03:07 PM, Chris Murphy wrote:
>>> On Thu, Jun 21, 2018 at 11:12 AM, Kyle Marek  wrote:
>>>
 I would greatly appreciate a move to a uniform GPT+EF02+EF00
 partitioning default with a shared boot loader config.
>>> An ESP on BIOS is perhaps weird when making so much of this booting
>>> and startup stuff user facing; but it's actually integrated in the
>>> Matthew Garrett derivative of the BootLoaderSpec and is used as $BOOT.
>> I suppose it is weird without the context of EFI, but there's no reason
>> for it to cause issues.
> Weird is better than inconsistency!

Agreed.

>>> In ancient times, there were some BIOS firmware floating about that
>>> would face plant on encountering GPT, we tried it by default and it
>>> caused too many problems and had to be reverted but that was long
>>> enough ago it's possible such hardware is quite rare.
>> Wow, that really sucks, especially since most GPTs implement a
>> compatibility/protective MBR, anyway.
>>
>> There's really no reason for this behavior to exist, but since it
>> did/does (and since it should be a minority), would it be reasonable to
>> invert the logic regarding inst.gpt? (default to GPT and use MBR only if
>> "inst.mbr" is set). I would think this is fair because the
>> choking-on-GPT issue really is more of a firmware bug than something
>> that should be treated as the lowest-common-denominator.
> Likely old hardware no longer getting firmware updates. And since GPT
> is a creation of the UEFI spec, I don't know whether BIOS firmware can
> be held accountable when splatting upon encountering GPT. I don't
> remember why it splats. (When I said earlier "we tried this" that's
> the Fedora Royal We, as in Anaconda once switched to GPT by default
> and it caused too many problems for blacklisting to resolve.) Since
> bugs in the RHBZ are immortal, there might be a hint buried in the bug
> archive. There's also a hint here:
>
> /usr/share/doc/syslinux/gpt.txt
>
> I've got way too much experience with the madness of hybrid MBR on
> Macs using "Bootcamp" to support Windows; I want all of those weeks of
> time refunded to me. This is a huge FUUUCK N!
>
> But the new protocol section of that file is eyebrow raising. This
> mbrgpt.bin thing is not new, it's been around a while, and makes me
> wonder if the "bug" was originally in GRUB's initial jump code written
> to the first 440 bytes of LBA 0. Huh! This could be a long ago solved
> problem by now, as it's never been revisited in Fedora land.

If it helps, I've only ever used GRUB on GPT when installing to BIOS
systems. I haven't encountered *any* issues so far.

> Anyway, as for GPT by default and inst.mbr as the fallback - the
> problem is that a failure puts the user into a black hole. It's
> totally opaque to them that they'd need to use inst.mbr to get out.
> I'd rather fail safe. Fail danger is only OK if the alternative is
> fail guaranteed. No I think we're obligated to demonstrate on X number
> of models known to splat with Fedora 18/19 (or whatever it was) GRUB +
> GPT, no longer splats with Fedora 28/29 GRUB +GPT.

From my own experience, I really feel as if this issue doesn't exist
anymore. However, for machines where it might, I think that good
documentation might be enough to mitigate the black hole issue.

In addition, decisions about default partitioning really only apply to
new installs on unpartitioned disks. All I am trying to say here is that
initial failure to install doesn't exactly result in a loss of anything.
Please correct me if I am wrong.

> And in any case before that time, it should be simple to pass inst.gpt
> within the Fedora compose system to get VM images that always use GPT.
> I've never heard of a VM BIOS having this bug, but if it's true it
> should be and can be fixed.
>> There's also the issue that a lot of PXE firmwares boot BIOS-only on EFI
>> machines. Personally, I address this by loading iPXE off of a flash
>> drive, but for organizations that can't/won't do this, it would be
>> useful to make an EFI-capable install even if the installer was booted
>> with BIOS for this reason.
>>
>>> OK anyway, I don't see broad BLS consensus forming yet, but I do see
>>> two items that aren't controversial and could move forward as part of
>>> this feature proposal:
>>>
>>> a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
>>> the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
>>> existing /boot ext4 volume currently used). i.e. do not put
>>> /loader/entries in /EFI/fedora
>> By "consistent", do you mean that both EFI and BIOS boot loaders will
>> reference the same entry files? I like that.
> Yes.
>
>> However, I don't like the entries existing on ESP mostly because of the
>> use case of md-RAID for /boot referenced in another thread. I think it
>> would be best to just put the GRUB EFI file on ESP and put the rest of
>>

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Thu, Jun 21, 2018 at 2:28 PM, Kyle Marek  wrote:
> On 06/21/2018 03:07 PM, Chris Murphy wrote:
>> On Thu, Jun 21, 2018 at 11:12 AM, Kyle Marek  wrote:
>>
>>> I would greatly appreciate a move to a uniform GPT+EF02+EF00
>>> partitioning default with a shared boot loader config.
>> An ESP on BIOS is perhaps weird when making so much of this booting
>> and startup stuff user facing; but it's actually integrated in the
>> Matthew Garrett derivative of the BootLoaderSpec and is used as $BOOT.
>
> I suppose it is weird without the context of EFI, but there's no reason
> for it to cause issues.

Weird is better than inconsistency!

>
>> In ancient times, there were some BIOS firmware floating about that
>> would face plant on encountering GPT, we tried it by default and it
>> caused too many problems and had to be reverted but that was long
>> enough ago it's possible such hardware is quite rare.
>
> Wow, that really sucks, especially since most GPTs implement a
> compatibility/protective MBR, anyway.
>
> There's really no reason for this behavior to exist, but since it
> did/does (and since it should be a minority), would it be reasonable to
> invert the logic regarding inst.gpt? (default to GPT and use MBR only if
> "inst.mbr" is set). I would think this is fair because the
> choking-on-GPT issue really is more of a firmware bug than something
> that should be treated as the lowest-common-denominator.

Likely old hardware no longer getting firmware updates. And since GPT
is a creation of the UEFI spec, I don't know whether BIOS firmware can
be held accountable when splatting upon encountering GPT. I don't
remember why it splats. (When I said earlier "we tried this" that's
the Fedora Royal We, as in Anaconda once switched to GPT by default
and it caused too many problems for blacklisting to resolve.) Since
bugs in the RHBZ are immortal, there might be a hint buried in the bug
archive. There's also a hint here:

/usr/share/doc/syslinux/gpt.txt

I've got way too much experience with the madness of hybrid MBR on
Macs using "Bootcamp" to support Windows; I want all of those weeks of
time refunded to me. This is a huge FUUUCK N!

But the new protocol section of that file is eyebrow raising. This
mbrgpt.bin thing is not new, it's been around a while, and makes me
wonder if the "bug" was originally in GRUB's initial jump code written
to the first 440 bytes of LBA 0. Huh! This could be a long ago solved
problem by now, as it's never been revisited in Fedora land.

Anyway, as for GPT by default and inst.mbr as the fallback - the
problem is that a failure puts the user into a black hole. It's
totally opaque to them that they'd need to use inst.mbr to get out.
I'd rather fail safe. Fail danger is only OK if the alternative is
fail guaranteed. No I think we're obligated to demonstrate on X number
of models known to splat with Fedora 18/19 (or whatever it was) GRUB +
GPT, no longer splats with Fedora 28/29 GRUB +GPT.

And in any case before that time, it should be simple to pass inst.gpt
within the Fedora compose system to get VM images that always use GPT.
I've never heard of a VM BIOS having this bug, but if it's true it
should be and can be fixed.

> There's also the issue that a lot of PXE firmwares boot BIOS-only on EFI
> machines. Personally, I address this by loading iPXE off of a flash
> drive, but for organizations that can't/won't do this, it would be
> useful to make an EFI-capable install even if the installer was booted
> with BIOS for this reason.
>
>> OK anyway, I don't see broad BLS consensus forming yet, but I do see
>> two items that aren't controversial and could move forward as part of
>> this feature proposal:
>>
>> a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
>> the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
>> existing /boot ext4 volume currently used). i.e. do not put
>> /loader/entries in /EFI/fedora
>
> By "consistent", do you mean that both EFI and BIOS boot loaders will
> reference the same entry files? I like that.

Yes.

> However, I don't like the entries existing on ESP mostly because of the
> use case of md-RAID for /boot referenced in another thread. I think it
> would be best to just put the GRUB EFI file on ESP and put the rest of
> the bulk GRUB stuff in its utility partition (which may be RAID-ed).

Right. The config + kernel + initramfs on traditional /boot can make
use of software raid, depending on static one time proper
configuration of each member drive's ESPs and now the ESPs never need
to be touched and thus not sync'd.

Whereas constantly changing the ESP, means we need some way to
establish a master and rsync to the extras.

>
> I think GRUB using its own partition is fair especially because GRUB
> isn't an EFI-only bootloader.
>
>> b. If GPT, installer always creates both EF02 and EF00 partitions. For
>> creating VM images, I think it's sane if the anaconda inst.gpt option
>> is always used to make sure the image is created with

Re: F29 System Wide Change: Make BootLoaderSpec the default

On 06/21/2018 03:07 PM, Chris Murphy wrote:
> On Thu, Jun 21, 2018 at 11:12 AM, Kyle Marek  wrote:
>
>> I would greatly appreciate a move to a uniform GPT+EF02+EF00
>> partitioning default with a shared boot loader config.
> An ESP on BIOS is perhaps weird when making so much of this booting
> and startup stuff user facing; but it's actually integrated in the
> Matthew Garrett derivative of the BootLoaderSpec and is used as $BOOT.

I suppose it is weird without the context of EFI, but there's no reason
for it to cause issues.

> In ancient times, there were some BIOS firmware floating about that
> would face plant on encountering GPT, we tried it by default and it
> caused too many problems and had to be reverted but that was long
> enough ago it's possible such hardware is quite rare.

Wow, that really sucks, especially since most GPTs implement a
compatibility/protective MBR, anyway.

There's really no reason for this behavior to exist, but since it
did/does (and since it should be a minority), would it be reasonable to
invert the logic regarding inst.gpt? (default to GPT and use MBR only if
"inst.mbr" is set). I would think this is fair because the
choking-on-GPT issue really is more of a firmware bug than something
that should be treated as the lowest-common-denominator.

I would think it's more useful to have the GPT-default installations
supporting EF02 and EF00 than to use only MBR or EFI-only-GPT if it is
true that the anti-GPT BIOSes are rare.

There's also the issue that a lot of PXE firmwares boot BIOS-only on EFI
machines. Personally, I address this by loading iPXE off of a flash
drive, but for organizations that can't/won't do this, it would be
useful to make an EFI-capable install even if the installer was booted
with BIOS for this reason.

> OK anyway, I don't see broad BLS consensus forming yet, but I do see
> two items that aren't controversial and could move forward as part of
> this feature proposal:
>
> a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
> the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
> existing /boot ext4 volume currently used). i.e. do not put
> /loader/entries in /EFI/fedora

By "consistent", do you mean that both EFI and BIOS boot loaders will
reference the same entry files? I like that.

However, I don't like the entries existing on ESP mostly because of the
use case of md-RAID for /boot referenced in another thread. I think it
would be best to just put the GRUB EFI file on ESP and put the rest of
the bulk GRUB stuff in its utility partition (which may be RAID-ed).

I think GRUB using its own partition is fair especially because GRUB
isn't an EFI-only bootloader.

> b. If GPT, installer always creates both EF02 and EF00 partitions. For
> creating VM images, I think it's sane if the anaconda inst.gpt option
> is always used to make sure the image is created with GPT
> partitioning, thereby making sure they get EF02 and EF00.

EF02 and EF00 good. I would prefer if this wasn't an EFI-only
conditional due to my above responses.

Thank you for your consideration!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/XYACTYIRY2WGJXCJXG4IIYCJ636HXSPS/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Thu, Jun 21, 2018 at 11:12 AM, Kyle Marek wrote:

> I would greatly appreciate a move to a uniform GPT+EF02+EF00
> partitioning default with a shared boot loader config.

An ESP on BIOS is perhaps weird when making so much of this booting
and startup stuff user facing; but it's actually integrated in the
Matthew Garrett derivative of the BootLoaderSpec and is used as $BOOT.

In ancient times, there were some BIOS firmware floating about that
would face plant on encountering GPT, we tried it by default and it
caused too many problems and had to be reverted but that was long
enough ago it's possible such hardware is quite rare.

OK anyway, I don't see broad BLS consensus forming yet, but I do see
two items that aren't controversial and could move forward as part of
this feature proposal:

a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
existing /boot ext4 volume currently used). i.e. do not put
/loader/entries in /EFI/fedora

b. If GPT, installer always creates both EF02 and EF00 partitions. For
creating VM images, I think it's sane if the anaconda inst.gpt option
is always used to make sure the image is created with GPT
partitioning, thereby making sure they get EF02 and EF00.

--
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/2PVYRSMT5ZVR63UM6MMWH4XBLMO3CQ7O/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On 06/21/2018 09:41 AM, Gerd Hoffmann wrote:
>   Hi,
>
>> From my perspective (Fedora CoreOS developer) that straddles
>> both physical and cloud for the server case, the problem is that
>> the virtualized case, and in particular public cloud, and really
>> specifically EC2 - no one really cares about EFI to boot their VMs.
> Indeed.  And it sucks.
>
>>>   Device   Start End Sectors  Size Type
>>>   /dev/sda1 2048   1023981924M BIOS boot
>>>   /dev/sda210240 1058815 1048576  512M EFI System
>> I wouldn't *oppose* that; in fact if you (or someone else)
>> wanted to push for that with e.g. Fedora CoreOS, I'd be happy
>> to discuss it.  But it's not like it has a truly compelling advantage
>> over what we ship today - it'd just be *another* weird variant
>> of things in the end right?
> Well, as *additional* variant it doesn't provide that much value.  More
> interesting would be to create all x86 cloud images that way, so they
> boot just fine on both bios and efi, and we don't have to bother
> creating two image variants.

I've been manually partitioning physical machines that I install Fedora
on for exactly this reason.

Similar to moving to VMs, it is especially useful when you need to
replace hardware (where the new hardware uses a different firmware), as
you can just put the drives in the new machine. Under the current
automatic partitioning, a switch from BIOS to EFI requires
repartitioning (both changing partition table type *and* partition sizes
to make room for ESP), which may even be "impossible" if the system is
on XFS.

I would greatly appreciate a move to a uniform GPT+EF02+EF00
partitioning default with a shared boot loader config.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/K4G7HVPH7AP35DSVDHLS5XFTAYJ654ZP/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Thu, Jun 21, 2018 at 10:04:03AM -0400, Colin Walters wrote:
> 
> 
> On Thu, Jun 21, 2018, at 9:41 AM, Gerd Hoffmann wrote:
> >
> > Well, as *additional* variant it doesn't provide that much value.  More
> > interesting would be to create all x86 cloud images that way, so they
> > boot just fine on both bios and efi, and we don't have to bother
> > creating two image variants.
> 
> We aren't creating two[1] image variants today.  Why would we?  Who
> would use them and why?

Well, sooner or later we will have to make the switch to uefi in
virtualization, following the trend on physical hardware which ships
with uefi firmware since years.  And having cloud images which work
fine with both bios and uefi will make the transition easier ...

cheers,
  Gerd
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/AMHRLHKX6FXKYFI4PFKNHPURUROMPTQ2/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-21 Thread Colin Walters

On Thu, Jun 21, 2018, at 9:41 AM, Gerd Hoffmann wrote:
>
> Well, as *additional* variant it doesn't provide that much value.  More
> interesting would be to create all x86 cloud images that way, so they
> boot just fine on both bios and efi, and we don't have to bother
> creating two image variants.

We aren't creating two[1] image variants today.  Why would we?  Who
would use them and why?

> Why do you need that?  Wouldn't FAH just drop a new file for the new
> version into /boot/loader/entries on updates?  So you have old and new
> version listed in the boot menu and can easily rollback to the old
> version if needed?

The entire design of libostree is around being able to transactionally
swap the bootloader configuration.  While preparing an update, we
temporarily have *three* deployments on disk (new, current, and rollback),
but only two bootloader entries.  If we weren't able to do a full
transactional replacement then we'd have to deal with possibly
having three entries, and be able to clean that up afterwards.
Which we could do - particularly now that we have
https://github.com/ostreedev/ostree/pull/1464
and it's easier for admins to control what's available locally, and we
can assume that we can just GC other things.

[1] For BIOS/UEFI - clearly there are a ton of image variants in general
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/H3O7QDPZWA4YXIWNSJOHFHJKOZKJC2SW/

Re: F29 System Wide Change: Make BootLoaderSpec the default

  Hi,

> From my perspective (Fedora CoreOS developer) that straddles
> both physical and cloud for the server case, the problem is that
> the virtualized case, and in particular public cloud, and really
> specifically EC2 - no one really cares about EFI to boot their VMs.

Indeed.  And it sucks.

> >   Device   Start End Sectors  Size Type
> >   /dev/sda1 2048   1023981924M BIOS boot
> >   /dev/sda210240 1058815 1048576  512M EFI System
> 
> I wouldn't *oppose* that; in fact if you (or someone else)
> wanted to push for that with e.g. Fedora CoreOS, I'd be happy
> to discuss it.  But it's not like it has a truly compelling advantage
> over what we ship today - it'd just be *another* weird variant
> of things in the end right?

Well, as *additional* variant it doesn't provide that much value.  More
interesting would be to create all x86 cloud images that way, so they
boot just fine on both bios and efi, and we don't have to bother
creating two image variants.

> And the fact that FAT has no ability to do atomic replacement bothers
> me a lot. (A wrinkle here is that ostree implements an atomic swap of
> /boot/loader - you can today boot FAH/Silverblue and just ls -al /boot
> to see it)

Why do you need that?  Wouldn't FAH just drop a new file for the new
version into /boot/loader/entries on updates?  So you have old and new
version listed in the boot menu and can easily rollback to the old
version if needed?

But anyway: The scheme works with and without separate /boot.

cheers,
  Gerd
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/L4Q7ILZRWGWGVH77UCVMWG52FMATLR5X/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-21 Thread Colin Walters

On Thu, Jun 21, 2018, at 3:30 AM, Gerd Hoffmann wrote:
>   Hi,
> 
> > And in my opinion, it's not simple to say: OK if you have this size
> > ESP to start, you get this layout, and if it's bigger you get this
> > other layout, and if it's BIOS you have this 3rd layout.

Chris, I have to say I'm glad you're part of the Fedora community - your
input on this topic has been very valuable!

> Well, for fresh installs[1] there is no reason to have efi and bios use
> different layouts.  You can just do this:

When you say "install" it really matters to say install of *what* - a
desktop system, a physical server, a VM, etc.

From my perspective (Fedora CoreOS developer) that straddles
both physical and cloud for the server case, the problem is that
the virtualized case, and in particular public cloud, and really
specifically EC2 - no one really cares about EFI to boot their VMs.
Except a special case here is "disaster recovery" scenarios where
a physical server is imaged and uploaded to the cloud as a VM,
and the topic of UEFI does come up there.  Apparently most
implementations of this convert back to BIOS.

Don't get me wrong, I agree with Lennart (indirectly) in that it is
kind of crazy how influentual the "Windows dual boot for desktop"
case is on everything Fedora, which also includes physical
servers.  But the virtualized case also pushes at this from the other
angle.

>   [root@ibm-p8-kvm-03-guest-02 ~]# fdisk -l /dev/sda
>   Disk /dev/sda: 4 GiB, 4294967296 bytes, 8388608 sectors
>   Units: sectors of 1 * 512 = 512 bytes
>   Sector size (logical/physical): 512 bytes / 512 bytes
>   I/O size (minimum/optimal): 512 bytes / 512 bytes
>   Disklabel type: gpt
>   Disk identifier: 92035660-BEFD-45D5-9883-B2B91EC429D1
> 
>   Device   Start End Sectors  Size Type
>   /dev/sda1 2048   1023981924M BIOS boot
>   /dev/sda210240 1058815 1048576  512M EFI System

I wouldn't *oppose* that; in fact if you (or someone else)
wanted to push for that with e.g. Fedora CoreOS, I'd be happy
to discuss it.  But it's not like it has a truly compelling advantage
over what we ship today - it'd just be *another* weird variant
of things in the end right?  

And the fact that FAT has no ability to do atomic replacement bothers
me a lot. (A wrinkle here is that ostree implements an atomic swap of
/boot/loader - you can today boot FAH/Silverblue and just ls -al /boot
to see it)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/MNRANIETDFYCPU5RN3MEW65W6PFSI7C2/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-21 Thread Lennart Poettering

On Mi, 20.06.18 19:28, Chris Murphy (li...@colorremedies.com) wrote:

> >> Except, it's not simple for installers to migrate to a new bigger ESP
> >> in the dual boot case. And having different layouts for UEFI and BIOS
> >> and whether there's dual boot or single boot, isn't simpler.
> >
> > Again, if you don't want to resize the ESP, then go for option #2
> > above. But if the ESP is usable, then go for option #1.
> 
> In 100% of the cases where the ESP already exists, it is too small to
> share. No one. Not Apple, not Microsoft, not Windows OEMs, not Fedora,
> not any distro, creates an ESP bigger than 550MB. Typical is 99MB for
> the Microsoft installer (I have a laptop partitioned by Microsoft's
> install, not an OEM installer, and it's 99MB), and 128MB for Apple,
> and 200MB for Linux distros. None of these are big enough to share.

On my Lenovo I got an ESP of 256M, and I use it happily and without
issues for systemd-boot. Maybe that's anecdotal, but all this entirely
besides the point.

Fedora is not a system that is exclusively dual-booted. it's entirely
fine to follow slightly different setups if fedora is installed onto a
system that already has Windows installed, or if a fresh full-disk
image is generated for it, that can be installed by "dd" or such.

All I am saying is that if you built a clean image, i.e. do not do the
augment-an-existing-windows-installation dance then there's really no
point in doing two partitions...

(And quite frankly, I still don't buy the "ESP resizing is totally
impossible" thing. It's not. When you install Fedora onto an existing
Windows installation disk, you have to resize/move the Windows NTFS
partitions anyway to make space for Fedora. And if you do, you might
as well move the ESP too. I mean resizing/moving the ESP is a lot
simpler and less dangerous than resizing/moving NTFS. But this
discussion is entirely pointless anyway, as $BOOT may be separate from
the ESP according the spec.)

> And the ESP partition is wedged in, again in 100% of cases. It can't
> be resized in place.
> 
> Therefore, Option #2 will be extremely common. What percent of Fedora
> users dual boot? I have no empirical data. I'd guess 1/2.

Fedora generates cloud images and suchlike. All those images really
don't need to bother with compat with Windows.

> You have to decide which is more important. Broad adoption, which will
> require equal doses of compromise and simplicity. Or narrow
> adoption.

Yupp, compromise is already built into the spec. If it wasn't for
compromise then $BOOT would not exist as a concept, and we'd just
always use the ESP.

> And as Fedora is right now looking to implement BLS, what did they
> actually do? Adopt the BLS file format and drop in concept, and
> abandon the other 90% of the spec by punting.

Yeah, what Fedora is doing has nothing to do with the boot loader
spec, that's true. It should really drop referencing the spec.

But seriously, $BOOT may be separate from the ESP. It's fine if Fedora
implements it separately, and totally conforming to the spec. I am not
sure what you even are insisting on here. You appear to say that
merging the two should be *against* the spec. But why do you even
care about that? You can totally choose to implement the "keep $BOOT
separate from the ESP" part, and ignore the "merge $BOOT with ESP"
part. It's *entirely* fine if Fedora does it that way. 

> I'll tell you what. Maybe consider a general purpose layout and a
> simplified layout. The typical layout represents a compromise no
> matter the firmware, and no matter what OS is already present - your
> option 2. This would be used for workstations, and any case where dual
> or multiboot is expected. And for things like Fedora VM images, IoT,
> possibly server, possibly ARM - where the sharing aspect of $BOOT is
> not expected or a consideration, go with the simplified layout - your
> option 1.

But this is what the spec pretty much already says! It says: merge it
if possible, split if if needed. How you define "possible" and
"needed" is up to you. All the spec tries to make sure though is that
once the decision is made for a specific image the other parties that
might want to process the entries know how to find the thing.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/TPWDFQ5GJLVHGG44PHI4Z7TPGHJCXSYJ/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-21 Thread Lennart Poettering

On Mi, 20.06.18 19:40, Chris Murphy (li...@colorremedies.com) wrote:

> > What's going on? What is this? Why is this called "Boot loader spec"
> > if it implements an entirely different logic, and misses the entire
> > point of the boot loader spec?
> >
> > Quite frankly, I am really surprised by this and this makes me wonder
> > what the whole point of this feature is at all, and very sure we
> > shouldn't have it like this.
> 
> I agree.
> 
> But I also think there is some incremental risk of namespace collision
> with /loader/entries as mentioned in Matthew Garrett's derivative of
> the spec, if it's not located in /EFI/fedora/
> 
> Could $BOOT/org/freedesktop/bls/entries instead be
> $BOOT/org.freedesktop.bls/entries ?

The boot loader spec is already implemented in various tools under the
very "$BOOT/loader/entries" name (just read it aloud, it tells you
exactly what this is), I doubt there's need to rename it now. it's not
that the namespace below $BOOT or ESP is particularly crowded,
anyway. Also, I like my bikesheds blue.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/2MTPVW265CZ6O4OUETABRLQE4OQOZDN6/

Re: F29 System Wide Change: Make BootLoaderSpec the default

  Hi,

> Therefore, Option #2 will be extremely common. What percent of Fedora
> users dual boot? I have no empirical data. I'd guess 1/2.

Sure?  I would expect in the age of virtualization people prefer
virtual machines, because you can run fedora and $someotheros at the
same time then.

The installer must be able to handle this even if it is 10% only, so the
numbers don't change much on the fundamental issue.

I'm just curious ...

cheers,
  Gerd
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/GZPZAAH5YPXJK7GZXPIKQNPOIKWN3LLU/

Re: F29 System Wide Change: Make BootLoaderSpec the default

Hi,

> And in my opinion, it's not simple to say: OK if you have this size
> ESP to start, you get this layout, and if it's bigger you get this
> other layout, and if it's BIOS you have this 3rd layout.

Well, for fresh installs[1] there is no reason to have efi and bios use
different layouts. You can just do this:

[root@ibm-p8-kvm-03-guest-02 ~]# fdisk -l /dev/sda
Disk /dev/sda: 4 GiB, 4294967296 bytes, 8388608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 92035660-BEFD-45D5-9883-B2B91EC429D1

Device Start End Sectors Size Type
/dev/sda1 2048 1023981924M BIOS boot
/dev/sda210240 1058815 1048576 512M EFI System
/dev/sda3 1058816 2107391 1048576 512M Linux swap
/dev/sda4 2107392 8386560 62791693G Linux root (x86-64)

systemd-boot plus bootloader config plus kernels goes to /dev/sda2.
grub2 (bios version, with BLS patches) goes to /dev/sda1. Short,
fixed config snippet instructs grub2 to read the bls config from
/dev/sda2.

That image can be booted in both efi mode and bios mode[2]. And because
they use the very same bls configuration the bios/efi configs can't go
out of sync on kernel updates. Oh, and systemd-nspawn can boot the
image too.

cheers,
Gerd

PS: The idea still works if you add a separate /boot partition, even
though I can't see a compelling reason to do so if you have a fresh
hard drive and can partition it as you like. And you have to use
grub2-efi instead of systemd-boot then.

[1] Dual boot installs obviously have to deal with whatever
mess they find on the harddrive ...
[2] Well, needs some manual editing due to some extra spaces.
https://bugzilla.redhat.com//show_bug.cgi?id=1588184
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QTL4WZ4ZDL7AOVTAHSIFRDLE57BGI5YJ/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-20 Thread Chris Murphy

On Wed, Jun 20, 2018 at 8:46 AM, Lennart Poettering
 wrote:
> On Do, 14.06.18 12:06, Jan Kurik (jku...@redhat.com) wrote:
>
>> The [https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/
>> Boot Loader Specification (BLS)] defines a scheme and file format to
>
> So, it appears the suggested implementation of this uses
> /EFI/fedora/loader/entries/ instead of the mandated /loader/entries/
> directory to place the drop-ins.
>
> What's the rationale for that? The idea of the boot loader spec is
> that multiple OSes or OS versions on the same medium won't fight for
> the boot loader and instead can drop-in their own boot entries easily
> in a non-conflicting way and make them available in the same boot menu
> that way. The strict idea is that *sharing* a drop-in dir is a good
> thing, and that exclusive ownership of the boot loader is a major
> problem.
>
> By using a fedora specific directory for the drop-ins you defeat the
> whole idea of the boot loader spec, as then suddenly the boot loaders
> will conflict again, because they can't share the directory anymore.
>
> What's going on? What is this? Why is this called "Boot loader spec"
> if it implements an entirely different logic, and misses the entire
> point of the boot loader spec?
>
> Quite frankly, I am really surprised by this and this makes me wonder
> what the whole point of this feature is at all, and very sure we
> shouldn't have it like this.

I agree.

But I also think there is some incremental risk of namespace collision
with /loader/entries as mentioned in Matthew Garrett's derivative of
the spec, if it's not located in /EFI/fedora/

Could $BOOT/org/freedesktop/bls/entries instead be
$BOOT/org.freedesktop.bls/entries ?


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ZC22JUNN4JOWILG7A64XAC5SQEAXMVRC/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-20 Thread Chris Murphy

On Wed, Jun 20, 2018 at 8:35 AM, Lennart Poettering
 wrote:
> On Di, 19.06.18 11:17, Chris Murphy (li...@colorremedies.com) wrote:
>
>> > Today, systemd has this generator that will automatically find the ESP
>> > for you and mount it to /efi or /boot. The idea behind that is that
>> > installers can choose whether they want to merge $BOOT and the ESP or
>> > not:
>> >
>> > 1. If they are merged, then the ESP (and thus also $BOOT) is mounted
>> >to /boot, automatically by the generator. No other preparation is
>> >needed, and /efi does not exist. (Distros could even make /efi a
>> >symlink → /boot, but I personally wouldn't bother).
>> >
>> > 2. If they are not merged, then the ESP is mounted to /efi,
>> >automatically by the generator. And /boot would be mounted as $BOOT
>> >via an /etc/fstab entry added by the installer.
>> >
>> > And as mentioned, I'd generally recommend everybody to go for option
>> > #1 because it is a lot simpler, and EFI has trivial access to all
>> > kernels and such.
>>
>> Except, it's not simple for installers to migrate to a new bigger ESP
>> in the dual boot case. And having different layouts for UEFI and BIOS
>> and whether there's dual boot or single boot, isn't simpler.
>
> Again, if you don't want to resize the ESP, then go for option #2
> above. But if the ESP is usable, then go for option #1.

In 100% of the cases where the ESP already exists, it is too small to
share. No one. Not Apple, not Microsoft, not Windows OEMs, not Fedora,
not any distro, creates an ESP bigger than 550MB. Typical is 99MB for
the Microsoft installer (I have a laptop partitioned by Microsoft's
install, not an OEM installer, and it's 99MB), and 128MB for Apple,
and 200MB for Linux distros. None of these are big enough to share.

And the ESP partition is wedged in, again in 100% of cases. It can't
be resized in place.

Therefore, Option #2 will be extremely common. What percent of Fedora
users dual boot? I have no empirical data. I'd guess 1/2.

And in my opinion, it's not simple to say: OK if you have this size
ESP to start, you get this layout, and if it's bigger you get this
other layout, and if it's BIOS you have this 3rd layout. And now we
have to document all of this, and train the installer, and openqa, and
all the people who help others out with their problems on #fedora and
Ask Fedora and users@. I understand it, and I think it's a clusterfuck
of complication. I can only imagine the end user confusion, people who
actually just want to get shit done.

It is possible to just ignore the ESP by treating it pretty much the
same as the MBR gap and BIOS Boot. And have one layout for all three
cases. That's simple.

>
>> If $BOOT is defined as where non-static bootloader config + kernel +
>> initramfs goes, and if shared $BOOT is a good thing for Linux distros,
>> then the $BOOT to always create is type 0xEA / bc13c2ff... and not
>> conflate it with the location for the bootloader binaries: the ESP on
>> UEFI, and either MBR gap or BIOSBoot on BIOS.
>
> Well, I am pretty sure legacy-free systems should not be bothered by
> having two partitions for that. That just complicates stuff.

No, it really doesn't. The *standard* Fedora installation today
already has two partitions for that, ESP and /boot.

You have to decide which is more important. Broad adoption, which will
require equal doses of compromise and simplicity. Or narrow adoption.
The BLS, as it is today, is barely even narrowly adopted, let alone
broadly adopted. And in my opinion it's because it's neither simple,
nor does it compromise.

And as Fedora is right now looking to implement BLS, what did they
actually do? Adopt the BLS file format and drop in concept, and
abandon the other 90% of the spec by punting.

I'll tell you what. Maybe consider a general purpose layout and a
simplified layout. The typical layout represents a compromise no
matter the firmware, and no matter what OS is already present - your
option 2. This would be used for workstations, and any case where dual
or multiboot is expected. And for things like Fedora VM images, IoT,
possibly server, possibly ARM - where the sharing aspect of $BOOT is
not expected or a consideration, go with the simplified layout - your
option 1.

*shrug*

>I mean,
> adding some minimal kludges to support Windows-cross-boot is fine, but
> adjusting everything with that case in the center of everything is
> quite wrong.

It isn't just Windows. It's macOS. It's all other Linux distros.

> Also note that we put together the boot loader spec with systemd-boot
> as our implementation of it, as a reference implementation if you so
> will. systemd-boot is a very simple, straight-forward boot loader,
> that adds a few things missing in UEFI itself, and doesn't contain
> code for parsing partition tables or file systems, for searching for
> devices and so on, like grub does. It implements the spec, but
> explicitly doesn't support splitting $BOOT and the ESP. I am pretty
> sure we as

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-20 Thread Lennart Poettering

On Do, 14.06.18 12:06, Jan Kurik (jku...@redhat.com) wrote:

> The [https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/
> Boot Loader Specification (BLS)] defines a scheme and file format to

So, it appears the suggested implementation of this uses
/EFI/fedora/loader/entries/ instead of the mandated /loader/entries/
directory to place the drop-ins.

What's the rationale for that? The idea of the boot loader spec is
that multiple OSes or OS versions on the same medium won't fight for
the boot loader and instead can drop-in their own boot entries easily
in a non-conflicting way and make them available in the same boot menu
that way. The strict idea is that *sharing* a drop-in dir is a good
thing, and that exclusive ownership of the boot loader is a major
problem.

By using a fedora specific directory for the drop-ins you defeat the
whole idea of the boot loader spec, as then suddenly the boot loaders
will conflict again, because they can't share the directory anymore.

What's going on? What is this? Why is this called "Boot loader spec"
if it implements an entirely different logic, and misses the entire
point of the boot loader spec?

Quite frankly, I am really surprised by this and this makes me wonder
what the whole point of this feature is at all, and very sure we
shouldn't have it like this.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/OTJ3WNYXTFOOCMZ6V7PRKYICVIYDZWS2/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-20 Thread Lennart Poettering

On Di, 19.06.18 11:17, Chris Murphy (li...@colorremedies.com) wrote:

> > Today, systemd has this generator that will automatically find the ESP
> > for you and mount it to /efi or /boot. The idea behind that is that
> > installers can choose whether they want to merge $BOOT and the ESP or
> > not:
> >
> > 1. If they are merged, then the ESP (and thus also $BOOT) is mounted
> >to /boot, automatically by the generator. No other preparation is
> >needed, and /efi does not exist. (Distros could even make /efi a
> >symlink → /boot, but I personally wouldn't bother).
> >
> > 2. If they are not merged, then the ESP is mounted to /efi,
> >automatically by the generator. And /boot would be mounted as $BOOT
> >via an /etc/fstab entry added by the installer.
> >
> > And as mentioned, I'd generally recommend everybody to go for option
> > #1 because it is a lot simpler, and EFI has trivial access to all
> > kernels and such.
> 
> Except, it's not simple for installers to migrate to a new bigger ESP
> in the dual boot case. And having different layouts for UEFI and BIOS
> and whether there's dual boot or single boot, isn't simpler.

Again, if you don't want to resize the ESP, then go for option #2
above. But if the ESP is usable, then go for option #1.

> If $BOOT is defined as where non-static bootloader config + kernel +
> initramfs goes, and if shared $BOOT is a good thing for Linux distros,
> then the $BOOT to always create is type 0xEA / bc13c2ff... and not
> conflate it with the location for the bootloader binaries: the ESP on
> UEFI, and either MBR gap or BIOSBoot on BIOS.

Well, I am pretty sure legacy-free systems should not be bothered by
having two partitions for that. That just complicates stuff. I mean,
adding some minimal kludges to support Windows-cross-boot is fine, but
adjusting everything with that case in the center of everything is
quite wrong. Note that ESP and $BOOT have the same semantics,
life-cycles and requirements, hence they should really be the same if
possible, and only be split if they can't.

Also note that we put together the boot loader spec with systemd-boot
as our implementation of it, as a reference implementation if you so
will. systemd-boot is a very simple, straight-forward boot loader,
that adds a few things missing in UEFI itself, and doesn't contain
code for parsing partition tables or file systems, for searching for
devices and so on, like grub does. It implements the spec, but
explicitly doesn't support splitting $BOOT and the ESP. I am pretty
sure we as the spec authors should keep our implementation and the
spec aligned.

That said, it's of course up to Fedora to implement the spec in
Fedora. If it always wants to split the two partitions, by all means,
go for it, but I think it's needless complication except if you
actually dual boot with Windows.

> Windows, macOS, the various distros - they all have substantial
> differences in how they boot. But the one commonality I most
> consistently see? The bootloader teaches the pre-boot environment,
> right off the bat, how to read a real file system, and from that real
> file system the kernel and initramfs are loaded.

MacOS has native apple file system read support in their firmware,
they rely on the firmware to read the stuff they need directly from
the final disk. We don't have that luxury.

The good thing about using VFAT for $BOOT is that it is the common
ground pretty much everything involved in booting groks, if they grok
a file system at all. UEFI knows it, and so does the Raspberry Pi boot
protocol. The Linux initrd knows it and so does the Linux host OS,
Windows knows it. MacOS knows it. Grub knows it.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/NBA5TJT5PKLLIJJYVJSMKWAB2P4D4SZO/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-19 Thread Chris Murphy

On Tue, Jun 19, 2018 at 3:40 AM, Lennart Poettering
 wrote:

> The boot loader spec suggests that $BOOT and the ESP are the same
> thing, and I am very sure this is the best and simplest approach for
> all images that have no explicit reason to depart from that. However,
> the spec does not actually require $BOOT to be the same as the
> ESP. The freedom to make $BOOT != ESP was added as a compromise,
> because some folks were adamant that resizing the ESP on multi-boot
> systems should not be done (i personally don't think it's as big a
> prob as people claim though...).

It's effectively not possible because in every layout, from Windows to
all distros, it's wedged in by other file systems. You literally can't
resize it.

All you could do is create a new bigger one, copy over the contents of
old to new, and then wipefs the old one, and change the partition type
GUID or remove the partition entry.

I seriously doubt the installer folks want to take responsibility for
such a thing, and update the various NVRAM boot entries accordingly.

>
> Today, systemd has this generator that will automatically find the ESP
> for you and mount it to /efi or /boot. The idea behind that is that
> installers can choose whether they want to merge $BOOT and the ESP or
> not:
>
> 1. If they are merged, then the ESP (and thus also $BOOT) is mounted
>to /boot, automatically by the generator. No other preparation is
>needed, and /efi does not exist. (Distros could even make /efi a
>symlink → /boot, but I personally wouldn't bother).
>
> 2. If they are not merged, then the ESP is mounted to /efi,
>automatically by the generator. And /boot would be mounted as $BOOT
>via an /etc/fstab entry added by the installer.
>
> And as mentioned, I'd generally recommend everybody to go for option
> #1 because it is a lot simpler, and EFI has trivial access to all
> kernels and such.

Except, it's not simple for installers to migrate to a new bigger ESP
in the dual boot case. And having different layouts for UEFI and BIOS
and whether there's dual boot or single boot, isn't simpler.

If $BOOT is defined as where non-static bootloader config + kernel +
initramfs goes, and if shared $BOOT is a good thing for Linux distros,
then the $BOOT to always create is type 0xEA / bc13c2ff... and not
conflate it with the location for the bootloader binaries: the ESP on
UEFI, and either MBR gap or BIOSBoot on BIOS.

And the volume format for 0xEA / bc13c2ff... I don't know that it's
possible to get consensus. But set that aside, it means the BLS file
format needs a way to reference files on another volume. Assuming all
paths are local doesn't seem workable except in the most narrow case,
and always narrow casing this is what prevents it from being adopted.

Windows, macOS, the various distros - they all have substantial
differences in how they boot. But the one commonality I most
consistently see? The bootloader teaches the pre-boot environment,
right off the bat, how to read a real file system, and from that real
file system the kernel and initramfs are loaded.

>The boot loader can start the EFI shell and its
> trivial to explore the contents and everything and manually boot any
> kernel you like. This approach also means that no /etc/fstab entry is
> needed, and thus things are a lot more self-sufficient and robust.
>
> It would be my assumption that all OS images generated in full by some
> image builder would go for option #1, and option #2 would only be used
> when a traditional installer such as anaconda is used that is supposed
> to add a Fedora installation to a system that is already set up
> otherwise, i.e. already has an EFI partition in place that is
> considered too small. i.e. option #2 is the multi-boot-with-windows
> usecase, while option #1 is for pretty much everything else.

Flowchart that paragraph, it's a frigging maze. This is a Choose Your
Own Adventure spec. This paragraph itself demonstrates the
inconsistency, depending on what firmware you have, and what layout
existed before hand. And that tells me there isn't enough abstraction
that things are that variable and messy.

I don't like the idea of asking users  helping users, to have to be so
familiar with such esoteric things. It makes support, repair,
maintenance, upgrades, testing, documentation harder. There are too
many exceptions.

Just give up. The ESP belongs to the OSLoaders and a static
configuration if needed, so that we can figure out where to jump to
immediately to get the BLS snippets, the kernel, and initramfs. And
make that location consistent no matter the firmware, no matter the
prior layout.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-19 Thread Chris Murphy

On Mon, Jun 18, 2018 at 5:37 PM, Andrew Lutomirski  wrote:
>> On Jun 18, 2018, at 3:54 PM, Chris Murphy  wrote:

>> Getting journal support in the bootloader isn't going to happen. I've
>> already talked to the various fs upstreams about it.
>>
>
> Why are you talking to the fs upstreams?  The problem is a bug in
> GRUB, full stop.

OK so you're going to blame uboot and syslinux and others for not
supporting JBD2 and journal replay as well? I disagree that its worth
the effort.

> All this freeze crap that Fedora does is just
> papering over the bug.

Yeah I don't like any of that either, and in retrospect I should not
have bought off on the idea. But that's getting off topic.

>IMO the right thing to do is to get *GRUB*
> upstream to have a fully functional implementation of *one*
> widely-supported fs.  Hmm, GRUB supports F2FS, and F2FS is
> log-structured, right?  So I don't see how GRUB could fail to read a
> dirty filesystem correctly even if it wanted to.

Journaled file systems have the journal bolted on. It's a completely
different beast. To read any structure, there must be code. There's
simply no code to read the XFS or ext3/4 log. It appears there was an
attempt to get GRUB (and I think it's GRUB legacy) to support JBD2 but
it was abandoned. I don't know the history.

> Or someone could design a very simple, highly reliable, filesystem
> designed to make it easy to do atomic-enough updates and to read
> reliably.  Think VFAT-like but with a full atomic swap of all FS
> metadata.  Or a dead-simple log-structured FS.  /me ducks.
>
> Seriously, though, F2FS might be a fantastic choice for this purpose.

UDF might be doable with multisession acting to ensure atomic
operations, but that seems like a lot of work. If UEFI had gone UDF
instead of FAT, it'd be a different story.

F2FS might be sane. I rather like the idea of Fedora IoT leveraging F2FS.

>> So add that to the list of packages that need an ESP syncing daemon if
>> they don't want to be responsible for dynamically mounting and
>> umounting the ESP.

Fair enough.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/3ZYMPEURCBOEVP6NJUYQGYCDKQCF4DXD/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Di, 19.06.18 11:14, Daniel P. Berrangé (berra...@redhat.com) wrote:

> On Tue, Jun 19, 2018 at 11:48:39AM +0200, Lennart Poettering wrote:
> > On Mo, 18.06.18 16:54, R P Herrold (herr...@owlriver.com) wrote:
> > 
> > > On Mon, 18 Jun 2018, Lennart Poettering wrote:
> > > 
> > > > On Do, 14.06.18 14:20, Chris Murphy (li...@colorremedies.com) wrote:
> > > > 
> > > > > The cited BLS spec is the original one, [1]
> > > 
> > > ... later: L.P.:
> > > > [reduce] the size of the spec if possible, and drop as many 
> > > > bits of it as we can, i.e. the stuff noone implements 
> > > > anyway.
> > > > 
> > > > > The cited BLS spec requires $BOOT be VFAT, are we doing that?
> > > 
> > > Will cgroup and SElinux protections work in VFAT ?
> > 
> > cgroups and file systems have little to do with each other.
> > 
> > VFAT won't store selinux labels of course, but you can assign a fixed
> > label to all files of a vfat file system when mounting it. It's what
> > Fedora does when dealing with the ESP already. So regarding selinux
> > it's not whether to do selinux or not to do it, but whether is really
> > necessary to label the initrd file and the kernel differently, or
> > whether it's ok to give all files in /boot the same label. I am pretty
> > sure that's actually what already happens anyway, even if you have
> > ext4, but then again i am not running grub nor ext4, so I don't really know.
> 
> Mostly everything is labelled with boot_t, but System.map files get
> given system_map_t, and there's a few filesystem house keeping labels
> too. You can view it with semanage:
> 
> # semanage fcontext -l | grep '^/boot'
> /boot  all files  
> system_u:object_r:boot_t:s0 
> /boot/.*   all files  
> system_u:object_r:boot_t:s0 
> /boot/System\.map(-.*)?regular file   
> system_u:object_r:system_map_t:s0 
> /boot/\.journalall files  <>
> /boot/a?quota\.(user|group)regular file   
> system_u:object_r:quota_db_t:s0 
> /boot/efi(/.*)?/System\.map(-.*)?  regular file   
> system_u:object_r:system_map_t:s0 
> /boot/lost\+found  directory  
> system_u:object_r:lost_found_t:s0 
> /boot/lost\+found/.*   all files  <>

Since the quota and lost+found stuff doesn't apply to vfat there are
only two labels left: boot_t and system_map_t. The question is whether
there's really benefit in separating these two... 

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/UAD6KQPIMEEQJKE2CTKGIWBOZMCYX75U/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-19 Thread Daniel P . Berrangé

On Tue, Jun 19, 2018 at 11:48:39AM +0200, Lennart Poettering wrote:
> On Mo, 18.06.18 16:54, R P Herrold (herr...@owlriver.com) wrote:
> 
> > On Mon, 18 Jun 2018, Lennart Poettering wrote:
> > 
> > > On Do, 14.06.18 14:20, Chris Murphy (li...@colorremedies.com) wrote:
> > > 
> > > > The cited BLS spec is the original one, [1]
> > 
> > ... later: L.P.:
> > > [reduce] the size of the spec if possible, and drop as many 
> > > bits of it as we can, i.e. the stuff noone implements 
> > > anyway.
> > > 
> > > > The cited BLS spec requires $BOOT be VFAT, are we doing that?
> > 
> > Will cgroup and SElinux protections work in VFAT ?
> 
> cgroups and file systems have little to do with each other.
> 
> VFAT won't store selinux labels of course, but you can assign a fixed
> label to all files of a vfat file system when mounting it. It's what
> Fedora does when dealing with the ESP already. So regarding selinux
> it's not whether to do selinux or not to do it, but whether is really
> necessary to label the initrd file and the kernel differently, or
> whether it's ok to give all files in /boot the same label. I am pretty
> sure that's actually what already happens anyway, even if you have
> ext4, but then again i am not running grub nor ext4, so I don't really know.

Mostly everything is labelled with boot_t, but System.map files get
given system_map_t, and there's a few filesystem house keeping labels
too. You can view it with semanage:

# semanage fcontext -l | grep '^/boot'
/boot  all files  
system_u:object_r:boot_t:s0 
/boot/.*   all files  
system_u:object_r:boot_t:s0 
/boot/System\.map(-.*)?regular file   
system_u:object_r:system_map_t:s0 
/boot/\.journalall files  <>
/boot/a?quota\.(user|group)regular file   
system_u:object_r:quota_db_t:s0 
/boot/efi(/.*)?/System\.map(-.*)?  regular file   
system_u:object_r:system_map_t:s0 
/boot/lost\+found  directory  
system_u:object_r:lost_found_t:s0 
/boot/lost\+found/.*   all files  <>


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/6B7P3Y7YCCKDODAHXWCJTVQX2SRQFO3Q/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 18.06.18 16:54, R P Herrold (herr...@owlriver.com) wrote:

> On Mon, 18 Jun 2018, Lennart Poettering wrote:
> 
> > On Do, 14.06.18 14:20, Chris Murphy (li...@colorremedies.com) wrote:
> > 
> > > The cited BLS spec is the original one, [1]
> 
> ... later: L.P.:
> > [reduce] the size of the spec if possible, and drop as many 
> > bits of it as we can, i.e. the stuff noone implements 
> > anyway.
> > 
> > > The cited BLS spec requires $BOOT be VFAT, are we doing that?
> 
> Will cgroup and SElinux protections work in VFAT ?

cgroups and file systems have little to do with each other.

VFAT won't store selinux labels of course, but you can assign a fixed
label to all files of a vfat file system when mounting it. It's what
Fedora does when dealing with the ESP already. So regarding selinux
it's not whether to do selinux or not to do it, but whether is really
necessary to label the initrd file and the kernel differently, or
whether it's ok to give all files in /boot the same label. I am pretty
sure that's actually what already happens anyway, even if you have
ext4, but then again i am not running grub nor ext4, so I don't really know.

> > Why would we? I mean the idea is that $BOOT can be shared among
> > multiple OSes installed. Which means one really should settle on a
> 
> I see a lot of need in [1] for re-partitioning and optionally 
> adding a /boot partition where none was specified, to make 
> this work
> 
> The move toward containers includes getting away from more 
> than a single partition (and so, a separate /boot partition, 
> as mostly irrelavant).  Getting rid of a separate /boot 
> partition is a win, as it  removes the need for a separate 
> mountpoint in /etc/fstab for a '/boot/'. partition, and all 
> the gyrations as to partitioning in [1]

Well, my personal opinion is that the ESP is where kernels should be
placed if at all possible, in order to simplify things. You need the
ESP anyway, there's no way around it, hence if you can just unify the
pre-root stuff there, and then only have the ESP and your root dir as
necessary partitions.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/H7GJBIFU56PESKBRNDDXZO5WHFV3JOK3/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 18.06.18 16:50, Ondřej Lysoněk (olyso...@redhat.com) wrote:

> Hi,
> 
> On 18.6.2018 15:27, Lennart Poettering wrote:
> > On Do, 14.06.18 14:20, Chris Murphy (li...@colorremedies.com) wrote:
> > 
> >> The cited BLS spec is the original one, not the more thoroughly
> >> discussed and thought through variant by Matthew Garrett [1] some
> >> years ago.
> > 
> > Quite frankly, as one of the authors of the original BLS spec, I can'd
> > say Matthew's version was much discussed with me...
> > 
> > I mean, I am open to extending the spec, but we should do this bit by
> > bit.
> > 
> > Zbigniew suggested to move the spec into docbook or markdown format,
> > and then accept changes via usual github PRs. If there's interest
> > still in extending the spec with some of Matthew's ideas we can
> > certainly look into that, but in general I'd actually prefer to reduce
> > the size of the spec if possible, and drop as many bits of it as we
> > can, i.e. the stuff noone implements anyway.
> 
> It would be great if we could extend the spec with optional support for
> multiple initrd images (the Tuned daemon depends on that). Fedora's
> GRUB2 already supports multiple initrd images (it allows specifying
> multiple lines with the "initrd" key), but I'd like to make sure that
> whoever implements BLS in the future and decides to support multiple
> initrds will not choose a different syntax for it. Would you be open to
> extending the spec with that?

Sure, allowing multiple initrd keys in the snippets makes a ton of sense.

Lennart

-- 
Lennart Poettering, Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/KKC7KFBNLSZQLA3756ECZEFJPIC2UO6W/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mo, 18.06.18 10:30, Chris Murphy (li...@colorremedies.com) wrote:

> >> The cited BLS spec requires $BOOT be VFAT, are we doing that?
> >
> > Why would we? I mean the idea is that $BOOT can be shared among
> > multiple OSes installed. Which means one really should settle on a
> > format anyone can read. And VFAT certainly qualifies as that, most
> > other file systems do not.
> 
> Do you mean "why wouldn't we?" Flipping over everyone's /boot to
> become the ESP on VFAT is a substantial change so I'm asking the
> question, yet again. This was asked a long time ago with the original
> conversations on this list about BootLoaderSpec, and those questions
> and answers are addressed in Matthew Garrett's derivative of the
> spec.

The boot loader spec suggests that $BOOT and the ESP are the same
thing, and I am very sure this is the best and simplest approach for
all images that have no explicit reason to depart from that. However,
the spec does not actually require $BOOT to be the same as the
ESP. The freedom to make $BOOT != ESP was added as a compromise,
because some folks were adamant that resizing the ESP on multi-boot
systems should not be done (i personally don't think it's as big a
prob as people claim though...).

Today, systemd has this generator that will automatically find the ESP
for you and mount it to /efi or /boot. The idea behind that is that
installers can choose whether they want to merge $BOOT and the ESP or
not:

1. If they are merged, then the ESP (and thus also $BOOT) is mounted
   to /boot, automatically by the generator. No other preparation is
   needed, and /efi does not exist. (Distros could even make /efi a
   symlink → /boot, but I personally wouldn't bother).

2. If they are not merged, then the ESP is mounted to /efi,
   automatically by the generator. And /boot would be mounted as $BOOT
   via an /etc/fstab entry added by the installer.

And as mentioned, I'd generally recommend everybody to go for option
#1 because it is a lot simpler, and EFI has trivial access to all
kernels and such. The boot loader can start the EFI shell and its
trivial to explore the contents and everything and manually boot any
kernel you like. This approach also means that no /etc/fstab entry is
needed, and thus things are a lot more self-sufficient and robust.

It would be my assumption that all OS images generated in full by some
image builder would go for option #1, and option #2 would only be used
when a traditional installer such as anaconda is used that is supposed
to add a Fedora installation to a system that is already set up
otherwise, i.e. already has an EFI partition in place that is
considered too small. i.e. option #2 is the multi-boot-with-windows
usecase, while option #1 is for pretty much everything else.

> > 1) The chance that the ESP remains in a clean state is maximized, as
> >the file system is unmounted whenever possible, and only mounted
> >for a short time window around actual disk accesses. This is the
> >behaviour you really want for something as fragile as the ESP.
> >
> > 2) It's compatible with current behaviour, as the path is always
> >accessible under a fixed name, requiring no explicit mounting.
> >
> > 3) There's no need to configure any lines for the ESP in /etc/fstab
> >anymore. Instead the system will discover the ESP automatically and
> >make it available. This means the installer can be simpler, and
> >things are generally more robust as /efi (or /boot) will follow
> >what is in place, instead of require a separate layer of
> >configuration that has a good chance of getting out of sync.
> 
> I've got no complaints with this although as mentioned in the other
> thread "f29 bootloader changes / raid1 installs + efi" this generator
> lacks the intelligence needed to support multiple ESPs for any RAID
> use case, e.g. md or LVM or Btrfs RAID1.

Well, if you really want to cover this, then using the automount stuff
actually allows you to solve this much nicer than traditional setups:
write a service (possibly just a script around dd with some locking
might suffice) that is pulled in by the .mount unit that the
.automount unit is backed by, and is ordered before the .mount
unit. This then means its ExecStart= and ExecStop= line would run
before and after the mount is around. In ExecStop= you could then dd
the mounted file system onto the other copies. And we'd do though
automatically after each series of accesses.

> > I'd love to see Fedora adopt this generator. Primarily this would mean
> > some chnages to anaconda I guess. It would make things simpler and
> > more robust. That said, the generator only cares about the ESP, not
> > for cases where $BOOT is backed by any other partition.
> 
> Ok so you're saying if $BOOT is type 0xEA, or type
> bc13c2ff-59e6-4262-a352-b275fd6f7172, the generator will not automount
> it at /boot ?

systemd will do the discovery and automount unit generation only for
the ESP, and it's very defensive,

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread Kyle Marek

On 06/18/2018 07:37 PM, Andrew Lutomirski wrote:
>> On Jun 18, 2018, at 3:54 PM, Chris Murphy  wrote:
>>
>> On Mon, Jun 18, 2018 at 3:42 PM, Andrew Lutomirski  wrote:
>>> As an extra plus, upgrading a kernel doesn’t require mounting the ESP,
>>> which means that the bootloader installation can sync the ESP across
>>> multiple disks and it will remain synced.
>> Yeah. But *le sigh* we have fwupd which wants /boot/efi mounted
>>
>> I use 'umask=0077,shortname=winnt,nofail,noauto,x-systemd.automount'
>> in fstab for /boot/efi and yet every boot:
>>
>> Jun 17 14:07:06 f28h.local systemd[1]: boot-efi.automount: Got
>> automount request for /boot/efi, triggered by 2268 (fwupd)
>>
>> So add that to the list of packages that need an ESP syncing daemon if
>> they don't want to be responsible for dynamically mounting and
>> umounting the ESP.
> I disagree.  fwupd doesn't need any ESP syncing.  In the very worst
> case, fwupd sticks a capsule in the ESP from which we booted, then
> that disk dies, and we don't apply the capsule.  No harm done.
>
> I really think the correct approach here is to have an ESP on each
> potentially bootable disk.  Each ESP will independently contain the
> code to handle BootLoaderSpec entries on a *different* partition.  The
> only time we need to modify all the ESP copies is when we upgrade GRUB
> or upgrade GRUB's configuration.  We do *not* need to propagate
> capsules or any other similar objects.  And we should not even try to
> impose a requirement that the filesystems be bitwise identical.
> They're *copies*, not RAID.

I've been thinking about this, too, and I agree that this seems like the
best solution.

I think EFI is one of the places where Ubuntu/Debian seems to do better
than other distros. Their GRUB .EFI file has a script and relative path
hardcoded that reads a UUID from a file accompanying the .EFI file, and
reads /grub/grub.cfg from the block device with the UUID specified in
the accompanying file in the ESP. This lets them sign the .EFI file for
secure boot, everyone gets the same .EFI file, yet it can load a grub
config file from a user-defined partition (which may be md-RAIDed, which
eliminates the need for complex ESP-syncing logic beyond initial
installation of original EFI file and UUID file).

Not to mention that their GRUB doesn't require the efi suffixes on
commands like "linux" and "initrd" so the same config file can be used
by both BIOS GRUB and EFI GRUB for non-dual-boot entries.

Keeping the bulk kernels/initrds in their own partition should also
mitigate size issues with dual booting with other distros. If ESP size
is a concern for one distro, it will probably be a bigger concern for
multiple distros that want to store kernels and initrds in ESP (though,
it is also fair to say that the user can re-partition to make a bigger
ESP. They're not exactly one-size-fits-all, anyway).

The necessity/automation of running efibootmgr for new drives will need
to be documented if this method ends up being used to provide a
redundant ESP on md-RAIDed systems.

Again, the structure I am referring to would be:

 1. grub${arch}.efi reads its embedded config file
 2. embedded config file says to read grub.cfg from file from ESP (maybe
hardcoded to /EFI/fedora/grub.cfg if we want a copy of the efi file
at /EFI/BOOT/BOOT${arch}.EFI to also work)
 3. ESP's grub.cfg says to read /grub2/grub.cfg from UUID=X
 4. /grub2/grub.cfg on UUID=X is where kernel and initrd lives and are
configured.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QJKGENMUYT2VN62G3JSSL4EAZEYJIQUL/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mon, Jun 18, 2018 at 3:54 PM, Tom Hughes  wrote:
> On 18/06/18 18:15, Peter Jones wrote:
>
>> That's true - though we actually shipped nearly all of the code to
>> implement this stuff f28, minus some parts of the upgrade story and the
>> anaconda bits to enable it by default.  You can go run
>> grub2-switch-to-blscfg today, and it will work.  I hope :)
>
>
> Unless you have /boot on your root partition like this machine
> seems to have for some reason... Then it breaks because the loader
> fragments use /vmlinuz... rather than /boot/vmlinuz etc.

Confirmed. When I check GRUB environment variables, the root variable
lacks a path. It's just root=hd0,gpt9.

And the bls snippet

linuxefi $root/vmlinuz-4.17.0-1.fc29.x86_64

But the actual path to the kernel is
/root28/boot/vmlinuz-4.17.0-1.fc29.x86_64 so the boot fails.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/2DR6RPWTOPYG57FGTIUFAEY5S6E4H4KL/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread Andrew Lutomirski

> On Jun 18, 2018, at 3:54 PM, Chris Murphy  wrote:
>
> On Mon, Jun 18, 2018 at 3:42 PM, Andrew Lutomirski  wrote:
>>> On Jun 18, 2018, at 10:02 AM, Javier Martinez Canillas 
>>>  wrote:
>
>>> Yes for EFI systems but no otherwise. On EFI the BLS snippets are in
>>> /boot/efi/EFI/fedora/loader/entries and on non-EFI systems are in
>>> /boot/loader/entries.
>>>
>>
>> I think this is the wrong approach. I see no valid reason that the
>> paths should be different on EFI.
>
> I don't like it either but I'm trying to keep an open mind. What I
> recall from the conversations years ago with the mgj59 variant, it's a
> lot easier to poke holes in any attempt to standardize bootloading,
> than to standardize bootloading. But if no one is willing to give any
> ground anywhere, it's way too easy to stop progress.
>
> The proposed change doesn't really fix any of the user facing
> problems, it just gets us away from depending on grubby and
> grub-mkconfig (we never really depended on grub-mkconfig, the
> installer uses it as a one shot). So in the most narrow sense, the
> change succeeds at doing the only thing it's intending to do. But
> yeah, I lament that we're not being more aggressive while we have the
> chance to fix past oversights.
>
>

And that's fine, as long as it's not done in a way that makes it
harder to improve in the future.

>>>
 If there's no room on the EFI System partition for all of this, will
 we following bullets 2 and 5 of the BLS spec under "The installer
>>>
>>> No, $BOOT is always the ESP where GRUB 2 is installed.
>>
>> I’m going to go out on a limb and make a stronger objection than
>> Chris’: I think that $BOOT SHOULD NOT be the ESP. The ESP is
>> problematic for any number of reasons. It’s usually vfat, so it’s
>> fragile. It does not support RAID safely. And it’s often small.
>
> Well as it turns out all the file systems are fragile as far as the
> bootloader is concerned :-P We've got bugs where the bootloader can't
> read needed files, because part of the commit is still in the journal
> rather than fully flushed out to file system metadata. So no matter
> what you can break a set up...
>

...

>
> Getting journal support in the bootloader isn't going to happen. I've
> already talked to the various fs upstreams about it.
>

Why are you talking to the fs upstreams?  The problem is a bug in
GRUB, full stop. All this freeze crap that Fedora does is just
papering over the bug.  IMO the right thing to do is to get *GRUB*
upstream to have a fully functional implementation of *one*
widely-supported fs.  Hmm, GRUB supports F2FS, and F2FS is
log-structured, right?  So I don't see how GRUB could fail to read a
dirty filesystem correctly even if it wanted to.

Or someone could design a very simple, highly reliable, filesystem
designed to make it easy to do atomic-enough updates and to read
reliably.  Think VFAT-like but with a full atomic swap of all FS
metadata.  Or a dead-simple log-structured FS.  /me ducks.

Seriously, though, F2FS might be a fantastic choice for this purpose.

>
>> As an extra plus, upgrading a kernel doesn’t require mounting the ESP,
>> which means that the bootloader installation can sync the ESP across
>> multiple disks and it will remain synced.
>
> Yeah. But *le sigh* we have fwupd which wants /boot/efi mounted
>
> I use 'umask=0077,shortname=winnt,nofail,noauto,x-systemd.automount'
> in fstab for /boot/efi and yet every boot:
>
> Jun 17 14:07:06 f28h.local systemd[1]: boot-efi.automount: Got
> automount request for /boot/efi, triggered by 2268 (fwupd)
>
> So add that to the list of packages that need an ESP syncing daemon if
> they don't want to be responsible for dynamically mounting and
> umounting the ESP.

I disagree.  fwupd doesn't need any ESP syncing.  In the very worst
case, fwupd sticks a capsule in the ESP from which we booted, then
that disk dies, and we don't apply the capsule.  No harm done.

I really think the correct approach here is to have an ESP on each
potentially bootable disk.  Each ESP will independently contain the
code to handle BootLoaderSpec entries on a *different* partition.  The
only time we need to modify all the ESP copies is when we upgrade GRUB
or upgrade GRUB's configuration.  We do *not* need to propagate
capsules or any other similar objects.  And we should not even try to
impose a requirement that the filesystems be bitwise identical.
They're *copies*, not RAID.

>
>
>> All that being said, $BOOT should not use security context xattrs —
>> getting that to work right across distros is probably impossible.
>
>
> It's a good point. I'm not really sure  how to prevent conflicts if
> there is to be a truly shared $BOOT unless it is a file system that
> will reject xattrs.
>

Mount with noxattr?

--Andy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread Tom Hughes

On 18/06/18 23:46, Javier Martinez Canillas wrote:

On Mon, Jun 18, 2018 at 11:54 PM, Tom Hughes wrote:

On 18/06/18 18:15, Peter Jones wrote:

That's true - though we actually shipped nearly all of the code to
implement this stuff f28, minus some parts of the upgrade story and the
anaconda bits to enable it by default. You can go run
grub2-switch-to-blscfg today, and it will work. I hope :)

Unless you have /boot on your root partition like this machine
seems to have for some reason... Then it breaks because the loader
fragments use /vmlinuz... rather than /boot/vmlinuz etc.

Yes, /boot not being a mount point was an issue (and also /boot being
a btrfs subvolme) but it got fixed [0] a couple of weeks ago. Now the
20-grub.install kernel-install script uses grub2-mkrelpath to get the
relative path of the kernel and initramfs images, which is the same
that's done by the /etc/grub.d/10_linux script. It's just that the
grub2 update didn't made to F28 yet.

Does that extend to grub2-switch-to-blscfg as well? That was what
broke my boot.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ABOGNY2QEU57N6NWGQZ47BPN56JC3JYT/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mon, Jun 18, 2018 at 3:42 PM, Andrew Lutomirski  wrote:
>> On Jun 18, 2018, at 10:02 AM, Javier Martinez Canillas  
>> wrote:

>> Yes for EFI systems but no otherwise. On EFI the BLS snippets are in
>> /boot/efi/EFI/fedora/loader/entries and on non-EFI systems are in
>> /boot/loader/entries.
>>
>
> I think this is the wrong approach. I see no valid reason that the
> paths should be different on EFI.

I don't like it either but I'm trying to keep an open mind. What I
recall from the conversations years ago with the mgj59 variant, it's a
lot easier to poke holes in any attempt to standardize bootloading,
than to standardize bootloading. But if no one is willing to give any
ground anywhere, it's way too easy to stop progress.

The proposed change doesn't really fix any of the user facing
problems, it just gets us away from depending on grubby and
grub-mkconfig (we never really depended on grub-mkconfig, the
installer uses it as a one shot). So in the most narrow sense, the
change succeeds at doing the only thing it's intending to do. But
yeah, I lament that we're not being more aggressive while we have the
chance to fix past oversights.

>
>>
>>
>>> If there's no room on the EFI System partition for all of this, will
>>> we following bullets 2 and 5 of the BLS spec under "The installer
>>
>> No, $BOOT is always the ESP where GRUB 2 is installed.
>
> I’m going to go out on a limb and make a stronger objection than
> Chris’: I think that $BOOT SHOULD NOT be the ESP. The ESP is
> problematic for any number of reasons. It’s usually vfat, so it’s
> fragile. It does not support RAID safely. And it’s often small.

Well as it turns out all the file systems are fragile as far as the
bootloader is concerned :-P We've got bugs where the bootloader can't
read needed files, because part of the commit is still in the journal
rather than fully flushed out to file system metadata. So no matter
what you can break a set up...

> Most of this can be solved by putting $BOOT on a different partition.
> Stick it on mdadm 1.1 if you want RAID (*not* 1.0 or 0.9 due to
> corruption risks [0]), and maybe even use a journaling filesystem that
> the bootloader can *correctly* read. (That means the bootloader should
> be able to parse the journal.).  And make it however big you want.

Getting journal support in the bootloader isn't going to happen. I've
already talked to the various fs upstreams about it.

> As an extra plus, upgrading a kernel doesn’t require mounting the ESP,
> which means that the bootloader installation can sync the ESP across
> multiple disks and it will remain synced.

Yeah. But *le sigh* we have fwupd which wants /boot/efi mounted

I use 'umask=0077,shortname=winnt,nofail,noauto,x-systemd.automount'
in fstab for /boot/efi and yet every boot:

Jun 17 14:07:06 f28h.local systemd[1]: boot-efi.automount: Got
automount request for /boot/efi, triggered by 2268 (fwupd)

So add that to the list of packages that need an ESP syncing daemon if
they don't want to be responsible for dynamically mounting and
umounting the ESP.

> All that being said, $BOOT should not use security context xattrs —
> getting that to work right across distros is probably impossible.

It's a good point. I'm not really sure  how to prevent conflicts if
there is to be a truly shared $BOOT unless it is a file system that
will reject xattrs.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/L4K3FPKNIVFQSLKCRZJ4NGXFTITKZKWZ/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread Javier Martinez Canillas

On Mon, Jun 18, 2018 at 11:54 PM, Tom Hughes  wrote:
> On 18/06/18 18:15, Peter Jones wrote:
>
>> That's true - though we actually shipped nearly all of the code to
>> implement this stuff f28, minus some parts of the upgrade story and the
>> anaconda bits to enable it by default.  You can go run
>> grub2-switch-to-blscfg today, and it will work.  I hope :)
>
>
> Unless you have /boot on your root partition like this machine
> seems to have for some reason... Then it breaks because the loader
> fragments use /vmlinuz... rather than /boot/vmlinuz etc.
>

Yes, /boot not being a mount point was an issue (and also /boot being
a btrfs subvolme) but it got fixed [0] a couple of weeks ago. Now the
20-grub.install kernel-install script uses grub2-mkrelpath to get the
relative path of the kernel and initramfs images, which is the same
that's done by the /etc/grub.d/10_linux script. It's just that the
grub2 update didn't made to F28 yet.

[0]: 
https://src.fedoraproject.org/rpms/grub2/c/db7cf3a089075af0f4a8b955af508aea3893465a

> Tom
>

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/UGUSZIBOLV4XUZPKZ3ZTYZ2HJO36KPES/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread Tom Hughes


On 18/06/18 18:15, Peter Jones wrote:


That's true - though we actually shipped nearly all of the code to
implement this stuff f28, minus some parts of the upgrade story and the
anaconda bits to enable it by default.  You can go run
grub2-switch-to-blscfg today, and it will work.  I hope :)


Unless you have /boot on your root partition like this machine
seems to have for some reason... Then it breaks because the loader
fragments use /vmlinuz... rather than /boot/vmlinuz etc.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/O36DGKINVAYYLHQGDA4GBIZ5JJYS7KBP/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread Andrew Lutomirski

> On Jun 18, 2018, at 10:02 AM, Javier Martinez Canillas  
> wrote:
>
>> On Thu, Jun 14, 2018 at 10:20 PM, Chris Murphy  
>> wrote:
>> On Thu, Jun 14, 2018 at 12:51 PM, Adam Williamson
>>  wrote a monolithic config
>

>
>> The cited BLS spec requires $BOOT be VFAT, are we doing that?
>>
>
> Yes for EFI systems but no otherwise. On EFI the BLS snippets are in
> /boot/efi/EFI/fedora/loader/entries and on non-EFI systems are in
> /boot/loader/entries.
>

I think this is the wrong approach. I see no valid reason that the
paths should be different on EFI.

>
>
>> If there's no room on the EFI System partition for all of this, will
>> we following bullets 2 and 5 of the BLS spec under "The installer
>
> No, $BOOT is always the ESP where GRUB 2 is installed.

I’m going to go out on a limb and make a stronger objection than
Chris’: I think that $BOOT SHOULD NOT be the ESP. The ESP is
problematic for any number of reasons. It’s usually vfat, so it’s
fragile. It does not support RAID safely. And it’s often small.

Most of this can be solved by putting $BOOT on a different partition.
Stick it on mdadm 1.1 if you want RAID (*not* 1.0 or 0.9 due to
corruption risks [0]), and maybe even use a journaling filesystem that
the bootloader can *correctly* read. (That means the bootloader should
be able to parse the journal.).  And make it however big you want.

As an extra plus, upgrading a kernel doesn’t require mounting the ESP,
which means that the bootloader installation can sync the ESP across
multiple disks and it will remain synced.

All that being said, $BOOT should not use security context xattrs —
getting that to work right across distros is probably impossible.

[0] I use mdadm a lot, and I never use 0.9 or 1.0. It’s too fragile.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/JURU4F7L5CLTXWINANC2WVTBTRMTE76T/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread R P Herrold

On Mon, 18 Jun 2018, Lennart Poettering wrote:

> On Do, 14.06.18 14:20, Chris Murphy (li...@colorremedies.com) wrote:
> 
> > The cited BLS spec is the original one, [1]

... later: L.P.:
> [reduce] the size of the spec if possible, and drop as many 
> bits of it as we can, i.e. the stuff noone implements 
> anyway.
> 
> > The cited BLS spec requires $BOOT be VFAT, are we doing that?

Will cgroup and SElinux protections work in VFAT ?

 
> Why would we? I mean the idea is that $BOOT can be shared among
> multiple OSes installed. Which means one really should settle on a

I see a lot of need in [1] for re-partitioning and optionally 
adding a /boot partition where none was specified, to make 
this work

The move toward containers includes getting away from more 
than a single partition (and so, a separate /boot partition, 
as mostly irrelavant).  Getting rid of a separate /boot 
partition is a win, as it  removes the need for a separate 
mountpoint in /etc/fstab for a '/boot/'. partition, and all 
the gyrations as to partitioning in [1]

N.B.: This is a 'Good Thing' as one does not get 'out of sync 
with each other between 'root of filesystem', and /boot/ when 
they are all in a single filesystem


> So, in systemd we ship a generator that automatically establishes
> automount points for the ESP. It will preferably use /efi as mount
> point if it exists and is empty. If it doesn't exist or isn't
> empty it will use /boot — if that exists or is empty. If neither exist
> or are empty it won't do anything

I think this last is a negative: 
>  ... it won't do anything

and I submit that the proper course, when no /boot partition 
is seen, is to properly mount the root of filesystem, and do 
a:
 mkdir /boot
and then continue on


To wrestle with all the failure modes, I see a lot of 
fall-through logic, and a lot more complexity, including 
re-partitioning by the boot loader on a device it may not have 
RW rights at the partition table level -- yikes, as to an 
added point of faiure -- outlined in the initial proposa [1]l, 
but not implied in Matthew Garrett's [2]

But for the fact that that kernel updates do not 'just apply' 
with the current grubby / dracut regime, the approach of a 
/boot/ directory (but not separate partition) works in 
production just fine and has for over a decade [3] 

I suspect that moving to such as an option, and adding a 
systemd umount RW / remount RO of 'root of filesystem' on the 
way to a shutdown, would ameriorate if not totally remedy [4] 
and [5] as well.  Just the remount RO by systemd will cause 
the needed 'sync' actions on the way down would solve: [6] and 
its Fedora twin: [7]

If a '/boot/' partition is absent, simply creating a /boot/ 
directory at the root of the file system does away with the 
need for many of the gyrations of [1].

-- Russ herrold

[1] https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/

[2] https://www.freedesktop.org/wiki/MatthewGarrett/BootLoaderSpec/

[3] https://bugzilla.redhat.com/show_bug.cgi?id=1498169

[4] https://bugzilla.redhat.com/show_bug.cgi?id=1533620

[5] https://bugzilla.redhat.com/show_bug.cgi?id=1569970

[6] https://bugzilla.redhat.com/show_bug.cgi?id=1464611

[7] https://bugzilla.redhat.com/show_bug.cgi?id=1466036
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QGECPUQBDZFIWDTYZRBJGIBHOVAIHALZ/

Re: F29 System Wide Change: Make BootLoaderSpec the default

2018-06-18 Thread Peter Jones

On Mon, Jun 18, 2018 at 12:14:31PM -0600, Chris Murphy wrote:
> Thanks for the reply.
> 
> I think the proposal title is misleading. The BLS file format is,
> depending on one's point of view, 5% of the spec. A bulk of the
> proposal isn't going to follow the spec at all. And even with regards
> to the file format, you're not following the spec which mandates all
> paths relative to $BOOT, which clearly isn't going to be the case. And
> that makes the BLS file format you're implementing, more close to GRUB
> legacy, and the Matthew Garrett BLS format derivative, than the
> original BLS spec format.
> 
> I think the feature proposal should be rename: 'Make using
> BootLoaderSpec style file format the default'

I've updated the page with a bunch of changes to try and help with this
confusion, including changing the top heading and adding a section about
the differences between the various specs and what's currently
implemented and a section about what our boot entry config files look
like, as well as notes about $kernelopts and $grub_users.

> The proposal doesn't follow the BLS spec in some of the most critical
> ways necessary to get it adopted by upstreams and other distros.
> 
> My summary of the change for most users (x86_64)
> - /etc/grub.d/10_linux will no longer contain Fedora entries, each
> menu entry will be a BLS format drop in script instead

Er, will no longer *generate* them, but yes.

> - grub.cfg still is responsible for multibooting Windows and other
> distros via /etc/grub.d/30_os-prober

Right, though this continues to become less and less relevant with
things like BitLocker storing keys in TPM.

> - users will no longer modify /etc/default/grub, they will duplicate
> (?) and modify BLS scripts directly if they need to make permanent
> changes to menu entries

As you mention in your next post, if you want to change the command line
globally, we're getting it from grubenv, which mkconfig is setting from
/etc/default/grub's value.

> - users will no longer need to run grub2-mkconfig, unless the grub.cfg
> is accidentally missing or malformed

Right.

> - users on BIOS systems who install another distro after Fedora, will
> need to inform the distro's installer to not overwrite the Fedora
> bootloader, or the user will need to reinstall the Fedora bootloader;
> until such time as distro bootloaders support the BLS format

Or they'll need to chainload to it, from a different disk for example.

-- 
  Peter
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/U6FQJYUYH7BCZ5TQC6RWFNANMR2UFBHB/

Re: F29 System Wide Change: Make BootLoaderSpec the default

On Mon, Jun 18, 2018 at 12:14 PM, Chris Murphy  wrote:

> My summary of the change for most users (x86_64)

> - users will no longer modify /etc/default/grub, they will duplicate
> (?) and modify BLS scripts directly if they need to make permanent
> changes to menu entries

I assumed wrong. I'm seeing the kernel parameters have been moved to
the grubenv. That is the single most significant user facing change.

Also the path to $BOOT is defined in grub.cfg, so the BLS format being
used is consistent with the original BLS spec's format, in that paths
are relative to $BOOT.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/TAIPRFXCMFJ24U2MKTWQWR6GP6YZIPC7/

Re: F29 System Wide Change: Make BootLoaderSpec the default