Re: F38 proposal: Unified Kernel Support Phase 1 (System-Wide Change proposal)

[Javier Martinez Canillas]

On Wed, Dec 21, 2022 at 6:13 PM Demi Marie Obenour
 wrote:
>

[...]

>
> Does vfat support atomic rename?  Is it possible to atomically upgrade
> a bootloader/UKI/etc?
> --

For Linux, renameat2 RENAME_EXCHANGE is supported in vfat since
version v6.0. The relevant commits FYI are:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=019a0c9e377c
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=204d03203a14
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=da87e1725ae2

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: future of dual booting Windows and Fedora, redux

[Javier Martinez Canillas]

On Wed, Jul 27, 2022 at 10:31 PM Lennart Poettering
 wrote:
>

[...]

> > The lack of an upgrade path, I think, is a bigger issue than a
> > system-wide change proposal to: switch to systemd-boot on UEFI,
> > including FAT /boot partition, for new clean installs.
>
> I don't think the upgrade path would be so bad. Takes some careful
> work to get into place, but we went through worse migrations in Fedora
> I am sure.
>

Indeed, replacing grub2 with sd-boot on an installed Fedora system
would just require something like the following commands (assuming
that the Secure Boot situation is sorted out somehow):

  $ sudo bootctl install
  $ sudo mkdir /boot/efi/$(cat /etc/machine-id)
  $ sudo rm -r /boot/loader
  $ sudo dnf reinstall kernel-core -y

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: Legacy Xorg Driver Removal (System-Wide Change proposal)

[Javier Martinez Canillas]

On Thu, Apr 21, 2022 at 10:09 PM Demi Marie Obenour
 wrote:
>

[snip]

> > I'm also not sure I agree it's clear that we'd find more bugs if the
> > fallback path didn't exist. People don't usually just boot straight in
> > "basic graphics mode", after all. They try a regular boot, and if it
> > fails, maybe they try "basic graphics mode". So they already *know*
> > there's a bug - and at least this way we give them a working system and
> > maybe they'd be more motivated to file the bug than if we just leave
> > them stuck. We *do* get bug reports when this happens, it's not like we
> > never hear about these experiences.
>
> Could basic graphics mode use the EFI framebuffer on EFI systems?
>

It does already.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F37 Change: Deprecate Legacy BIOS (System-Wide Change proposal)

[Javier Martinez Canillas]

On Wed, Apr 6, 2022 at 5:51 PM Jared Dominguez  wrote:

[snip]

>
> Per my reply to you yesterday, I would be grateful if you would list out 
> examples here. This is the second time I've heard this, and it's not concrete 
> enough for a constructive conversation on that topic.
>
>> 2. The packages are locked down so there is no way for the community to help
>> 3. At various times, people have explicitly said "patches NOT welcome"
>
>
> Robbie already responded to this, but I would like to add that if any of this 
> ever actually becomes true, I would like to know so that I can address any 
> such issue with this Red Hat team. From what I've seen, we very much welcome 
> community involvement. In fact, we are collaborating on a daily basis with 
> the bootloader community on grub2 and shim development.
>

As Robbied said, I added the /etc/dnf/protected.d/grub2-*.conf to the
grub2 package after Neal filed that BZ and he also mentioned to me
back then that pull requests were disabled for the grub2 dist-git and
I re-enabled it after asking Peter if he was OK with that (and he told
me that sure go ahead).

BTW, Peter also did the same for shim and added
/etc/dnf/protected.d/shim.conf even though the dist-git for shim was
open at the time. So I don't see how grub2 dist-git not having pull
requests enabled (for whatever reasons) could be a blocker since a
change for shim wasn't proposed either. I mean, one could attach a
patch in the BZ, send an email to a maintainer, etc. Is not that PRs
in dist-git is not the only possible way to share a diff...

Anyway, I also personally never said "patches NOT welcome" to anyone.
It may be that I didn't have time to review/apply some proposed but
that's a different thing.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change proposal: Replace the fbdev drivers with simpledrm and the DRM fbdev emulation layer (System-Wide Change proposal)

[Javier Martinez Canillas]

On Tue, Oct 26, 2021 at 6:01 PM Neal Gompa  wrote:
>
> On Tue, Oct 26, 2021 at 11:27 AM Ben Cotton  wrote:
> >
> > https://fedoraproject.org/wiki/Changes/ReplaceFbdevDrivers
> >
> > == Summary ==
> >
> > This change replaces the legacy Linux frame buffer device (fbdev)
> > drivers that are still used in Fedora, with the latest simpledrm
> > driver and the DRM fbdev emulation layer.
> >
>
> I'd like to see this coupled with the addition of a flag so we can
> change "basic graphics mode" to force this. There's already a proposed
> patch series upstream for this[1], so I think this isn't too hard to
> achieve.
>

Agreed. I've set up a COPR that has a kernel build with the needed
config changes and the patches you mentioned. In case someone wants to
give a try to this:

https://copr.fedorainfracloud.org/coprs/javierm/simpledrm/

> This would unblock closing the remaining gaps on the X11->Wayland
> transition, since Mutter doesn't support fbdev for Wayland, and KWin's
> Wayland session doesn't work with fbdev on our kernel for some reason.
>

Indeed, I think that using simpledrm would be an improvement over efifb.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Grub 2 protected packages

[Javier Martinez Canillas]

On Mon, Apr 12, 2021 at 11:51 PM Chris Murphy  wrote:
>
> On Mon, Apr 12, 2021 at 4:01 AM Neal Gompa  wrote:
> >
> > Ideally, we should change to a system similar to what openSUSE does
> > and have the RPMs install bootloader content into /usr, then execute a
> > helper program that copies things over to /boot and configures things
> > properly (we should still have the files %ghosted in /boot, though!).
> > This makes it much more straightforward to support updating or
> > downgrading bootloader files when needed, which is why openSUSE does
> > it this way to support full system snapshots with rollback
> > functionality.
>
> That's the intent of bootupd.
> https://github.com/coreos/bootupd/
>
> And also include 'grub2-install' on BIOS systems to ensure the
> embedded instance is also kept up to date. And a possible future
> feature is EFI system partition syncing in the case of multiple ESPs.

Agreed that packages shouldn't install anything in /boot or update the
bootloader images (e.g: the embedding gap for x86 legacy BIOS, PReP
partition for ppc64le, etc) and instead all that setup logic should be
done by bootupd.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Grub 2 protected packages

[Javier Martinez Canillas]

On Sun, Apr 11, 2021 at 2:29 PM PGNet Dev  wrote:
>
> > Ordinarily, no. But in this case, since GRUB 2.06~rc1 is required to
> > solve major critical vulnerabilities and it's very difficult to pull
> > the patch set that fixes it (>115 patches!) backwards, GRUB got moved
> > forward instead.
> >
> > GRUB 2.06~rc1 was pretty much released to release the patch set...
>
> got it.
>

As Neal said, the other option was to backport all the CVE fixes and
SBAT support patches which was a riskier option.

Also, we are trying to get rid of the huge patch-set that the Fedora
package is currently carrying, not adding more to the set.

> then will stick with v2.06, and try to get it re-patched for Xen @ the bug.
>

I dropped it by mistake when rebasing to 2.06, sorry about that. I've
fixed it now in F33, F34 and Rawhide.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Xen support dead?

[Javier Martinez Canillas]

On Fri, Feb 5, 2021 at 2:40 PM PGNet Dev  wrote:
>
> > Actually, the buggy file (/etc/grub.d/20_linux_xen) belongs to the grub2
> > package, so the bug is assigned to a wrong package.
>
> Not that it matters, but I'd  originally assigned it to grub.  It was ignored 
> there as well.
> I switched it to Xen after I was told in #irc to do so.
>
> Too much pushing rocks uphill for my taste.  Building my own.

Sorry I didn't see this bug report until Neal pointed it to me.

The problem was in the 20_linux_xen script as Marek mentioned. Could
you please test the proposed fix in bugzilla? I don't have a Xen setup
to test.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)

[Javier Martinez Canillas]

On Thu, Jan 14, 2021 at 10:48 AM Chris Murphy  wrote:
>
>
>
> On Thu, Jan 14, 2021, 2:24 AM Vitaly Zaitsev via devel 
>  wrote:
>>
>> On 31.12.2020 13:36, Javier Martinez Canillas wrote:
>> > I'll update the proposal based on the feedback.
>>
>> And what about users, who use Fedora with other GNU/Linux distributions?
>> These distributions can also switch to the same GRUB2 configuration.
>> They will all start fighting for the same file. That's why a separate
>> /boot/efi/EFI/$distro_name/grub.cfg configuration file was introduced.
>
>
>
> That's Fedora specific. Upstream puts it in /boot/grub/ regardless of 
> firmware.
>

Also, this only solves the problem when installing different distros.
The problem still exists if multiple Fedora are installed since all
will share the same EFI vendor directory.

But yes, multi boot should probably also be taken into account in the
design. That's why it will be important to agree with GRUB upstream on
this and have buy-in from the other distros.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)

[Javier Martinez Canillas]

On Wed, Jan 13, 2021 at 4:48 AM Michel Alexandre Salim
 wrote:

[snip]

> > As discussed in detail here:
> > https://pagure.io/fedora-workstation/issue/206
> > we really should be moving away from that. As discussed there Suse
> > already has grub-patches to instead store the grubenv in an part of
> > the btrfs filesystem header which is reserved for bootloader use.
> >
> That's one of the option discussed, yes. One issue with doing it this
> way is we'll have to reimplement it for XFS and other future
> filesystems.
>

Yes, I don't think is worth it to cherry-pick Suse's patch only for
btrfs and instead should aim for a general solution.

> > As @javierm says in the linked fedora-workstation issue, that is
> > also the long term plan for Fedora, but we really want to discuss
> > and develop a solution for this with/at grub-upstream so that we
> > don't end up with conflicting solutions between distributions
> > which stomp all over each-others data.
> >
> Agreed. But if we decide to use a separate partition with a properly
> supported filesystem, we might as well pick it now rather than make
> changes twice, right?
>

The thing is that implementing this proposal should be straightforward
and something that could be done for F34 but solving the grubenv issue
will require more time, since we first will need to agree with GRUB
upstream on the approach.

Also, switching existing installations to the configuration scheme
mentioned in this proposal should be feasible but changing the
partition layout would be much more risky. That's why I believe that
even if we solve the grubenv issue (i.e: in F35), this should only be
for new installations and storing grubenv as a file has to be
supported for backward compatibility.

> Chris suggested using a BIOS Boot partition, but another possible
> option is to use XBOOTLDR from
> https://systemd.io/BOOT_LOADER_SPECIFICATION/ - which the BLS actually
> prefers over the ESP if found.
>

Chris' suggestion (unless I misunderstood) is not to use a partition
with a filesystem but instead a partition where the grubenv could be
stored as raw data. That way GRUB should be able to read and write the
grubenv block without using any of its filesystem code (that's
supposed to be read-only).

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)

[Javier Martinez Canillas]

On Tue, Jan 12, 2021 at 8:24 PM Brian C. Lane  wrote:

[snip]

> >
> > The `$prefix` variable will be set to the device partition where
> > `/boot/grub2/grub.cfg` is stored, using the partition filesystem's
> > Universally Unique IDentifier (UUID). That way the correct GRUB
> > configuration file will be loaded, making it as secure as the current
> > approach where the configuration file in the ESP is used.
>
> I don't see how adding more files and splitting information between
> them helps simplify things. You still need a system specific grub.cfg in
> the EFI directory so you haven't removed anything.
>

Yes, but this file should be static and never change. As mentioned, it
will just search the correct partition using GRUB's search command,
set a new $prefix variable and finally run `configfile
$prefix/grub.cfg`.

> And now users who are trying to debug things have more places to look
> for problems.
>
> If you want a consistent way to find the grub.cfg on bios and efi, why not
> create a symlink in /boot/grub2/ like we already do for grubenv?
> Actually, we already have symlinks in /etc/grub2.cfg and
> /etc/grub2-efi.cfg
>

Symlinks have been proven to be error prone and a constant source of
issues for users. In fact one of the goals of this change is to get
rid of the grubenv symlink.

Very often we have bugs filed about users removing the symlink, then
creating a new grubenv with `grub2-editenv create` in
/boot/grub2/grubenv and then complaining about the GRUB env variables
not being updated (because /boot/grub2/grubenv is now a real file
instead of a symlink to the one in the ESP).

So the goal is for users to only care about /boot/grub2/grub{.cfg,env}
and never having to touch the files in the ESP. That's what most
distros do (and even we do on images generated with coreos-assembler
and OSBuild), to avoid making EFI a special case and to have
consistency across EFI, x86_64 legacy BIOS and ppc64le.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)

[Javier Martinez Canillas]

On Thu, Dec 31, 2020 at 12:54 PM Vitaly Zaitsev via devel
 wrote:
>
> On 31.12.2020 12:37, Peter Robinson wrote:
> > Of course it could, who do you propose to do that work and support all
> > the various options and code required?
> It can be easily installed during Fedora installation by executing the
> following:
>
> 1. Make sure the ESP partition has more than 512 MB of free space
> (systemd-boot uses an ESP partition to store kernels; the separate /boot
> is no longer needed).
> 2. Add Fedora boot flags instead of the /etc/default/grub to the
> /etc/kernel/cmdline.
> 3. Install systemd-boot to the ESP partition: bootctl --path=/boot/efi
> install.
> 4. Execute kernel-install scripts (I think this stage will be the same
> as under GRUB2): kernel-install add $(uname -r) /lib/modules/$(uname
> -r)/vmlinuz.
> 5. Installation completed.
>

As Peter said this was already discussed in the "The future of legacy
BIOS support in Fedora" thread [0].  I mentioned there that Anaconda
already supports an option to use extlinux, so the same could be done
for sd-boot.

It shouldn't be a lot of work as you mentioned but someone will have
to propose the patches to Anaconda.

[0]: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/QBANCA2UAJ5ZSMDVVARLIYAJE66TYTCD/

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)

[Javier Martinez Canillas]

Hello Tomasz,

On Thu, Dec 31, 2020 at 10:55 AM Tomasz Torcz  wrote:

[snip]

> > I think either never fixing this, or never updating systems to the
> > "new way" are both untenable. We saw with the BLS switch many users
> > depend on doing in place upgrades. Many were pushing 4 or more years.
>
>   I think conversion script would be needed for people doing upgrades.
> But first of all - documentation! It should be clear what GRUB

Agree.

> expectations are, where the config file should be and so on.
> BLS conversion was hard because it was undocumented. One of the crucial
> scripts (grubby? install-kernel?) has been changing behaviour upon
> existence of some file or directory in /boot (was is loader/?). It was
> undocumented and so counter-intuitive that cannot recall details after
> merely two years.
>
>   Right now the best information about how Fedora boots and what happens
> on kernel installation can be founds on AdamW's blog. This is not
> perfect :(
>

Yes, we need to close the documentation gap.

The process is still quite complex. I think the long term goal should
be to align with the https://systemd.io/BOOT_LOADER_INTERFACE/ and
https://systemd.io/BOOT_LOADER_SPECIFICATION/ (and extend those specs
to cover all the missing bits for the bootloaders used by Fedora) in
order to make the interface between the bootloader and OS well
defined. That will make easy to mix and match bootloaders and OS, but
there is still a lot of work to do in order to achieve that.

>
> --
> Tomasz Torcz   72->|   
> 80->|
> to...@pipebreaker.pl   72->|   
> 80->|

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)

[Javier Martinez Canillas]

Hello Chris,

Thanks a lot for the comments.

On Thu, Dec 31, 2020 at 1:02 AM Chris Murphy  wrote:

[snip]

>
> That problem was the result of quite old core.img in the MBR gap (or
> BIOS Boot partition). As that change simultaneously depended on
> shipping a new GRUB module without a way to update the core.img with
> up-to-date GRUB modules, there was a known weak spot that we even knew
> of in advance.
>

Correct.

> Upgrades of customized configurations that deviate significantly from
> defaults aren't supported. It's best effort. We can't be blocking on
> people's customizations.
>

I agree with you but users have different expectations I think. If
they deviated from the default and that didn't cause issues for them
after a system wide upgrade, they expect that to be the case on the
next update.

> I think we can come pretty close to atomically renaming
>
> /EFI/fedora/grub.cfg /EFI/fedora/grub.cfg.old
> /EFI/fedora/grub.cfg.new to /EFI/fedora/grub.cfg
>
> And at least ensure the user can boot the old one, but even this I
> think is pretty unlikely. It's really a teeny tiny window of failure
> opportunity. And based on my reading of rename() if the files are
> already all present, and all we're doing is renaming them, there
> shouldn't be a case where grub.cfg is either missing entirely or zero
> bytes.
>
> But dm-log-writes can help confirm or deny this. What I don't know is
> if this can be done with bash. The convert script probably needs to be
> done in C. Or at least the rename and sync parts.
>

Yes, for me is less about this tiny window but more about users having
modifications in their GRUB config file that will be overwritten when
generating a new one. For example in the BLS conversation some users
update their kernel cmdline in the grub.cfg and that didn't match the
GRUB_CMDLINE_LINUX in /etc/default/grub. Maybe what we can do is to
not generate a new /boot/grub2/grub.cfg but instead copy the one in
the ESP to cover these corner cases ?

I'll update the proposal based on the feedback.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)

[Javier Martinez Canillas]

Hello Adam,

Thanks a lot for your feedback.

On Wed, Dec 30, 2020 at 9:22 PM Adam Williamson
 wrote:
>

[snip]

> > == Upgrade/compatibility impact ==
> >
> > The changes will only be for new installations, existing systems will
> > not be impacted and will continue using the grub.cfg and grubenv files
> > that are located in the ESP.
>
> To me several of the benefits seem to not really be true, so long as
> this is the plan for upgrades.
>
> * We wouldn't have a "consistent configuration" across everybody,
> really, because anyone who upgraded from pre-F34 would still have the
> old config; every bootloader debugging session ever would start by
> figuring out which case this was.
>
> * We can't really use the same "documentation and commands" for the
> same reason. We either have to document both possibilities forever, or
> accept that our docs will be incorrect for anyone who upgraded from
> pre-F34.
>
> * We can't really make the tools "more robust" in the way cited because
> they'll still have to handle both cases as long as both cases exist. If
> anything this makes them more fragile: the more divergent paths a tool
> has to support, the more likely it is something will break.
> --

These are all fair points. My worry is that trying to switch to the
new configuration on upgrades could lead to issues for people that
have custom GRUB configs. That was the case when we did the switch to
using BLS snippets and I don't really want to repeat that experience
for users.

That's why I went with the conservative approach of only do this for
new installs, to prevent breaking users configuration (or even worse,
their booting). Maybe a middle ground could be to provide a tool for
users to do the switch and make it opt-in?

Another option is to stick with the status quo but then we will never
be able to attempt improving this.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: The future of legacy BIOS support in Fedora.

[Javier Martinez Canillas]

On Mon, Jul 6, 2020 at 10:39 AM Jóhann B. Guðmundsson
 wrote:
>
> On 5.7.2020 18:34, Javier Martinez Canillas wrote:
> > On Sat, Jul 4, 2020 at 6:27 PM Lennart Poettering  
> > wrote:
> >
> > [snip]
> >
> >> Please submit additions to the spec as PRs to systemd github. We added
> >> a number of new keys in the past that sd-boot itself doesn't make use
> >> of (devicetree and such), and we'd be delighted to add more if they
> >> make sense and that helps.
> >>
> > Thanks. I'll discuss this with the rest of the bootloader folks and
> > think how the spec could be extended to cover the remaining cases
> > where variable expansion is still used for GRUB. The new keys could be
> > generic and even benefit other bootloaders if they implement these
> > features at some point (e.g: boot entries protection).
>
> Since you are in contact with and thus presumably you are one of the
> "bootloader folks" could you clarify who those individual are and what
> role they play and which bootloaders they represent in the distribution
> and on which arch etc. and where they can be contacted ( mailinglist )
> since I don't find any documentation about any bootloader WG existing
> within Fedora yet such a team seems to exist since it's being mentioned.
>

Sure, I meant the members of the Red Hat bootloader team (Peter Jones,
Jan Hlavac and me) and people who are not part of the bootloader team
but work very closely with us and help to improve the boot stack in
general. Mostly Hans de Goede and Christian Kellner but also others.

Peter maintains all the projects in https://github.com/orgs/rhboot and
their respective packages in Fedora. And I help him with that work. We
are also involved in the upstream communities of the bootloaders that
are used in the architectures supported by Fedora. These are:

- GRUB (x86_64 legacy BIOS and EFI, aarch64 EFI and ppc64le OF)
- Petitboot (ppc64le OPAL)
- zipl (s390x)
- u-boot (armv7).

But for the last two most of the work and the package maintenance is
done by Dan Horák (s390utils-base) and Peter Robinson
(uboot-images-armv7).

All these people can be contacted in the Fedora devel mailing list. I
hope this answers your question, please let me know if you need more
details.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: The future of legacy BIOS support in Fedora.

[Javier Martinez Canillas]

On Sat, Jul 4, 2020 at 6:27 PM Lennart Poettering  wrote:

[snip]

>
> Please submit additions to the spec as PRs to systemd github. We added
> a number of new keys in the past that sd-boot itself doesn't make use
> of (devicetree and such), and we'd be delighted to add more if they
> make sense and that helps.
>

Thanks. I'll discuss this with the rest of the bootloader folks and
think how the spec could be extended to cover the remaining cases
where variable expansion is still used for GRUB. The new keys could be
generic and even benefit other bootloaders if they implement these
features at some point (e.g: boot entries protection).

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: The future of legacy BIOS support in Fedora.

[Javier Martinez Canillas]

On Wed, Jul 1, 2020 at 4:36 PM Lennart Poettering  wrote:
>
> On Mi, 01.07.20 13:14, Javier Martinez Canillas (jav...@dowhile0.org) wrote:
>
> > I'm not sure if this is completely fair, it's true that GRUB's blscfg
> > module diverged from the spec by adding support for variables but it
> > can also parse BLS snippets that follow the spec verbatim.
>
> You missed the point of the whole spec:
>
> the spec was that every party which interfaces with boot loaders knows
> where to find and how to deal with boot loader entries, and to make
> them as simple that every party can easily parse them and make sense
> of them, and share them. This means that many parties can *read* them
> without trouble and *drop-in* them without trouble either.
>
> By turning them into a programming language you broke that contract,
> because suddenly your files cannot be read without any embedded
> tremplating language interpretor. You own your files, but everyone
> else cannot make sense of it.
>

I've already acknowledged that using variables in the options field
was a mistake, since that is a known field according to the spec and
so it broke the contract. And also mentioned that this was already
fixed in F33, other boot loaders should now be able to parse the BLS
snippets (assuming that ignore other fields only relevant to GRUB for
boot entries auth).

> Note that the spec has extension points (i.e. it's permissible to add
> new fields without this breaking the spec), but turning it into a
> programming lnaguage is wy outside of it...
>

I wouldn't consider adding variable expansion support to turn them
into a programming language. But yes, you are right that we diverged
from the spec and that caused issues to other bootloaders (i.e: I had
this same conversation with the LinuxBoot folks).

But rather than keep pointing out what we got wrong, I would prefer to
figure out how to make the GRUB implementation to align with the spec
while still supporting all the features that are available in a
non-BLS configuration. This could also allow to have a single
kernel-install plugin instead of having specific plugins for
GRUB/Petitboot and zipl.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: The future of legacy BIOS support in Fedora.

[Javier Martinez Canillas]

On Wed, Jul 1, 2020 at 11:37 AM Lennart Poettering  wrote:

[snip]

>
> My suggestion would be: don't standardize on boot loaders, standardize
> on the boot loader spec. And I mean, the real boot loader spec, i.e

Agreed. In part that's already the case for Fedora, since now besides
GRUB the zipl (s390x) and Petitboot (ppc64le OPAL) bootloaders are
also using BLS snippets to populate their boot menu.

I don't see why one bootloader has to be dropped in favour of the
other, and not just allow users to choose what better fits their
needs.

Just like Anaconda currently provides an extlinux option to use as the
bootloader instead of GRUB for legacy BIOS installs, a sd-boot option
could be added to choose using this bootloader for EFI installs.

That way users who want a simple bootloader can use sd-boot and those
needing features like network boot, LUKS support, etc can choose to
use GRUB instead.

> not this terrible template language that Fedora now supports in Grub,
> which is just the same old grub complexity again. They stole the "Boot

Yes, not storing the kernel command line options in the BLS snippets
and using a GRUB variable was indeed a mistake that caused more harm
than good.

That has been fixed for F33, but there are other reasons why the GRUB
implementation needed variables. For instance to support
authentication and authorization of boot entries:

https://www.gnu.org/software/grub/manual/grub/html_node/Authentication-and-authorisation.html

But we can look at how the spec could be extended to cover that case
and stop using the templating for GRUB altogether to align with the
spec.

> Loader Spec" name and turned it into something that is not related at
> all to the real thing.
>

I'm not sure if this is completely fair, it's true that GRUB's blscfg
module diverged from the spec by adding support for variables but it
can also parse BLS snippets that follow the spec verbatim.

For example Fedora CoreOS uses it and the BLS snippets created by
OSTree are completely aligned with the spec as far as I can tell.
And since F33 I think that even the BLS snippets created for GRUB
could be parsed by sd-boot (or any other bootloader following the spec
like LinuxBoot)
since the options field doesn't have a variable anymore.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Does the installer detects when a distro have already created BLS?

[Javier Martinez Canillas]

On Mon, May 25, 2020 at 4:00 AM Chris Murphy  wrote:
> On Sun, May 24, 2020 at 7:48 PM James Cassell
>  wrote:
> > On Sun, May 24, 2020, at 9:39 PM, Chris Murphy wrote:
> > > On Sun, May 24, 2020 at 6:42 PM Paul Dufresne via devel
> > >  wrote:
> > > > Le 20-05-24 à 19 h 34, Naheem Zaffar a écrit :

[snip]

> > > But an additional difficulty to fully implementing the spec is so far
> > > upstream GRUB don't want to follow it. So that means Fedora has to
> > > carry patches for GRUB to support it. And it's just yet another of
> > > 100+ patches Fedora carries for GRUB, and makes it difficult for the
> > > Fedora and RH boot teams.  The resources so far implement some of the
> > > parts of BLS that help make things better on Fedora, but it's not a
> > > complete implementation. Drop-in snippets to add new kernels is crash
> > > safe, worst case the previous kernel is booted and you just reinstall
> > > the kernel; but writing out a new grub.cfg or modifying it, wasn't
> > > ever crash safe. Also, modifying the snippets is easier, they're just
> > > a few lines and fairly self-describing compared to what users often
> > > did, which was wade neck deep into editing grub.cfg. Or the Rube
> > > Goldberg machine that is editing /etc/default/grub, running a script
> > > (grub-mkconfig), which then runs 20 other scripts to create a
> > > configuration file that is actually a script.

Yes, what was implemented in GRUB and also the other bootloaders used
in Fedora (Petiboot and zipl) was support to parse BLS snippets. The
goal was to have a consistent bootloader configuration scheme and file
format across all the different architectures and bootloaders
supported by Fedora. This allowed to simplify and improve the
bootloader configuration tooling for the reasons Chris already
mentioned. We could get rid of the old grubby tool that was used
before to parse and modify the bootloader specific configuration
files, which was quite fragile since users also changed these files
breaking the assumptions made by grubby.

Using BLS snippets for all Fedora variants also aligned better with
OSTree based ones, that were already using BLS snippets for the boot
entries.

> >
> > Even so, isn't the canonical way of persistently updating kernel args, 
> > still, to edit /etc/default/grub and run the script? (If not, are there 
> > docs for the new way?)
>
> It does still work, but by indirection. You set it in
> /etc/default/grub but grub2-mkconfig puts it into grub.cfg first and
> then it goes into grubenv. The grubenv variables are loaded by GRUB at
> boot time, and the BLS snippets reference those variables.
>

This still works from F33 onwards. But instead of grub2-mkconfig
updating a variable in grubenv, the options field of the BLS snippets
are updated with the kernel cmdline set in /etc/default/grub. This is
done for backward compatibility because as mentioned in this thread,
is the canonical form to update the kernel cmdline and documented
everywhere.

> I've resorted to using 'grub2-editenv' to directly modify grubenv. But
> as grubenv is fragile, using it will be abandoned.
>

There's a grubby script that is provided for backward compatibility,
that just modifies the BLS snippets. So this can also be used to
update the kernel cmdline.

I agree that we need better tools for users to manage the BLS snippets
and also improve the documentation.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Grub, EFI, and SELinux

[Javier Martinez Canillas]

On Mon, May 4, 2020 at 10:20 AM Panu Matilainen  wrote:

[snip]

> > I've been closing as duplicates of #1722766 but we are just getting
> > too many bugs filed for this issue.
>
> It's an entirely cosmetical issue in rpm SELinux plugin but as innocent
> maintainers are apparently getting bombarded because of it:
>
> https://bodhi.fedoraproject.org/updates/FEDORA-2020-feefa460b1
> https://bodhi.fedoraproject.org/updates/FEDORA-2020-54205e879b
>

Thanks a lot!

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Grub, EFI, and SELinux

[Javier Martinez Canillas]

On Sun, May 3, 2020 at 4:40 AM Jerry James  wrote:
>
> On Sat, May 2, 2020 at 4:33 AM Christopher  wrote:
> > Those are bugs filed against RPM. Is the RPM package responsible for
> > executing lsetfilecon, or is it the grub2 package? If the grub2
> > package, it seems to me that they should know that EFI partitions will
> > never support lsetfilecon and they should never try. If it's RPM, then
> > it looks like it is suppressed upstream and the fix will be
> > incorporated eventually. I guess I don't know which component is
> > actually responsible for causing the execution of lsetfilecon.
>
> You're right, but there is discussion of the grub2 issue in bug
> 1722766.  A number of bugs have been filed against grub2 specifically:
>

Nothing in the grub2 package executes restorecon for the files in
/boot/efi. The problem is that rpm calls lgetxattr() for each entry in
%files, regardless if the filesystem supports extended attributes or
not:

https://bugzilla.redhat.com/show_bug.cgi?id=1722766#c43
https://github.com/rpm-software-management/rpm/pull/976

> https://bugzilla.redhat.com/show_bug.cgi?id=1819817
> https://bugzilla.redhat.com/show_bug.cgi?id=1827922
> https://bugzilla.redhat.com/show_bug.cgi?id=1829137
> https://bugzilla.redhat.com/show_bug.cgi?id=1830399
>
> So far, though, no word from the maintainer on those bugs.

I've been closing as duplicates of #1722766 but we are just getting
too many bugs filed for this issue.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Does grub2-pc do anything on an EFI system?

[Javier Martinez Canillas]

Hello Richard,

On Sat, Oct 26, 2019 at 1:23 PM Richard Shaw  wrote:
>
> The grub2-pc package doesn't seem to do anything except provide a broken link 
> in /etc to grub2.cfg...
>

You are correct, the package is only needed for legacy BIOS installs.
There was a bug in Anaconda that caused grub2-pc to wrongly be
included for EFI installs. This was fixed by:

https://github.com/rhinstaller/anaconda/commit/8a8cc7a1a9bee434d07836e43ea9d43220752bab

>  ll /etc/grub2.cfg
> lrwxrwxrwx. 1 root root 22 May 20 12:19 /etc/grub2.cfg -> 
> ../boot/grub2/grub.cfg
>
> The /boot/grub2/grub.cfg must be a %ghost entry in the rpm as the file 
> doesn't actually exist:
>

Yes is a %ghost file and created by Anaconda on installation using the
grub2-mkconfig tool. Just like the /etc/grub2-efi.cfg file on EFI
installs, that's also a %ghost file from the grub2-efi package.

> # rpm -ql grub2-pc
> /boot/grub2/grub.cfg
> /boot/loader/entries
> /etc/grub2.cfg
>
> The description from the info about the package is also useless:
>
> The GRand Unified Bootloader (GRUB) is a highly configurable and
> customizable bootloader with modular architecture.  It supports a rich
> variety of kernel formats, file systems, computer architectures and
> hardware devices.
>
> This subpackage provides support for %{1} systems.
> ---
>
> It only has the standard description and a line with a variable that's no 
> longer getting expanded.
>

This has also been fixed:

https://src.fedoraproject.org/rpms/grub2/c/af06f22ee4f945189d5cd02ba4b437b9a63ca7c9

> I removed it from my system and I'm about to reboot :)
>

Yes, it's OK to remove the grub2-pc package on an EFI system.

> Thanks,
> Richard
>

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: removing Xen from Fedora release criteria

[Javier Martinez Canillas]

On Tue, Oct 8, 2019 at 3:53 AM Chris Murphy  wrote:
>
> Release criterion
> https://fedoraproject.org/wiki/Fedora_31_Final_Release_Criteria#Xen_DomU
>
> Bug since Fedora 30 also affects Fedora 31 and has been proposed as a
> Fedora 31 blocker bug
> https://bugzilla.redhat.com/show_bug.cgi?id=1703700
>

This bug has been fixed now  and the Xen folks have tested that it
works for them. I've also backported the fix for F30.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: always update the bootloader during major upgrades

[Javier Martinez Canillas]

Hello Chris,

Thanks for bringing this topic and sorry for the late reply.

On Wed, Jun 26, 2019 at 10:20 PM Chris Murphy  wrote:
>
> Hi,
>
> This is not a formal proposal, this is for discussion and identifying
> liabilities. This email has an x86 GRUB bias only because that's the
> bootloader regime I'm most familiar with. I think it should apply to
> other archs as well, i.e. their bootloaders shouldn't be permitted to
> become stale.
>

I agree with you on this. I've commented my thoughts in the
fedora-workstation issue you created:

https://pagure.io/fedora-workstation/issue/97#comment-580794

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

On Thu, Feb 14, 2019 at 9:12 PM Chris Murphy  wrote:
>
> On Thu, Feb 14, 2019 at 3:33 AM Javier Martinez Canillas
>  wrote:
>
> > > - new Fedora 30 installations, and upgrades of Fedora 28 and 29 to Fedora 
> > > 30
> >
> > Correct. Although I would just say upgrades from F29 to F30. Or can
> > you skip a release when doing a system upgrade?
>
> Fedora 28 folks will see an option to upgrade to Fedora 30 in Gnome
> Software, and it is a direct upgrade.
>

I see, I was not aware of that. It should also work but I'll test to
make sure that's the case.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

Hello Wolfgang,

On Thu, Feb 14, 2019 at 1:51 PM Wolfgang Ulbrich  wrote:
>
> How can i disable this new feature complete?

You can disable it by removing the GRUB_ENABLE_BLSCFG=true from
/etc/default/grub and re-generating your grub2.cfg with grub2-mkconfig
(i.e: grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg). You need to
install the grubby-deprecated package so the old grubby updates the
menu entries in the grub2.cfg file.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

On Thu, Feb 14, 2019 at 5:06 AM Chris Murphy  wrote:
>
> On Tue, Feb 12, 2019 at 5:08 AM Javier Martinez Canillas
>  wrote:
> > You are correct, we will work on more documentation and also better
> > tooling to manage the BLS snippets, grubenv file, etc. For example
> > grub2-editenv is very fragile as was mentioned in this thread.
>
> I'm looking at this:
> https://docs.fedoraproject.org/en-US/fedora/f29/system-administrators-guide/kernel-module-driver-configuration/Working_with_the_GRUB_2_Boot_Loader/
>
> I'm noticing that grubby on F30 is actually grubby-bls. On the one hand:
>

Yes. The grubby-bls script supports all the grubby options for
backward compatibility, but under the hood it just manages the BLS
snippets.

> # grubby --set-default /boot/vmlinuz-5.0.0-0.rc5.git0.1.fc30.x86_64
> # cat /boot/efi/EFI/fedora/grubenv
> # GRUB Environment Block
> saved_entry=8a6804fc8b1447b49bf2b522e419f2ae-5.0.0-0.rc5.git0.1.fc30.x86_64
> [snip]
>
> That appears to work. However,
>
> [chris@localhost ~]$ sudo grubby --remove-args=quiet

That's not a valid grubby command, even for the old grubby tool:

$ sudo grubby --info=/boot/vmlinuz-5.0.0-0.rc6.git0.1.fc30.x86_64 | grep args
args="ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root
rd.lvm.lv=fedora/swap rhgb quiet"

$ sudo grubby --remove-args=quiet
grubby: no action specified

$ sudo grubby --update-kernel=/boot/vmlinuz-5.0.0-0.rc6.git0.1.fc30.x86_64
--remove-args=quiet

$ sudo grubby --info=/boot/vmlinuz-5.0.0-0.rc6.git0.1.fc30.x86_64 | grep args
args="ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root
rd.lvm.lv=fedora/swap rhgb"

> [chris@localhost ~]$ sudo cat /boot/efi/EFI/fedora/grubenv
> [snip]
> kernelopts=root=/dev/mapper/fedora_localhost--live-root ro
> resume=/dev/mapper/fedora_localhost--live-swap
> rd.lvm.lv=fedora_localhost-live/root
> rd.luks.uuid=luks-8dc728e5-4420-4f90-a1c5-aaf87a69d88b
> rd.lvm.lv=fedora_localhost-live/swap rhgb quiet
> boot_indeterminate=0
>
> That doesn't work. Nor does --remove-args="quiet" - nor is there an
> error message. It just fails silently. And the --args argument doesn't
> appear to add arguments to the grubenv (or the bls snippets), also
> fails silently. Is it a bug?
>

Yes, the tool not printing an error message is a bug. I've fixed it
now, thanks a lot for reporting this.

The following will work with the new grubby script (and also the old
grubby tool):

$ sudo grubby --info=/boot/vmlinuz-5.0.0-0.rc6.git0.1.fc30.x86_64 | grep args
args="ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root
rd.lvm.lv=fedora/swap rhgb quiet"

$ sudo grubby --update-kernel=/boot/vmlinuz-5.0.0-0.rc6.git0.1.fc30.x86_64
--remove-args=quiet

$ grubby --info=/boot/vmlinuz-5.0.0-0.rc6.git0.1.fc30.x86_64 | grep args
args="ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root
rd.lvm.lv=fedora/swap rhgb"

Now, the grubby script doesn't work as you expected it. The
$kernelopts variable in grubenv is used to set a kernel command line
that's shared by all the BLS snippets, that allows you to have a
single place where you can update the cmdline for all the entries.

You have two ways to update this:

1) Using grub2-editenv set option

2) Setting GRUB_CMDLINE_LINUX in /etc/default/grub and running
grub2-mkconfig again.

Option (2) is what most people are used to and what we have documented
in many places, so we supported for backward compatibility. The
grub2-mkconfig tool will set $kernelopts with the value in
GRUB_CMDLINE_LINUX.

The grubby script (and the old grubby tool) are used to change
individual boot entries. So you need to specify a kernel with
--update-kernel (you can also specify DEFAULT for the default entry or
ALL to update all the entries).

Since the $kernelopts is shared by all the entries, the grubby script
can't change, since that will affect all the others. So what we
decided in this case is to make the grubby-bls script to copy-on-write
the kernel cmdline. That is, when you change an cmdline for an entry
it will expand the $kernelopts variable, do the changes and store the
modified cmdline in the BLS options field instead of $kernelopts. So
from now on the modified entry won't share the $kernelopts anymore and
will have it's own cmdline. Any changes to $kernrelopts won't affect
this entry anymore and the grubby script should be used for any future
updates.

As an example:

$ sudo grep kernelopts /boot/grub2/grubenv
kernelopts=root=/dev/mapper/fedora-root ro
resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root
rd.lvm.lv=fedora/swap rhgb quiet

$ sudo grep options
/boot/loader/entries/036f3661be4047b6b678d491f76df6f4-5.0.0-0.rc6.git0.1.fc30.x86_64.conf
options $kernelopts

$ sudo grubby --update-kernel=/boot/vmlinuz-5.0.0-0.rc6.git0.1.fc30.x86_64
--remove-args=quiet

$ sudo grep kernelopts /boot/grub2/grubenv
kernelopts=root=/dev/mapper/fedora-root

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

Hello Chris,

On Wed, Feb 13, 2019 at 7:49 PM Chris Murphy  wrote:
>
> I glossed over the fact upgrades to Fedora 30 will be converted to the
> "bls way" of things. So I want to be sure I understand feature scope:
>
> - all Fedora 30 editions and spins and archs, *except* 32-bit ARM

Yes, because for ARMv7 we are still booting from u-boot directly, so
it needs the old grubby tool to update the extlinux.conf that's used
by u-boot.

There's a plan to use the u-boot uEFI stub to chain-load grub2 as it's
already done for aarch64 boards AFAIK. If that's the case we could
also use BLS for ARMv7.

The Changes wiki page for that feature says that it's targeted for
F30, but AFAIU it probably won't happen in time and we will have to
keep a non-BLS configuration for ARMv7 in F30.

https://fedoraproject.org/wiki/Changes/uEFIforARMv7

> - new Fedora 30 installations, and upgrades of Fedora 28 and 29 to Fedora 30

Correct. Although I would just say upgrades from F29 to F30. Or can
you skip a release when doing a system upgrade?

> - does not apply to install media (right now I'm seeing install media
> still contains monolithic grub.conf, grub.cfg and isolinux.cfg that
> have all menu entries)
>

Yes, I'm not that familiar with how the install media is created and
its boot configuration. But I think that BLS makes less sense there
since the menu entries will always be a static configuration anyways.

> Thanks,
> ---
> Chris Murphy

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

Hello Nicolas,

On Tue, Feb 12, 2019 at 2:58 PM Nicolas Mailhot
 wrote:
>
> Le 2019-02-12 13:32, Javier Martinez Canillas a écrit :
> Hi Javier
>
> > I didn't get how it will break the symlink.
>
> You get a "environment block too small" on kernel update
>
> You type "environment block too small" in google because you want the
> update to work
>
> You get some ubuntu advice that says "rm grubenv grub-editenv grubenv"
>

I see, I would had thought that the advice would had been to use
grub-editenv create instead. But still grub2-editenv create will only
prevent removing the symlink, it won't help with the issue of the
kernelopts variable not being set.

We should make editing the grubenv more reliable to avoid these issues.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

On Mon, Feb 11, 2019 at 8:53 PM Chris Murphy  wrote:
>
> On Mon, Feb 11, 2019 at 5:40 AM Nicolas Mailhot
>  wrote:
> >

[snip]

> >
> > FYI I had to rescue two EFI rawhide system this week-end borked by grub
> > changes. As far as I could reconstruct:
> >
> > 1. the new grub needs the env file to be regenerated or kernel scriplets
> > will fail "environment block too small"
>
> The new behavior only applies to Fedora 30 clean installs; upgraded
> Fedora 29 and older should not get the new behavior, as I understand
> it.
>

The plan is to switch the GRUB configuration to BLS on a F29 -> F30 upgrade.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

Hello Nicolas,

On Mon, Feb 11, 2019 at 1:41 PM Nicolas Mailhot
 wrote:
>
> Le 2019-02-10 20:05, Chris Murphy a écrit :
> > On Wed, Feb 6, 2019 at 1:08 AM Javier Martinez Canillas
>
> > Between this feature for F30, and the F29 feature to hide the grub
> > menu which comes with boot success+fail marking by using the grubenv,
> > there are substantial changes in bootloading on Fedora that exist no
> > where else and near as I can tell there is no documentation at all. I
> > can't really call specs we don't fully follow, or feature pages, to be
> > documentation.
>
>
> FYI I had to rescue two EFI rawhide system this week-end borked by grub
> changes. As far as I could reconstruct:
>
> 1. the new grub needs the env file to be regenerated or kernel scriplets
> will fail "environment block too small"

Yes, grub2-editenv is too fragile. The problem is when the grubenv
file is modified without grub2-editenv, that breaks the assumption
grub2-editenv makes that the file will be filled with # characters to
always have a size of 1024 bytes. This was also the case on non-BLS
configuration, but in that case only the saved_entry variable was set
while for BLS also kernelopts is set.

> 2. there are *two* versions of this file, one in EFI directory space
> another in /boot/grub2
> 3. half our tools think the correct path if the first one, the other the
> second is the correct one
> 4. they all depend on things written by the other half in a "common"
> file
> 5. it only works because the boot/grub2 is a symlink to the EFI version,
> syncing all the tools
> 6. but nothing makes sure it is
> 7. you you follow net advice blindly, you will break the symlink while
> regenerating the env file and fixing the error in 1.

I didn't get how it will break the symlink. IIRC grub2-editenv create
will just follow the symlink and "empty" the grubenv file (where empty
for GRUB means adding the '# GRUB Environment Block' header and
filling with # characters up to 1024 bytes).

> 8. the result is unbootable, it will miss the kernelopts line in the
> file the EFI bootloader reads. No kernelopts line, no root to pivot to
> in initramfs

I also wonder how you got in this situation, the
grub2-switch-to-blscfg script should had restored the original
(non-BLS) grub2.cfg file if grub2-editenv set failed. I tried to
reproduce your issue with a borked grubenv but couldn't, I'll try to
dig deeper on this.

>
> Regards,
>
> --
> Nicolas Mailhot

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

Hello Chris,

Sorry for the late response but I was on vacation last week.

On Sun, Feb 10, 2019 at 8:06 PM Chris Murphy  wrote:
>
> On Wed, Feb 6, 2019 at 1:08 AM Javier Martinez Canillas
>  wrote:
> >
> > On Wed, Feb 6, 2019 at 5:28 AM Chris Murphy  wrote:
> > >
> > > https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault
> > >
> > > I want this change to succeed but I'm experiencing a regression, and
> > > while trying to troubleshoot it I'm finding it difficult to understand
> > > the myriad differences:
> > >
> >
> > Is this on F29 or Rawhide? Also, it's on an EFI or BIOS install?
>
> Rawhide UEFI. I thought it was a regression compared to grubby and
> grub2-mkconfig, because the problem doesn't happen with them. But
> those tools work directly on the grub.cfg which on UEFI only ever
> appears on FAT. Meanwhile, blscfg's .conf files are located at
> /boot/loader/entries which on this particular test setup is subject to
> zstd compression, and GRUB doesn't yet support zstd compression. GRUB
> couldn't read the file contents, therefore it's not a regression, it's
> a missing feature. Removing the compression solved the problem.
>

I see, thanks a lot for the explanation.

>
> > > there's a lot of grub2 documentation in Fedora and outside Fedora, all
> > > of which conflicts with this feature. And Fedora's experts in QA and
> >
> > Could you please elaborate a little bit? If you see a big mismatch
> > between some documentation and how the BLS support behaves, please
> > point out so we can try to make it more transparent for users.
>
> If a user has bootloader related issues, I ordinarily would only ask
> for a copy of the grub.cfg. But with bls systems, I need to ask for
> the grub.cfg, all the .conf files, and the grubenv file. All of them
> need to be interpreted. Each of those files are only described in
> generic terms by upstream GRUB, and no one else is using those files
> in the specific way they're being used in Fedora.
>

Yes, a BLS configuration comes with a different set of trade offs. Not
having the boot entries in the grub2.cfg has the advantage that the
file doesn't have to be modified on kernel install / removal, but as
you said it also means that it doesn't contain all the information of
your boot configuration.

> Between this feature for F30, and the F29 feature to hide the grub
> menu which comes with boot success+fail marking by using the grubenv,
> there are substantial changes in bootloading on Fedora that exist no
> where else and near as I can tell there is no documentation at all. I
> can't really call specs we don't fully follow, or feature pages, to be
> documentation.
>

You are correct, we will work on more documentation and also better
tooling to manage the BLS snippets, grubenv file, etc. For example
grub2-editenv is very fragile as was mentioned in this thread.

>
>
> > On Rawhide the blscfg command now supports to load BLS entries to
> > troubleshoot these cases. For example you should be able to do:
> >
> > blscfg (hd0,gpt2)/loader/entries/
> >
> > to load all the BLS snippets from a directory or:
> >
> > blscfg 
> > (hd0,gpt2)/loader/entries/loader/entries/036f3661be4047b6b678d491f76df6f4-5.0.0-0.rc4.git3.1.fc30.x86_64.conf
> >
> > to load a single BLS snippet.
>
> Ok that is a useful troubleshooting technique. In my case this was
> totally silent, ergo it found nothing even though there were files
> there. Adding 'set debug=all' and then following that up with blscfg
> exposed what directories it was looking at and the contents weren't
> readable - and that's when I clued in.
>

Got it, I'm glad that you figured out the issue.

>
>
> --
> Chris Murphy

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 change, bootloaderspec by default

[Javier Martinez Canillas]

Hello Chris,

On Wed, Feb 6, 2019 at 5:28 AM Chris Murphy  wrote:
>
> https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault
>
> I want this change to succeed but I'm experiencing a regression, and
> while trying to troubleshoot it I'm finding it difficult to understand
> the myriad differences:
>

Is this on F29 or Rawhide? Also, it's on an EFI or BIOS install?

> - I can't find the code. I assume all of it is in grub as blscfg.mod
> and grub2-mkconfig and its associated scripts in /etc/grub.d but I'm

That's correct, most of changes are in /etc/grub.d/10_linux and
blscfg.mod as you said:

https://github.com/rhboot/grub2/blob/fedora-30/util/grub.d/10_linux.in
https://github.com/rhboot/grub2/blob/fedora-30/grub-core/commands/blscfg.c

> not seeing code or code reference in the change or here
> https://apps.fedoraproject.org/packages/grub2-efi-x64-modules/
>
> - I can't find any documentation. In particular documentation that
> explains how it differs from upstream logic and what files the user
> should be interacting with to do typical tasks (typical for those who
> tend to interact with the bootloader which should be a minority of
> user); I know those differences are really substantial, not least of
> which is how to change boot parameters. This is important because

It depends on how you do it. Changing GRUB_CMDLINE_LINUX in
/etc/default/grub and running grub2-mkconfig should have the same
effect that with a non-bls setup. Using the grubby options (through
the BLS wrapper) is also supported for users using grubby.

> there's a lot of grub2 documentation in Fedora and outside Fedora, all
> of which conflicts with this feature. And Fedora's experts in QA and

Could you please elaborate a little bit? If you see a big mismatch
between some documentation and how the BLS support behaves, please
point out so we can try to make it more transparent for users.

> those who interface with user as volunteer support agents, will need
> to thoroughly understand the change, otherwise I'm concerned the
> common advice will become "reinstall" or even just disable this
> feature.
>
> -How does the blscfg module find /boot/loader/entries? Is this path
> hard wired somehow? I've got three different installations, all three

It tries to look for them in ($root)/loader/entries/ and
($root)/boot/loader/entries/ for EFI and ($boot)/loader/entries/ and
($boot)/boot/loader/entries/ for BIOS. The blsdir variable in grubenv
can also be if you want a custom BLS directory path.

> have identical grub environment variables shown by "set" command, and
> yet one of the installations doesn't find /boot/loader/entries so the
> only menu entry is to enter firmware.
>

Is there any difference between the systems that work and the one who
doesn't? Could you please fill a bug for it, that will make the issue
easier to track.

On Rawhide the blscfg command now supports to load BLS entries to
troubleshoot these cases. For example you should be able to do:

blscfg (hd0,gpt2)/loader/entries/

to load all the BLS snippets from a directory or:

blscfg 
(hd0,gpt2)/loader/entries/loader/entries/036f3661be4047b6b678d491f76df6f4-5.0.0-0.rc4.git3.1.fc30.x86_64.conf

to load a single BLS snippet.

> - Is upstream expected to eventually accept this change? Have they
> given feedback on the change? Are there any other distributions

Not yet, it's hard to engage with upstream GRUB and even to upstream
simple patches take time.

> planning on adopting it, maybe downstreams of ostree?
>

Besides Fedora and RHEL, Endless OS is using our GRUB blscfg patches.
We have also been talking with the ostree folks to use the module so
they can drop the ostree admin instutil grub2-generate command that
genrates a GRUB config from the generated BLS snippets.

I'll be on PTO since today and without good connectivity, so I may not
be able to answer more emails until next week. Please ping Peter Jones
if you need help with this.

>
> --
> Chris Murphy

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F30 Self-Contained Change proposal: Improved GRUB menu

[Javier Martinez Canillas]

Hello Zbyszek,

Thanks a lot for your feedback.

On Tue, Feb 5, 2019 at 10:33 AM Zbigniew Jędrzejewski-Szmek
 wrote:
>
> On Tue, Jan 29, 2019 at 05:40:30AM -0500, Matthew Miller wrote:
> > On Tue, Jan 29, 2019 at 05:29:51AM -0500, Ben Cotton wrote:
> > > Improve the GRUB menu by only having the default boot option for each
> > > installed operating system in the main menu, and the other options
> > > into a sub-menu. This would better organize the boot options and lead
> > > to an easier and seamless boot experience.
> >
> > +1. I often see new users asking "why are there multiple Fedora choices?", 
> > or
> > "which kernel should I use?"
>
> The Change page does not make it clear: is it possible to opt out of the
> change, i.e. keep the current menu structure after grub2-mkconfig has run?
>

Yes, the Change page isn't clear. What I was thinking was to have a
GRUB_ENABLE_SIMPLE_MENU option in /etc/default/grub that would control
how the /etc/grub.d/* snippets would generate the grub menu. So users
can opt-in by setting it to true and re-running grub2-mkconfig again.
Conversely, users may opt-out by removing GRUB_ENABLE_SIMPLE_MENU from
/etc/default/grub (or not setting it to true) and running
grub2-mkconfig again. It's not possible to opt-out after
grub2-mkconfig as run since that it's what generates the grub2.cfg
that will contain the menu entries. But it's possible to re-generate
the grub2.cfg file.

I also think that Anaconda should only set
GRUB_ENABLE_SIMPLE_MENU=true for Workstation, like we do with the GRUB
menu auto hide feature. I'll update the Changes page to reflect this.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: F29 System Wide Change: Make BootLoaderSpec the default

[Javier Martinez Canillas]

On Thu, Jun 28, 2018 at 3:34 AM, Dennis Gilmore  wrote:
> El jue, 14-06-2018 a las 12:06 +0200, Jan Kurik escribió:
>> = Proposed System Wide Change: Make BootLoaderSpec the default =
>> https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault

[snip]

>> Fedora already has a lot of infrastructure in place to not require
>> modifying bootloader configuration files for boot menu entries. The
>> BootLoaderSpec and drop-in BLS fragments can be used instead, and the
>> kernel-install script can do any additional task that is currently
>> done by grubby. The kernel-install script has a pluggable design that
>> uses a drop-in directory for scripts to extend its functionality. So
>> if needed, any bootloader specific logic can be implemented as
>> kernel-install scripts.
>> With this setup the bootloader configuration could be static and not
>> modified after installation.
>> The missing piece was the lack of BLS support on all the supported
>> bootloaders, but all of them have support to parse BLS fragments now.
>> So we can default to install BLS files on kernel installation and
>> drop
>> grubby.
>
> There is no BLS support in u-boot, how is it planned to support ARM
> systems? it is not an issue for aarch64 systems using u-boot as they
> use the efi implementation and load grub2, that is not true on 32 bit
> arm at all.
>

I talked with Peter Robinson (on cc) and the plan is to do the same
for ARMv7 in Fedora 29.

> Dennis
>

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/UA6DK5Q23H3MVPM5F526DL5RPFT3M3CT/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

[Javier Martinez Canillas]

On Tue, Jun 26, 2018 at 10:38 PM, Javier Martinez Canillas
 wrote:
> On Tue, Jun 26, 2018 at 5:36 PM, Javier Martinez Canillas
>  wrote:
>> On Tue, Jun 26, 2018 at 4:33 PM, Peter Jones  wrote:
>
> [snip]
>
>>>
>>>> It attempts to sort using the id field, and if this isn't defined
>>>> other fields are used as fallback in the following order: version,
>>>> title, linux.
>>>
>>> Yeah, so maybe we want to make that:
>>>
>>>   version, id, , title, linux
>>>
>>
>> Yes, although it probably should be:
>>
>> , id, version, title, linux
>>
>
> Ok, I noticed that said something stupid here. The filename is the
> only thing expected to be unique so there's no point to use anything
> as a fallback for it. So either we should only use the file name to
> sort or it should be the last thing used.
>
> I propose to just do: version, 
>
> and use the config file name (without the .conf) as the grub2 menu
> entry id and drop the id field since it won't be needed anymore.
>
> That way it will cover ostree case (version has precedence over
> filename) and all bootloaders will use the file name to sort.
> Alternatively we can just use the file name and change ostree to use
> the version instead of the index as its trailing number.
>

I've discussed with Colin and he agreed for ostree to use
ostree-$version-$ID-$VARIANT_ID.conf as filename, I've made that
change in [0].

That means that now both Fedora CoreOS and classic Fedora will have
BLS filenames that can be used as sort criteria. So I've proposed a
change [1] for our grub2 to just search by the BLS filename and use it
as the menu entry id, dropping the separate id field as suggested by
Zbyszek.

[0]: https://github.com/ostreedev/ostree/pull/1654
[1]: https://github.com/rhboot/grub2/pull/18

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/3EPPTSYQWZUYVG3TI5LF22HHRLL6WJGT/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

[Javier Martinez Canillas]

On Tue, Jun 26, 2018 at 5:36 PM, Javier Martinez Canillas
 wrote:
> On Tue, Jun 26, 2018 at 4:33 PM, Peter Jones  wrote:

[snip]

>>
>>> It attempts to sort using the id field, and if this isn't defined
>>> other fields are used as fallback in the following order: version,
>>> title, linux.
>>
>> Yeah, so maybe we want to make that:
>>
>>   version, id, , title, linux
>>
>
> Yes, although it probably should be:
>
> , id, version, title, linux
>

Ok, I noticed that said something stupid here. The filename is the
only thing expected to be unique so there's no point to use anything
as a fallback for it. So either we should only use the file name to
sort or it should be the last thing used.

I propose to just do: version, 

and use the config file name (without the .conf) as the grub2 menu
entry id and drop the id field since it won't be needed anymore.

That way it will cover ostree case (version has precedence over
filename) and all bootloaders will use the file name to sort.
Alternatively we can just use the file name and change ostree to use
the version instead of the index as its trailing number.

The advantage of the latter is that ostree will also work with
Petitboot and zipl (I don't know if that's a goal).

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/GYMHGRS5P3M52ZDHGD6VZHZXBWCOEZER/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

[Javier Martinez Canillas]

On Tue, Jun 26, 2018 at 4:33 PM, Peter Jones  wrote:
> On Tue, Jun 26, 2018 at 03:46:59PM +0200, Javier Martinez Canillas wrote:
>> > That raises two questions:
>> > 1. Why isn't just the bls-snippet filename used as the key? It's
>> >necessarily unique and should be usable for the purpose of uniquely
>> >identifying the boot entry without creating a separate field.
>>
>> I'll let Peter answer this question since he wrote the grub2
>> implementation. Doing this will be pretty trivial, but in that case we
>> should also sort using the filenames if the id field isn't defined.
>
> I don't know that I have a specific reason in mind when I wrote that bit
> of code.  Most likely the real answer is: because when I realized we
> have to sort them, I was writing parser code not directory iterating
> code.
>
> Way back when the config files were on VFAT on UEFI machines, it
> also meant I didn't have to even worry about the (entirely likely)
> situation arising where I have to re-implement this with some other API
> where I don't know if I'm going to get long filenames or not.
>
> For me, having it in the file seems more natural while I'm staring at
> the file with my eyes, since that's where I'm reading other values used
> in the same context.  But I see the point being made as well.  Maybe the
> thing to do here is prefer "id" if it's in the file, but use the
> filename if it isn't, and to disambiguate collisions.
>
>> This is already the case for Petitboot since the maintainer asked for
>> the filename to be used as the key as well. For zipl the key is the
>> version field because currently the kernel version is used as the
>> zipl.conf section name and the maintainer wanted to keep the same
>> behavior with BLS.
>
> Does this effectively mean we just have a different resolution order
> there?  Like are we comparing id and linux if version collides, or just
> not using them?
>

Sorry, I should had been more clear on this point. Both Petitboot and
zipl sort the entries using the filename and the strverscmp(3)
function. The fields are not used for sorting there since the
filenames should be unique.

The difference I mentioned was that Petitboot uses the filename as the
Petitboot boot entry ID and zipl uses the version field as the zipl
boot entry ID. For Petitboot there can't be duplicates since the
filenames are used, for zipl it will fail to parse the BLS if there
are two snippets with the same version, but that's already the case
without BLS if two zipl boot sections have the same name, so we won't
change that behavior.

>> Sorting using the filename won't work with ostree though. The BLS
>> snippets are named ostree-$ID-$VARIANT_ID-$index.conf but the entries
>> are sorted using the version field, and the index is the inverse of
>> the version.
>>
>> One option is to change ostree so the BLS snippets are named
>> ostree-$ID-$VARIANT_ID-$version.conf, but I don't know if there's a
>> reason to have the index there. Another option is to give more
>> precedence to the version field and only sort using the filenames if
>> neither the id nor the version fields are defined.
>>
>> > 2. Why is "id" supposed to be sortable? What sorting would grub2 do
>> >with it?
>>
>> It's sortable so grub2 can display the boot menu entries in the
>> correct order. This is also answered in the Changes page:
>>
>> id - provides us with a sort key, sorted using rpm's version
>> comparison function. Generally of the form:
>>  fedora-20180530145228-4.16.13-300.fc28.x86_64
>
> Put another way: if you don't have a saved_entry set in grub.cfg, or if
> it's saved_entry=0, if we don't have a sort field then you're going to
> get whatever comes out of your filesystem's directory storage algorithm
> first.  Sometimes that's a linked-list that may or may not be sorted,
> sometimes it's a b+tree, sometimes it's a hash bucket.  It's kind of
> hard to think anyone wants that behavior.
>
>> It attempts to sort using the id field, and if this isn't defined
>> other fields are used as fallback in the following order: version,
>> title, linux.
>
> Yeah, so maybe we want to make that:
>
>   version, id, , title, linux
>

Yes, although it probably should be:

, id, version, title, linux

That way all bootloaders will default to the filename to sort the
entries. That only leaves us with ostree since its BLS filenames can't
be used for sorting.

Colin, do you think that it may be feasible to change the ostree BLS
filenames so the trailing number is the version instead of the index?

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/P24M437GSWOF2EOSEMPN43CYGXLYOWP3/

Re: F29 System Wide Change: Make BootLoaderSpec the default ['id' field]

[Javier Martinez Canillas]

On Mon, Jun 25, 2018 at 6:14 PM, Zbigniew Jędrzejewski-Szmek
 wrote:
> On Mon, Jun 18, 2018 at 04:40:41PM +0200, Ondřej Lysoněk wrote:
>> On 14.6.2018 12:06, Jan Kurik wrote:
>> I noticed the official spec defines a field named "machine-id". AFAICS,
>> GRUB2 doesn't implement that option, but it supports a field named "id".
>> Are these used for the same thing? If they are, why are they named
>> differently?
>
> This questions wasn't addressed yet afaics.
>

The reason why are named differently is that the BLS isn't generated
on the machine but at kernel build time and shipped in the kernel
package. So there's a need for a machine independent id. The ostree
BLS snippets also don't define a machine-id BTW, but I don't know if
there's a reason for that.

> The partial answer is that "id" serves a different purpose to "machine-id":
> - "machine-id" is used to specify the machine for which the entry was 
> installed
> - "id" is used by grub2 as a unique identifier usable for saving entries
>

Yes, the "Differences from BootLoaderSpec" section in the Changes page [0] says:

id is also used for menuentry's --id parameter, so you can use it in saved_entry

> That raises two questions:
> 1. Why isn't just the bls-snippet filename used as the key? It's
>necessarily unique and should be usable for the purpose of uniquely
>identifying the boot entry without creating a separate field.

I'll let Peter answer this question since he wrote the grub2
implementation. Doing this will be pretty trivial, but in that case we
should also sort using the filenames if the id field isn't defined.

This is already the case for Petitboot since the maintainer asked for
the filename to be used as the key as well. For zipl the key is the
version field because currently the kernel version is used as the
zipl.conf section name and the maintainer wanted to keep the same
behavior with BLS.

Sorting using the filename won't work with ostree though. The BLS
snippets are named ostree-$ID-$VARIANT_ID-$index.conf but the entries
are sorted using the version field, and the index is the inverse of
the version.

One option is to change ostree so the BLS snippets are named
ostree-$ID-$VARIANT_ID-$version.conf, but I don't know if there's a
reason to have the index there. Another option is to give more
precedence to the version field and only sort using the filenames if
neither the id nor the version fields are defined.

> 2. Why is "id" supposed to be sortable? What sorting would grub2 do
>with it?
>

It's sortable so grub2 can display the boot menu entries in the
correct order. This is also answered in the Changes page:

id - provides us with a sort key, sorted using rpm's version
comparison function. Generally of the form:
 fedora-20180530145228-4.16.13-300.fc28.x86_64

It attempts to sort using the id field, and if this isn't defined
other fields are used as fallback in the following order: version,
title, linux.

> Zbyszek

[0]: https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/3GAGU3MV2LNLFMJ37DDW2IRKTZNE2GI7/

Re: F29 System Wide Change: Make BootLoaderSpec the default

[Javier Martinez Canillas]

On Thu, Jun 21, 2018 at 11:19 PM, Chris Murphy  wrote:

[snip]

>>
>>> OK anyway, I don't see broad BLS consensus forming yet, but I do see
>>> two items that aren't controversial and could move forward as part of
>>> this feature proposal:
>>>
>>> a. Consistent $BOOT/loader/entries for UEFI and BIOS where $BOOT is
>>> the ESP on UEFI, and $BOOT is "other" on BIOS (most likely the
>>> existing /boot ext4 volume currently used). i.e. do not put
>>> /loader/entries in /EFI/fedora
>>
>> By "consistent", do you mean that both EFI and BIOS boot loaders will
>> reference the same entry files? I like that.
>
> Yes.
>
>> However, I don't like the entries existing on ESP mostly because of the
>> use case of md-RAID for /boot referenced in another thread. I think it
>> would be best to just put the GRUB EFI file on ESP and put the rest of
>> the bulk GRUB stuff in its utility partition (which may be RAID-ed).
>
> Right. The config + kernel + initramfs on traditional /boot can make
> use of software raid, depending on static one time proper
> configuration of each member drive's ESPs and now the ESPs never need
> to be touched and thus not sync'd.
>
> Whereas constantly changing the ESP, means we need some way to
> establish a master and rsync to the extras.
>

So the consensus seems to be to have the BLS fragments in
$BOOT/loader/entries even on EFI, where $BOOT is the boot partition
mounted on /boot. That will give us the following advantages as
mentioned in this thread:

1. The BLS will not be stored in vfat, so ostree could keep using
symlinks to do atomic swap of its /boot/loader/entries
2. The ESP won't be modified on kernels install / removal, that will
make it easier for software RAID.
3. There will be consistency on where the BLS fragments are installed
regardless of the firmware interface (EFI, BIOS, OPAL on ppc64le and
zipl on s390x will all use /boot/loader/entries).

I've updated the wiki page to reflect this and we will also change the
implementation.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/B3SMCSWFFEYIJIWPESY27MB6AMH3DH52/

Re: F29 System Wide Change: Make BootLoaderSpec the default

[Javier Martinez Canillas]

On Mon, Jun 18, 2018 at 11:54 PM, Tom Hughes  wrote:
> On 18/06/18 18:15, Peter Jones wrote:
>
>> That's true - though we actually shipped nearly all of the code to
>> implement this stuff f28, minus some parts of the upgrade story and the
>> anaconda bits to enable it by default.  You can go run
>> grub2-switch-to-blscfg today, and it will work.  I hope :)
>
>
> Unless you have /boot on your root partition like this machine
> seems to have for some reason... Then it breaks because the loader
> fragments use /vmlinuz... rather than /boot/vmlinuz etc.
>

Yes, /boot not being a mount point was an issue (and also /boot being
a btrfs subvolme) but it got fixed [0] a couple of weeks ago. Now the
20-grub.install kernel-install script uses grub2-mkrelpath to get the
relative path of the kernel and initramfs images, which is the same
that's done by the /etc/grub.d/10_linux script. It's just that the
grub2 update didn't made to F28 yet.

[0]: 
https://src.fedoraproject.org/rpms/grub2/c/db7cf3a089075af0f4a8b955af508aea3893465a

> Tom
>

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/UGUSZIBOLV4XUZPKZ3ZTYZ2HJO36KPES/

Re: F29 System Wide Change: Make BootLoaderSpec the default

[Javier Martinez Canillas]

On Thu, Jun 14, 2018 at 10:20 PM, Chris Murphy  wrote:
> On Thu, Jun 14, 2018 at 12:51 PM, Adam Williamson
>  wrote:
>> On Thu, 2018-06-14 at 12:06 +0200, Jan Kurik wrote:
>>> == Scope ==
>>> * Proposal owners:
>>> ** Generate BLS snippets at kernel build time and ship in the kernel 
>>> packages.
>>> ** Make kernel-install scripts to copy the BLS, kernel and initramfs
>>> images and do any architecture specific task.
>>> ** Make GRUB 2, zipl and Petitboot bootloaders to populate their boot
>>> menu entries from the information in BLS files.
>>> ** Have a grubby wrapper for backward compatbility that manipulates BLS 
>>> files.
>>> ** Modify packages that use grubby to instead install BLS fragments
>>> (memtest86+, tuned).
>>> ** Make sure this is all properly documented in release-notes, etc.
>>
>> What exactly is the plan for upgrades, here?
>
>
> "users upgrading from a previous version of Fedora will keep the old
> behaviour. "
> https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault#Upgrade.2Fcompatibility_impact
>
> I'm on the fence whether I think it's better to support two bootloader
> configurations, or compel upgrades to use the new method at some point
> and when, rather than having a community with multiple personalities
> confusion.
>
> The cited BLS spec is the original one, not the more thoroughly
> discussed and thought through variant by Matthew Garrett [1] some
> years ago.
>
> What are we getting from the cited spec? All of it? Are there
> exceptions? Where are the exceptions written?
>

The BootLoaderSpec document was cited mostly for context in case
someone was not familiar with the BLS concept. We support multiple
architectures in Fedora and not all of them use UEFI (e.g: ppc64le and
s390x), so we used the spec as a reference rather than following it
verbatim.

The value I think is in having the same file format for all supported
architectures by Fedora, so we can make tools able to parse and manage
them without needing to care about different file formats per
bootloader. It's true that we don't need BLS for that, since we could
do the same than other distros and use the grub config file for
everything (for example SuSE chain-loads zipl with grub-emu on s390x
so they can use a grub.cfg instead of a zipl.conf there). But the
advantage of BLS is that allows to have system configuration changes
as atomic operations that don't require parsing a monolithic config
file.

> The cited BLS spec requires $BOOT be VFAT, are we doing that?
>

Yes for EFI systems but no otherwise. On EFI the BLS snippets are in
/boot/efi/EFI/fedora/loader/entries and on non-EFI systems are in
/boot/loader/entries.

> The cited BLS spec requires kernels and initramfs go on $BOOT, are we
> doing that?
>

No, the kernel and initramfs images are in /boot. By default in Fedora
we keep 3 kernel + initramfs images (installonly_limit=3 in
/etc/dnf/dnf.conf) + the rescue images (only the rescue initramfs is
~500MB). So as you said it's not realistic to have all the images in
the ESP.

> Are we going to stop doing the diabolical (and widespread) nested
> mount nonsense, e.g. /boot/efi? Are we getting rid of the persistent
> mounting of these volumes in favor of mounting/unmounting dynamically
> only by the programs that are authorized to make changes to these
> volumes?
>

That remains the same for now, the proposed change is only about
populating the boot menu entries from BLS files instead of the
bootloader configuration file.

> If there's no room on the EFI System partition for all of this, will
> we following bullets 2 and 5 of the BLS spec under "The installer

No, $BOOT is always the ESP where GRUB 2 is installed.

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/EZKEX565RVCWKIBJLWSKZWR7HJUWBZNO/

Re: GCL and SELinux: help requested

[Javier Martinez Canillas]

On Thu, Nov 23, 2017 at 10:21 AM, Lukas Vrabec <lvra...@redhat.com> wrote:
> On 11/23/2017 10:17 AM, Javier Martinez Canillas wrote:
>>
>> Hello,
>>
>> On Fri, Oct 20, 2017 at 2:12 PM, Lukas Vrabec <lvra...@redhat.com> wrote:
>>
>> [snip]
>>
>>>
>>> Hello community,
>>> We, as Red Hat SELinux team, apologise for recent delays with our answers
>>> to
>>> your requests and questions related to SELinux. We have been quite busy
>>> last
>>> couple of weeks so we decided to set a lower priority for Fedora work. We
>>> already responded and resolved what was needed and we are ready to react
>>> more flexibly in the future.
>>>
>>> Note: If you are interested in writing custom SELinux policy for your
>>> package, you can follow the
>>> https://fedoraproject.org/wiki/SELinux/IndependentPolicy documentation on
>>> wiki.
>>>
>>
>> To update the tpm2-abrmd [0] package to the latest version, I need to
>> add a SELinux policy due recent upstream changes in the upstream
>> project. But after reading the documents referred in this thread, is
>> still not clear to me if the preferred method nowadays is to propose
>> adding the SELinux policy to the system wide selinux-policy package or
>> to ship a custom SELinux security module for the package.
>>
>
>
> Hi,
>
> SELinux policy for this project is already existing? If not I can help you

It doesn't exist in Fedora yet, so currently the tpm2-abrmd daemon
runs in an unconfined domain. A policy module was added to the project
repo [0] though, but I don't know how correct it is (I'm not a SELinux
expert).

The specific problem is that now the daemon uses sockets to
communicate with a library, but the dbus-daemon in the system bus
isn't allowed to read/write to sockets created by processes in an
unconfined domain. It used pipes before and that was allowed.

> with creating policy for this project. From SELinux team it's prefered to

No worries, I think I can sort it out using the SELinux policy in the
tpm2-abrmd repo as a base. I just asked since wasn't clear to me which
approach was preferred.

> add policy to your package. Guidelines how to do that is in progress to be
> part of rpm packaging guidelines.
>

Awesome, I'll re-read [1] then and ad d the policy to the package.
Thanks a lot for your help!

> Lukas.
>

[0]: 
https://github.com/intel/tpm2-abrmd/pull/205/commits/3621742344534a5d0d5d255d1d5bc698f3d39a57
[1]: https://fedoraproject.org/wiki/SELinux/IndependentPolicy

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: GCL and SELinux: help requested

[Javier Martinez Canillas]

Hello,

On Fri, Oct 20, 2017 at 2:12 PM, Lukas Vrabec  wrote:

[snip]

>
> Hello community,
> We, as Red Hat SELinux team, apologise for recent delays with our answers to
> your requests and questions related to SELinux. We have been quite busy last
> couple of weeks so we decided to set a lower priority for Fedora work. We
> already responded and resolved what was needed and we are ready to react
> more flexibly in the future.
>
> Note: If you are interested in writing custom SELinux policy for your
> package, you can follow the
> https://fedoraproject.org/wiki/SELinux/IndependentPolicy documentation on
> wiki.
>

To update the tpm2-abrmd [0] package to the latest version, I need to
add a SELinux policy due recent upstream changes in the upstream
project. But after reading the documents referred in this thread, is
still not clear to me if the preferred method nowadays is to propose
adding the SELinux policy to the system wide selinux-policy package or
to ship a custom SELinux security module for the package.

> Regards,
> Lukas
>

[0]: https://github.com/intel/tpm2-abrmd

Best regards,
Javier
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org