Re: F39 proposal: mkosi-initrd (Self-Contained Change proposal)
I think this is a great idea by the way, funnily enough I had thought the same thoughts numerous times over the last 18 months. "Why does dracut do it's own job scheduling for generating initramfs, when systemd has all this job scheduling functionality already?" but now someone has actually made those thoughts come to real life :) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: F39 proposal: mkosi-initrd (Self-Contained Change proposal)
On Wed, Apr 26, 2023 at 12:17:22AM +0200, Dan Čermák wrote: > Would it be possible to build bootable images using mkosi-initrd in > koji? Then these could be booted in openQA directly and catch simple > regressions like unbootable images quite easily. If I understand correctly, you are suggesting building the initrd image as a stand-alone job similarly to how we produce installation images and other deliverables. In theory … maybe? But haven't really thought about it in this way, but: - for delivery to users we actually want the result to be packaged as rpm, so if we built a standalone image as a task in koji, then we'd need another job to build an rpm out of this. This seems quite complicated. - to actually test the initrd, it'd not enough to "boot it" on its own. To say that it works, we need to test switch-root to the host system. So it seems more natural to me to build an rpm, and then do the testing starting from this rpm. Users would consume the initrd as an rpm, so we want to test it the same way. E.g. in openqa, install a system with the initrd rpm, and check if it boots. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: F39 proposal: mkosi-initrd (Self-Contained Change proposal)
Hi, > Would it be possible to build bootable images using mkosi-initrd in > koji? Then these could be booted in openQA directly and catch simple > regressions like unbootable images quite easily. You might want have a look at kernel-uki-virt.rpm, which ships an UKI you can boot directly (that is built using dracut not mkosi-initrd though). take care, Gerd ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: F39 proposal: mkosi-initrd (Self-Contained Change proposal)
Ben Cotton writes: > https://fedoraproject.org/wiki/Changes/mkosi-initrd > > This document represents a proposed Change. As part of the Changes > process, proposals are publicly announced in order to receive > community feedback. This proposal will only be implemented if approved > by the Fedora Engineering Steering Committee. > > > == Summary == > > `mkosi-initrd` is an alternative builder for initrds. > It will be packaged in Fedora, so that users can use it to build > initrds locally. > A `kernel-install` plugin will be provided to build the initrd when a > kernel package is installed. > As a stretch goal, initrds will be build in koji and delivered via rpm > packages. > As a further stretch goal, pre-built initrds will be used in Unified > Kernel Images that can be delivered via rpm packages. > > Only a subset of installation types will be supported. > > == Owner == > * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]] > * Name: [[User:lnykryn| Lukáš Nykrýn ]] > * Name: [[User:Daandemeyer| Daan De Meyer]] > * Email: zbyszek - at - in.waw.pl, lnykryn - at - redhat.com, > daandemeyer - at - fb.com > > > > == Detailed Description == > The process by which we create initrds is complicated and inefficient. > Initrds contain duplicate functionality and require a lot of maintainer > effort. > The aim of this proposal is to introduce a vastly simplified mechanism > of initrd creation and simplified initrd contents. > > The `mkosi-initrd` project is a set of config files for `mkosi`. > `mkosi` is a program to build operating system images from system packages. > An initrd is built by invoking `mkosi` with the config provided by > `mkosi-initrd`. > > Instead of building initrds by scraping the file system and figuring > out dependencies again, > existing packages and normal package installation via `dnf`/`rpm` is > used to populate the initrd. > This also means that the package manager is responsible for satisfying > dependencies. > At runtime, `systemd` is responsible for setting up the execution > environment and invoking programs. > > Currently, initrds built in this way are bigger than initrds built by dracut. > They also have limited functionality: > many common types of systems work just fine, but more exotic > configurations are not supported. > See [[#Scope|Scope]] below for a list of known good/bad features. > > The goal of this change is to provide an ''alternative'' mechanism. > If the feedback is positive, we may consider using initrds built with > `mkosi-initrd` as default in certain scenarios. > There are no plans to remove `dracut` in the foreseeable future. > This means that for any case not supported or not working well, > `dracut` remains a natural fallback. > In this way this change is similar to > [[Changes/Unified_Kernel_Support_Phase_1]], > as it provides a preview of a new technology as an alternative to the > current established approach. > > == Feedback == > > > == Benefit to Fedora == > Current initrd generation with `dracut` is showing its age. > As upstream packages evolve, > repeating the dependency resolution in `dracut` is a constant drain of > maintainer time. > Our `dracut` initrds are already using `systemd`. > But `systemd` is constantly evolving and adding new functionality > related to early boot: > decryption of disks, access to secrets, new configuration mechanisms, > state checks and boot counting. > More and more, `dracut` runtime scripting is a thin wrapper around `systemd`. > We have two job queues: the `dracut` initqueue, and the `systemd` job queue. > This duplication makes everything harder, both during preparation and > at runtime, for little benefit. > > The design principle of the new approach is to remove duplicate functionality: > * package `Requires` replace dracut module dependency logic and > automatic installation of libraries based on `ldd` output > * `systemd` job management replaces the remainder of `dracut` runtime > * the environment in the initrd is just a normal linux system (albeit > on a temporary root fs) > * normal package contents replace special scripts crafted for the initrd > > Generally, the new scheme requires very little new stuff. > We reuse things that are already available (and used): > `dnf` and `rpm`, > packages for all stuff that is used in the initrd, > `systemd`, > special systemd units for the initrd. > > The new component is a mechanism that builds the initrd out of packages. > But it is a relatively simple step that requires very little maintenance. > The biggest part of the initial work is the creation of package lists > to install in the initrd, > and adjusting packages to to function correctly in the initrd and not > pull in unnecessary dependencies. > Afterwards, there might be occasional maintenance related to bugs or > package splits. > > Initrds built with `mkosi-initrd` should be fully reproducible (in the > sense of reproducible builds). > > The work done in packages has external benefits: > package minimization
F39 proposal: mkosi-initrd (Self-Contained Change proposal)
https://fedoraproject.org/wiki/Changes/mkosi-initrd This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == `mkosi-initrd` is an alternative builder for initrds. It will be packaged in Fedora, so that users can use it to build initrds locally. A `kernel-install` plugin will be provided to build the initrd when a kernel package is installed. As a stretch goal, initrds will be build in koji and delivered via rpm packages. As a further stretch goal, pre-built initrds will be used in Unified Kernel Images that can be delivered via rpm packages. Only a subset of installation types will be supported. == Owner == * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]] * Name: [[User:lnykryn| Lukáš Nykrýn ]] * Name: [[User:Daandemeyer| Daan De Meyer]] * Email: zbyszek - at - in.waw.pl, lnykryn - at - redhat.com, daandemeyer - at - fb.com == Detailed Description == The process by which we create initrds is complicated and inefficient. Initrds contain duplicate functionality and require a lot of maintainer effort. The aim of this proposal is to introduce a vastly simplified mechanism of initrd creation and simplified initrd contents. The `mkosi-initrd` project is a set of config files for `mkosi`. `mkosi` is a program to build operating system images from system packages. An initrd is built by invoking `mkosi` with the config provided by `mkosi-initrd`. Instead of building initrds by scraping the file system and figuring out dependencies again, existing packages and normal package installation via `dnf`/`rpm` is used to populate the initrd. This also means that the package manager is responsible for satisfying dependencies. At runtime, `systemd` is responsible for setting up the execution environment and invoking programs. Currently, initrds built in this way are bigger than initrds built by dracut. They also have limited functionality: many common types of systems work just fine, but more exotic configurations are not supported. See [[#Scope|Scope]] below for a list of known good/bad features. The goal of this change is to provide an ''alternative'' mechanism. If the feedback is positive, we may consider using initrds built with `mkosi-initrd` as default in certain scenarios. There are no plans to remove `dracut` in the foreseeable future. This means that for any case not supported or not working well, `dracut` remains a natural fallback. In this way this change is similar to [[Changes/Unified_Kernel_Support_Phase_1]], as it provides a preview of a new technology as an alternative to the current established approach. == Feedback == == Benefit to Fedora == Current initrd generation with `dracut` is showing its age. As upstream packages evolve, repeating the dependency resolution in `dracut` is a constant drain of maintainer time. Our `dracut` initrds are already using `systemd`. But `systemd` is constantly evolving and adding new functionality related to early boot: decryption of disks, access to secrets, new configuration mechanisms, state checks and boot counting. More and more, `dracut` runtime scripting is a thin wrapper around `systemd`. We have two job queues: the `dracut` initqueue, and the `systemd` job queue. This duplication makes everything harder, both during preparation and at runtime, for little benefit. The design principle of the new approach is to remove duplicate functionality: * package `Requires` replace dracut module dependency logic and automatic installation of libraries based on `ldd` output * `systemd` job management replaces the remainder of `dracut` runtime * the environment in the initrd is just a normal linux system (albeit on a temporary root fs) * normal package contents replace special scripts crafted for the initrd Generally, the new scheme requires very little new stuff. We reuse things that are already available (and used): `dnf` and `rpm`, packages for all stuff that is used in the initrd, `systemd`, special systemd units for the initrd. The new component is a mechanism that builds the initrd out of packages. But it is a relatively simple step that requires very little maintenance. The biggest part of the initial work is the creation of package lists to install in the initrd, and adjusting packages to to function correctly in the initrd and not pull in unnecessary dependencies. Afterwards, there might be occasional maintenance related to bugs or package splits. Initrds built with `mkosi-initrd` should be fully reproducible (in the sense of reproducible builds). The work done in packages has external benefits: package minimization automatically benefits other "small" installations. == Scope == * Proposal owners: ** package `mkosi-initrd` ([https://copr.fedorainfracloud.org/coprs/zbyszek/mkosi-initrd/builds/ copr], review-request
F39 proposal: mkosi-initrd (Self-Contained Change proposal)
https://fedoraproject.org/wiki/Changes/mkosi-initrd This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == `mkosi-initrd` is an alternative builder for initrds. It will be packaged in Fedora, so that users can use it to build initrds locally. A `kernel-install` plugin will be provided to build the initrd when a kernel package is installed. As a stretch goal, initrds will be build in koji and delivered via rpm packages. As a further stretch goal, pre-built initrds will be used in Unified Kernel Images that can be delivered via rpm packages. Only a subset of installation types will be supported. == Owner == * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]] * Name: [[User:lnykryn| Lukáš Nykrýn ]] * Name: [[User:Daandemeyer| Daan De Meyer]] * Email: zbyszek - at - in.waw.pl, lnykryn - at - redhat.com, daandemeyer - at - fb.com == Detailed Description == The process by which we create initrds is complicated and inefficient. Initrds contain duplicate functionality and require a lot of maintainer effort. The aim of this proposal is to introduce a vastly simplified mechanism of initrd creation and simplified initrd contents. The `mkosi-initrd` project is a set of config files for `mkosi`. `mkosi` is a program to build operating system images from system packages. An initrd is built by invoking `mkosi` with the config provided by `mkosi-initrd`. Instead of building initrds by scraping the file system and figuring out dependencies again, existing packages and normal package installation via `dnf`/`rpm` is used to populate the initrd. This also means that the package manager is responsible for satisfying dependencies. At runtime, `systemd` is responsible for setting up the execution environment and invoking programs. Currently, initrds built in this way are bigger than initrds built by dracut. They also have limited functionality: many common types of systems work just fine, but more exotic configurations are not supported. See [[#Scope|Scope]] below for a list of known good/bad features. The goal of this change is to provide an ''alternative'' mechanism. If the feedback is positive, we may consider using initrds built with `mkosi-initrd` as default in certain scenarios. There are no plans to remove `dracut` in the foreseeable future. This means that for any case not supported or not working well, `dracut` remains a natural fallback. In this way this change is similar to [[Changes/Unified_Kernel_Support_Phase_1]], as it provides a preview of a new technology as an alternative to the current established approach. == Feedback == == Benefit to Fedora == Current initrd generation with `dracut` is showing its age. As upstream packages evolve, repeating the dependency resolution in `dracut` is a constant drain of maintainer time. Our `dracut` initrds are already using `systemd`. But `systemd` is constantly evolving and adding new functionality related to early boot: decryption of disks, access to secrets, new configuration mechanisms, state checks and boot counting. More and more, `dracut` runtime scripting is a thin wrapper around `systemd`. We have two job queues: the `dracut` initqueue, and the `systemd` job queue. This duplication makes everything harder, both during preparation and at runtime, for little benefit. The design principle of the new approach is to remove duplicate functionality: * package `Requires` replace dracut module dependency logic and automatic installation of libraries based on `ldd` output * `systemd` job management replaces the remainder of `dracut` runtime * the environment in the initrd is just a normal linux system (albeit on a temporary root fs) * normal package contents replace special scripts crafted for the initrd Generally, the new scheme requires very little new stuff. We reuse things that are already available (and used): `dnf` and `rpm`, packages for all stuff that is used in the initrd, `systemd`, special systemd units for the initrd. The new component is a mechanism that builds the initrd out of packages. But it is a relatively simple step that requires very little maintenance. The biggest part of the initial work is the creation of package lists to install in the initrd, and adjusting packages to to function correctly in the initrd and not pull in unnecessary dependencies. Afterwards, there might be occasional maintenance related to bugs or package splits. Initrds built with `mkosi-initrd` should be fully reproducible (in the sense of reproducible builds). The work done in packages has external benefits: package minimization automatically benefits other "small" installations. == Scope == * Proposal owners: ** package `mkosi-initrd` ([https://copr.fedorainfracloud.org/coprs/zbyszek/mkosi-initrd/builds/ copr], review-request
