Re: Firecracker microVM manager
On Mon, Apr 24, 2023 at 4:10 PM Demi Marie Obenour wrote: > > On 4/24/23 08:33, Neal Gompa wrote: > > On Mon, Apr 24, 2023 at 4:19 AM Peter Robinson wrote: > >> > There is no problem technically; the Copr repo[2] is building > Firecracker RPMs with musl. Maintainers of both Rust and musl seemed > to be against it in Fedora. From this thread: > >>> Why does Fedora not want to ship Firecracker statically linked to musl? > >>> That is the supported and tested configuration upstream. Using glibc > >>> or dynamic linking is not supported for production use. > >> > >> Because glibc is Fedora's supported libc implementation and supporting > >> two different implementations at the same time is complex > > > > And importantly, as the musl maintainer, I recommended against it. We > > should take the opportunity to engage with upstream to fully support > > glibc instead. > > Can they support glibc without either taking on a huge maintenance burden > or weakening the sandbox? Whatever the community wants to support will be supported. That's how open source projects work. Previously it seemed like nobody was interested in Firecracker, so nothing happened. If there's interest now, things will change. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On 4/24/23 08:33, Neal Gompa wrote: > On Mon, Apr 24, 2023 at 4:19 AM Peter Robinson wrote: >> There is no problem technically; the Copr repo[2] is building Firecracker RPMs with musl. Maintainers of both Rust and musl seemed to be against it in Fedora. From this thread: >>> Why does Fedora not want to ship Firecracker statically linked to musl? >>> That is the supported and tested configuration upstream. Using glibc >>> or dynamic linking is not supported for production use. >> >> Because glibc is Fedora's supported libc implementation and supporting >> two different implementations at the same time is complex > > And importantly, as the musl maintainer, I recommended against it. We > should take the opportunity to engage with upstream to fully support > glibc instead. Can they support glibc without either taking on a huge maintenance burden or weakening the sandbox? -- Sincerely, Demi Marie Obenour (she/her/hers) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Mon, Apr 24, 2023 at 4:19 AM Peter Robinson wrote: > > > > There is no problem technically; the Copr repo[2] is building > > > Firecracker RPMs with musl. Maintainers of both Rust and musl seemed > > > to be against it in Fedora. From this thread: > > Why does Fedora not want to ship Firecracker statically linked to musl? > > That is the supported and tested configuration upstream. Using glibc > > or dynamic linking is not supported for production use. > > Because glibc is Fedora's supported libc implementation and supporting > two different implementations at the same time is complex And importantly, as the musl maintainer, I recommended against it. We should take the opportunity to engage with upstream to fully support glibc instead. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
> > There is no problem technically; the Copr repo[2] is building > > Firecracker RPMs with musl. Maintainers of both Rust and musl seemed > > to be against it in Fedora. From this thread: > Why does Fedora not want to ship Firecracker statically linked to musl? > That is the supported and tested configuration upstream. Using glibc > or dynamic linking is not supported for production use. Because glibc is Fedora's supported libc implementation and supporting two different implementations at the same time is complex ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Sat, Apr 22, 2023 at 10:48 AM Matthew Miller wrote: > On Sat, Apr 22, 2023 at 10:13:31AM -0400, David Michael wrote: > > > Would it be possible to add a warning to this effect? Without any form > > > of sandboxing Firecracker is not suitable for production use. > > Where would such a warning be placed? The sandboxing is done by a > > standalone program[0] which is not built in the package, so it should > > be clear that it isn't available. > > Silly question: would it make any sense at all to use _podman_ as a > replacement for firecracker's jailer? That would certainly be convenient and support use cases like with krun, but I'd need to do more research around producing a compatible podman runtime. I've seen a few container runtime integration projects for Firecracker, some abandoned. Maybe pursuing the Kata runtime would be the best fit for Fedora since it's already packaged? (Although I see Kata still wants Firecracker's jailer program, and I don't know if it's optional or not.) Thanks. David ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On 4/22/23 10:13, David Michael wrote: > On Fri, Apr 21, 2023 at 10:02 PM Demi Marie Obenour > wrote: >> On 4/21/23 11:13, David Michael wrote: >>> Hi, >>> >>> Following up on this, Firecracker has been accepted and submitted to >>> Fedora. Thanks to Fabio for all of the Rust reviews. >>> >>> F37 https://bodhi.fedoraproject.org/updates/FEDORA-2023-dca8124d3b >>> F38 https://bodhi.fedoraproject.org/updates/FEDORA-2023-edcbcf18e0 >>> >>> Some quick comments on the TODO from the original e-mail: >>> >>> On Sat, Mar 4, 2023 at 12:40 PM David Michael wrote: - The musl package adds /usr paths for compatibility with the compiler --sysroot option. - The rust compiler adds musl target subpackages. >>> >>> Targeting musl was dropped after the initial discussion, so there is >>> no sandbox or seccomp filter in Fedora until that can be implemented >>> with dynamic linking glibc upstream. I'll keep the Copr repo[0] >>> active for a while to provide musl builds. >> >> Would it be possible to add a warning to this effect? Without any form >> of sandboxing Firecracker is not suitable for production use. > > Where would such a warning be placed? The sandboxing is done by a > standalone program[0] which is not built in the package, so it should > be clear that it isn't available. Package description perhaps? >> Does the >> sandboxed portion of Firecracker do anything that would require an NSS >> (name service switch) lookup, such as DNS or username resolution? > > I don't think NSS lookups are an issue since the program takes > numerical UID/GID values as command-line arguments. The main breaking > issue with the jailer is that it requires Firecracker to be a single > static binary[1] (which is the musl target's default output upstream). > Their documentation also says glibc isn't supported, but I haven't > tried making a static glibc binary to see what fails. The seccomp > filter being unimplemented for glibc is a separate issue from the > jailer. glibc does not support static linking well. >> If >> not, then I don’t see why musl (which Fedora already ships!) would be a >> problem. > > There is no problem technically; the Copr repo[2] is building > Firecracker RPMs with musl. Maintainers of both Rust and musl seemed > to be against it in Fedora. From this thread: Why does Fedora not want to ship Firecracker statically linked to musl? That is the supported and tested configuration upstream. Using glibc or dynamic linking is not supported for production use. -- Sincerely, Demi Marie Obenour (she/her/hers) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Sat, Apr 22, 2023 at 10:13:31AM -0400, David Michael wrote: > > Would it be possible to add a warning to this effect? Without any form > > of sandboxing Firecracker is not suitable for production use. > Where would such a warning be placed? The sandboxing is done by a > standalone program[0] which is not built in the package, so it should > be clear that it isn't available. Silly question: would it make any sense at all to use _podman_ as a replacement for firecracker's jailer? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Fri, Apr 21, 2023 at 10:02 PM Demi Marie Obenour wrote: > On 4/21/23 11:13, David Michael wrote: > > Hi, > > > > Following up on this, Firecracker has been accepted and submitted to > > Fedora. Thanks to Fabio for all of the Rust reviews. > > > > F37 https://bodhi.fedoraproject.org/updates/FEDORA-2023-dca8124d3b > > F38 https://bodhi.fedoraproject.org/updates/FEDORA-2023-edcbcf18e0 > > > > Some quick comments on the TODO from the original e-mail: > > > > On Sat, Mar 4, 2023 at 12:40 PM David Michael wrote: > >> - The musl package adds /usr paths for compatibility with the compiler > >> --sysroot option. > >> - The rust compiler adds musl target subpackages. > > > > Targeting musl was dropped after the initial discussion, so there is > > no sandbox or seccomp filter in Fedora until that can be implemented > > with dynamic linking glibc upstream. I'll keep the Copr repo[0] > > active for a while to provide musl builds. > > Would it be possible to add a warning to this effect? Without any form > of sandboxing Firecracker is not suitable for production use. Where would such a warning be placed? The sandboxing is done by a standalone program[0] which is not built in the package, so it should be clear that it isn't available. > Does the > sandboxed portion of Firecracker do anything that would require an NSS > (name service switch) lookup, such as DNS or username resolution? I don't think NSS lookups are an issue since the program takes numerical UID/GID values as command-line arguments. The main breaking issue with the jailer is that it requires Firecracker to be a single static binary[1] (which is the musl target's default output upstream). Their documentation also says glibc isn't supported, but I haven't tried making a static glibc binary to see what fails. The seccomp filter being unimplemented for glibc is a separate issue from the jailer. > If > not, then I don’t see why musl (which Fedora already ships!) would be a > problem. There is no problem technically; the Copr repo[2] is building Firecracker RPMs with musl. Maintainers of both Rust and musl seemed to be against it in Fedora. From this thread: On Sat, Mar 4, 2023 at 5:51 PM Neal Gompa wrote: > As the musl package maintainer, I don't particularly want Fedora > packages depending on it unless they absolutely have to. > [...] > As in musl is statically linked into the binaries? Or the Rust code is > statically linked. The former is not okay, From the Firecracker package review: On Fri, Apr 7, 2023 at 11:27 AM wrote: > 2. Building for the *-musl Rust targets is not going to be supported in Fedora For what it's worth, Arch and openSUSE seem to be building with glibc and so are in the same position with this package. Without Rust's musl target, the only path forward is to try to improve glibc support upstream. Thanks. David [0] https://github.com/firecracker-microvm/firecracker/blob/main/docs/jailer.md [1] https://github.com/firecracker-microvm/firecracker/blob/v1.3.1/src/jailer/src/env.rs#L380 [2] https://copr.fedorainfracloud.org/coprs/dm0/Firecracker ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On 4/21/23 11:13, David Michael wrote: > Hi, > > Following up on this, Firecracker has been accepted and submitted to > Fedora. Thanks to Fabio for all of the Rust reviews. > > F37 https://bodhi.fedoraproject.org/updates/FEDORA-2023-dca8124d3b > F38 https://bodhi.fedoraproject.org/updates/FEDORA-2023-edcbcf18e0 > > Some quick comments on the TODO from the original e-mail: > > On Sat, Mar 4, 2023 at 12:40 PM David Michael wrote: >> - The musl package adds /usr paths for compatibility with the compiler >> --sysroot option. >> - The rust compiler adds musl target subpackages. > > Targeting musl was dropped after the initial discussion, so there is > no sandbox or seccomp filter in Fedora until that can be implemented > with dynamic linking glibc upstream. I'll keep the Copr repo[0] > active for a while to provide musl builds. Would it be possible to add a warning to this effect? Without any form of sandboxing Firecracker is not suitable for production use. Does the sandboxed portion of Firecracker do anything that would require an NSS (name service switch) lookup, such as DNS or username resolution? If not, then I don’t see why musl (which Fedora already ships!) would be a problem. If it does, could the lookups be moved to a separate process? -- Sincerely, Demi Marie Obenour (she/her/hers) OpenPGP_0xB288B55FFF9C22C1.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
Hi, Following up on this, Firecracker has been accepted and submitted to Fedora. Thanks to Fabio for all of the Rust reviews. F37 https://bodhi.fedoraproject.org/updates/FEDORA-2023-dca8124d3b F38 https://bodhi.fedoraproject.org/updates/FEDORA-2023-edcbcf18e0 Some quick comments on the TODO from the original e-mail: On Sat, Mar 4, 2023 at 12:40 PM David Michael wrote: > - The musl package adds /usr paths for compatibility with the compiler > --sysroot option. > - The rust compiler adds musl target subpackages. Targeting musl was dropped after the initial discussion, so there is no sandbox or seccomp filter in Fedora until that can be implemented with dynamic linking glibc upstream. I'll keep the Copr repo[0] active for a while to provide musl builds. > - The kernel must set CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y to be usable as > a guest. I fixed this in rawhide during the 6.3 cycle, so it should eventually trickle down into stable. > - About two dozen Rust crates must be added to Fedora (but a handful are > just new versions of existing packages). All crate dependencies have been added to Fedora 37+. If anyone would like to test the Bodhi updates, there are some example commands to start a VM on the Copr repo. Thanks. David [0] https://copr.fedorainfracloud.org/coprs/dm0/Firecracker ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Sat, Mar 18, 2023, 03:37 Smith, Stewart via devel < devel@lists.fedoraproject.org> wrote: > On Mar 5, 2023, at 10:19 AM, Kevin Kofler via devel < > devel@lists.fedoraproject.org> wrote: > > > > > > David Michael wrote: > >> - Firecracker can be built with Fedora's libc (glibc), but it is > >> officially unsupported upstream[3]. Functionality would be harmed by > >> not using musl, e.g. seccomp filters are not used. > > > > Upstream's refusal to write seccomp filters that work with glibc should > be a > > red flag. It is definitely possible to sandbox glibc applications with > > seccomp, e.g., Chromium does it. It does need updates/fixes to the > seccomp > > rules with almost every new version of glibc, but it is possible. > > I’m happy to engage with the Firecracker team and get everyone together to > talk through the issues. > > We did used to package Firecracker for Amazon Linux (in an AL2 Extra), but > it had literally zero users from our repos (lambda and others build their > own). This could be due to just Firecracker by itself isn’t too useful > without some other easy integration with something like containerd. That > being said, I’d be interested in what use cases people have for it packaged > in fedora. > We are using it internally at our company to spawn multiple VMs that emulate our platform and due to the memory footprint libvirt has and lot of unnecessary features, we use firecracker. And would prefer to have it already packaged so that we won't be doing that ourselves. That said, integration with some other tools (e.g. OpenStack or even some custom scheduler to implement things like simple Lambda) would be great. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Mar 5, 2023, at 10:19 AM, Kevin Kofler via devel wrote: > > > David Michael wrote: >> - Firecracker can be built with Fedora's libc (glibc), but it is >> officially unsupported upstream[3]. Functionality would be harmed by >> not using musl, e.g. seccomp filters are not used. > > Upstream's refusal to write seccomp filters that work with glibc should be a > red flag. It is definitely possible to sandbox glibc applications with > seccomp, e.g., Chromium does it. It does need updates/fixes to the seccomp > rules with almost every new version of glibc, but it is possible. I’m happy to engage with the Firecracker team and get everyone together to talk through the issues. We did used to package Firecracker for Amazon Linux (in an AL2 Extra), but it had literally zero users from our repos (lambda and others build their own). This could be due to just Firecracker by itself isn’t too useful without some other easy integration with something like containerd. That being said, I’d be interested in what use cases people have for it packaged in fedora. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On 3/6/23 13:38, Richard W.M. Jones wrote: > On Sun, Mar 05, 2023 at 12:18:18AM +0100, Kevin Kofler via devel wrote: >> David Michael wrote: >>> - Firecracker can be built with Fedora's libc (glibc), but it is >>> officially unsupported upstream[3]. Functionality would be harmed by >>> not using musl, e.g. seccomp filters are not used. >> >> Upstream's refusal to write seccomp filters that work with glibc should be a >> red flag. It is definitely possible to sandbox glibc applications with >> seccomp, e.g., Chromium does it. It does need updates/fixes to the seccomp >> rules with almost every new version of glibc, but it is possible. > > And since we're talking hypervisors, qemu also manages to use glibc & > implement a seccomp filter. Is it an allowlist or a blocklist? Allowlists are much more secure, but require knowing about *everything* that a program will ever execute. Static linking makes this much easier to ensure. IIUC the main problem with musl is the lack of NSS support, which means that name lookups won’t work. One option would be to split Firecracker into two processes: a launcher that does all name lookups, and a sandboxed process that does everything else. -- Sincerely, Demi Marie Obenour (she/her/hers) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
Richard W.M. Jones wrote: > And since we're talking hypervisors, qemu also manages to use glibc & > implement a seccomp filter. Good to know. I was not aware that qemu has a seccomp filter, that is nice. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Sun, Mar 05, 2023 at 12:18:18AM +0100, Kevin Kofler via devel wrote: > David Michael wrote: > > - Firecracker can be built with Fedora's libc (glibc), but it is > > officially unsupported upstream[3]. Functionality would be harmed by > > not using musl, e.g. seccomp filters are not used. > > Upstream's refusal to write seccomp filters that work with glibc should be a > red flag. It is definitely possible to sandbox glibc applications with > seccomp, e.g., Chromium does it. It does need updates/fixes to the seccomp > rules with almost every new version of glibc, but it is possible. And since we're talking hypervisors, qemu also manages to use glibc & implement a seccomp filter. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Sun, Mar 5, 2023 at 1:31 AM David Michael wrote: > > Okay, thanks for all the feedback. I interpret this as essentially > requiring the use of the glibc Rust target for inclusion in Fedora, so > the changes on the Fedora side would be reduced to adding a couple > dozen crates and ideally supporting the kernel as a guest. That > shifts the bulk of the work to getting glibc fully supported upstream, > and running a not-quite-as-secure build until then. I'll investigate > this, and if there are no other blockers, maybe I can start sending > review requests for the glibc build. Sounds great! If you need help with the Rust packaging side of things, feel free to reach out. The Rust SIG is available on the "rust" list, in "#fedora-rust" on irc.libera.chat, or in the "Fedora Rust" room on the Fedora Matrix server. Either me or somebody else should be able to help pretty quickly if you're having trouble with Rust packaging toolchain, or if you need reviewers for packages for new crates. Fabio ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Sat, Mar 4, 2023 at 7:31 PM David Michael wrote: > > On Sat, Mar 4, 2023 at 5:51 PM Neal Gompa wrote: > > On Sat, Mar 4, 2023 at 12:41 PM David Michael wrote: > > > Hi, > > > > > > Firecracker[0] is a minimal virtual machine manager (a la QEMU) > > > written in Rust that uses KVM to start Linux VMs extremely quickly and > > > securely. It is used by AWS Lambda and Fargate among other things to > > > make VM startup time comparable to containers. I've built it for > > > Fedora x86_64 and shared it in a Copr repository[1] which includes > > > some example commands for starting VMs. > > > > > > Making it build for Fedora required changes across a few components, > > > so I'm writing to ask if this is acceptable for inclusion in Fedora. > > > The Copr specs are all dumped in a Git repository[2] for readability. > > > Changes include: > > > > > > - The musl package adds /usr paths for compatibility with the > > > compiler --sysroot option. > > > > As the musl package maintainer, I don't particularly want Fedora > > packages depending on it unless they absolutely have to. I originally > > maintained it in Copr, but brought it into Fedora for the Enarx folks. > > I still maintain that generally you don't want to use this unless you > > *really* know what you're doing and can deal with the consequences of > > using an alternative libc. > > It sounds like there are enough issues with musl in Fedora that glibc > will have to be used. > > > That said, the changes you make are not currently compatible with the > > packaging guidelines. Can't you just use --prefix /usr/ > > instead? > > It was for using --sysroot directly on gcc calls. > > > > - The rust compiler adds musl target subpackages. > > > > This will require discussion with the Fedora Rust SIG. > > > > > - The kernel must set CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y to be > > > usable as a guest. > > > > This will require talking to the kernel maintainers to add the flag. > > Okay, I'll ask about getting this applied even if Firecracker is not > included in Fedora. It still might be useful to use for a quick guest > kernel, and the option is just to support a command-line parameter[0], > so it shouldn't be disruptive. > > [0] > https://github.com/torvalds/linux/blob/v6.2/Documentation/admin-guide/kernel-parameters.txt#L6697 > > > > - About two dozen Rust crates must be added to Fedora (but a handful > > > are just new versions of existing packages). > > > - Unrelated, but in the Copr repo anyway: The musl package is fixed > > > to allow multilib installs, and Rust includes both 32- and 64-bit > > > targets. > > > > > > I used upstream-preferred settings when adding things, but they may be > > > in conflict with Fedora guidelines. Here are some concerns: > > > > > > - Firecracker can be built with Fedora's libc (glibc), but it is > > > officially unsupported upstream[3]. Functionality would be harmed by > > > not using musl, e.g. seccomp filters are not used. > > > > Why not drive this to be fixed upstream? Also, openSUSE builds > > Firecracker against glibc and I see the seccomp stuff is also produced > > and shipped[1]. I am not sure whether this means openSUSE's package is > > broken or something else. > > > > [1]: > > https://code.opensuse.org/package/firecracker/blob/master/f/firecracker.spec > > I can look into what they're doing more thoroughly later, but I would > assume they are just running with the empty seccomp filter. I am > aware of two issues with glibc: The jailer program allegedly does not > work with non-musl (although it builds) and is disabled upstream[1], > and there is no non-musl seccomp filter[2][3]. > > [1] https://github.com/firecracker-microvm/firecracker/pull/2125 > [2] > https://github.com/firecracker-microvm/firecracker/blob/v1.3.0/src/vmm/build.rs#L25 > [3] > https://github.com/firecracker-microvm/firecracker/tree/v1.3.0/resources/seccomp > > > > - Upstream Rust wants musl targets to be statically linked by > > > default[4]. It can be changed by patching (Gentoo does this) if > > > dynamic linking is still a priority with Rust binaries, but I haven't > > > tested that. > > > > As in musl is statically linked into the binaries? Or the Rust code is > > statically linked. The former is not okay, the latter is what we do > > already. > > The former, the musl targets produce static binaries by default. > > > > - Firecracker uses two crates forked from crates.io, but they are > > > not vendored/bundled nor published to a registry. I'm currently > > > manually bundling them as if they were vendored to avoid package name > > > conflicts since nothing else uses them, but I don't know the preferred > > > way to deal with those. > > > > > > > That's probably the right way to deal with it. > > > > > So does any of that sound like a showstopper for being included in > > > Fedora? Is there any other interest in the project from the > > > community? > > > > > > > It'd be interesting to have in Fedora, for sure,
Re: Firecracker microVM manager
On Sat, Mar 4, 2023 at 5:51 PM Neal Gompa wrote: > On Sat, Mar 4, 2023 at 12:41 PM David Michael wrote: > > Hi, > > > > Firecracker[0] is a minimal virtual machine manager (a la QEMU) > > written in Rust that uses KVM to start Linux VMs extremely quickly and > > securely. It is used by AWS Lambda and Fargate among other things to > > make VM startup time comparable to containers. I've built it for > > Fedora x86_64 and shared it in a Copr repository[1] which includes > > some example commands for starting VMs. > > > > Making it build for Fedora required changes across a few components, > > so I'm writing to ask if this is acceptable for inclusion in Fedora. > > The Copr specs are all dumped in a Git repository[2] for readability. > > Changes include: > > > > - The musl package adds /usr paths for compatibility with the > > compiler --sysroot option. > > As the musl package maintainer, I don't particularly want Fedora > packages depending on it unless they absolutely have to. I originally > maintained it in Copr, but brought it into Fedora for the Enarx folks. > I still maintain that generally you don't want to use this unless you > *really* know what you're doing and can deal with the consequences of > using an alternative libc. It sounds like there are enough issues with musl in Fedora that glibc will have to be used. > That said, the changes you make are not currently compatible with the > packaging guidelines. Can't you just use --prefix /usr/ > instead? It was for using --sysroot directly on gcc calls. > > - The rust compiler adds musl target subpackages. > > This will require discussion with the Fedora Rust SIG. > > > - The kernel must set CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y to be > > usable as a guest. > > This will require talking to the kernel maintainers to add the flag. Okay, I'll ask about getting this applied even if Firecracker is not included in Fedora. It still might be useful to use for a quick guest kernel, and the option is just to support a command-line parameter[0], so it shouldn't be disruptive. [0] https://github.com/torvalds/linux/blob/v6.2/Documentation/admin-guide/kernel-parameters.txt#L6697 > > - About two dozen Rust crates must be added to Fedora (but a handful > > are just new versions of existing packages). > > - Unrelated, but in the Copr repo anyway: The musl package is fixed > > to allow multilib installs, and Rust includes both 32- and 64-bit > > targets. > > > > I used upstream-preferred settings when adding things, but they may be > > in conflict with Fedora guidelines. Here are some concerns: > > > > - Firecracker can be built with Fedora's libc (glibc), but it is > > officially unsupported upstream[3]. Functionality would be harmed by > > not using musl, e.g. seccomp filters are not used. > > Why not drive this to be fixed upstream? Also, openSUSE builds > Firecracker against glibc and I see the seccomp stuff is also produced > and shipped[1]. I am not sure whether this means openSUSE's package is > broken or something else. > > [1]: > https://code.opensuse.org/package/firecracker/blob/master/f/firecracker.spec I can look into what they're doing more thoroughly later, but I would assume they are just running with the empty seccomp filter. I am aware of two issues with glibc: The jailer program allegedly does not work with non-musl (although it builds) and is disabled upstream[1], and there is no non-musl seccomp filter[2][3]. [1] https://github.com/firecracker-microvm/firecracker/pull/2125 [2] https://github.com/firecracker-microvm/firecracker/blob/v1.3.0/src/vmm/build.rs#L25 [3] https://github.com/firecracker-microvm/firecracker/tree/v1.3.0/resources/seccomp > > - Upstream Rust wants musl targets to be statically linked by > > default[4]. It can be changed by patching (Gentoo does this) if > > dynamic linking is still a priority with Rust binaries, but I haven't > > tested that. > > As in musl is statically linked into the binaries? Or the Rust code is > statically linked. The former is not okay, the latter is what we do > already. The former, the musl targets produce static binaries by default. > > - Firecracker uses two crates forked from crates.io, but they are > > not vendored/bundled nor published to a registry. I'm currently > > manually bundling them as if they were vendored to avoid package name > > conflicts since nothing else uses them, but I don't know the preferred > > way to deal with those. > > > > That's probably the right way to deal with it. > > > So does any of that sound like a showstopper for being included in > > Fedora? Is there any other interest in the project from the > > community? > > > > It'd be interesting to have in Fedora, for sure, but this should be > done as an opportunity to bring our integration concerns upstream. Okay, thanks for all the feedback. I interpret this as essentially requiring the use of the glibc Rust target for inclusion in Fedora, so the changes on the Fedora side would
Re: Firecracker microVM manager
On Sat, Mar 4, 2023 at 6:18 PM Kevin Kofler via devel wrote: > > David Michael wrote: > > - Firecracker can be built with Fedora's libc (glibc), but it is > > officially unsupported upstream[3]. Functionality would be harmed by > > not using musl, e.g. seccomp filters are not used. > > Upstream's refusal to write seccomp filters that work with glibc should be a > red flag. It is definitely possible to sandbox glibc applications with > seccomp, e.g., Chromium does it. It does need updates/fixes to the seccomp > rules with almost every new version of glibc, but it is possible. > That is not a glibc specific problem, since glibc API churn is often the result of changes in the kernel UAPI. It happens with musl too, just less frequently. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
David Michael wrote: > - Firecracker can be built with Fedora's libc (glibc), but it is > officially unsupported upstream[3]. Functionality would be harmed by > not using musl, e.g. seccomp filters are not used. Upstream's refusal to write seccomp filters that work with glibc should be a red flag. It is definitely possible to sandbox glibc applications with seccomp, e.g., Chromium does it. It does need updates/fixes to the seccomp rules with almost every new version of glibc, but it is possible. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Firecracker microVM manager
On Sat, Mar 4, 2023 at 12:41 PM David Michael wrote: > > Hi, > > Firecracker[0] is a minimal virtual machine manager (a la QEMU) > written in Rust that uses KVM to start Linux VMs extremely quickly and > securely. It is used by AWS Lambda and Fargate among other things to > make VM startup time comparable to containers. I've built it for > Fedora x86_64 and shared it in a Copr repository[1] which includes > some example commands for starting VMs. > > Making it build for Fedora required changes across a few components, > so I'm writing to ask if this is acceptable for inclusion in Fedora. > The Copr specs are all dumped in a Git repository[2] for readability. > Changes include: > > - The musl package adds /usr paths for compatibility with the > compiler --sysroot option. As the musl package maintainer, I don't particularly want Fedora packages depending on it unless they absolutely have to. I originally maintained it in Copr, but brought it into Fedora for the Enarx folks. I still maintain that generally you don't want to use this unless you *really* know what you're doing and can deal with the consequences of using an alternative libc. That said, the changes you make are not currently compatible with the packaging guidelines. Can't you just use --prefix /usr/ instead? > - The rust compiler adds musl target subpackages. This will require discussion with the Fedora Rust SIG. > - The kernel must set CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y to be > usable as a guest. This will require talking to the kernel maintainers to add the flag. > - About two dozen Rust crates must be added to Fedora (but a handful > are just new versions of existing packages). > - Unrelated, but in the Copr repo anyway: The musl package is fixed > to allow multilib installs, and Rust includes both 32- and 64-bit > targets. > > I used upstream-preferred settings when adding things, but they may be > in conflict with Fedora guidelines. Here are some concerns: > > - Firecracker can be built with Fedora's libc (glibc), but it is > officially unsupported upstream[3]. Functionality would be harmed by > not using musl, e.g. seccomp filters are not used. Why not drive this to be fixed upstream? Also, openSUSE builds Firecracker against glibc and I see the seccomp stuff is also produced and shipped[1]. I am not sure whether this means openSUSE's package is broken or something else. [1]: https://code.opensuse.org/package/firecracker/blob/master/f/firecracker.spec > - Upstream Rust wants musl targets to be statically linked by > default[4]. It can be changed by patching (Gentoo does this) if > dynamic linking is still a priority with Rust binaries, but I haven't > tested that. As in musl is statically linked into the binaries? Or the Rust code is statically linked. The former is not okay, the latter is what we do already. > - Firecracker uses two crates forked from crates.io, but they are > not vendored/bundled nor published to a registry. I'm currently > manually bundling them as if they were vendored to avoid package name > conflicts since nothing else uses them, but I don't know the preferred > way to deal with those. > That's probably the right way to deal with it. > So does any of that sound like a showstopper for being included in > Fedora? Is there any other interest in the project from the > community? > It'd be interesting to have in Fedora, for sure, but this should be done as an opportunity to bring our integration concerns upstream. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
