Re: Packaging rules for build from source vs BPF byte code ?
On Tue, Nov 3, 2020 at 3:49 AM Daniel P. Berrangé wrote: > > In QEMU there's a desire to make use of BPF programs for implementing > some networking features. The current patches are proposing adding > prebuilt BPF byte code to the QEMU repo, with source available, but > not actually building from source during a build. > > I was wondering if we had any specific guidance or rules covering the > shipping BPF programs in particular ? I wonder if this trend in BPF might be related, and needs to be accounted for? BPF binaries: BTF, CO-RE, and the future of BPF perf tools http://www.brendangregg.com/blog/2020-11-04/bpf-co-re-btf-libbpf.html Fedora does have CONFIG_DEBUG_INFO_BTF=y -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
Dne 05. 11. 20 v 13:03 Daniel P. Berrangé napsal(a): On Tue, Nov 03, 2020 at 11:58:54AM +0100, Fabio Valentini wrote: On Tue, Nov 3, 2020 at 11:49 AM Daniel P. Berrangé wrote: In QEMU there's a desire to make use of BPF programs for implementing some networking features. The current patches are proposing adding prebuilt BPF byte code to the QEMU repo, with source available, but not actually building from source during a build. I was wondering if we had any specific guidance or rules covering the shipping BPF programs in particular ? To me it feels like BPF programs should fall under normal Fedora practice that expects everything to be built from master source. We do have the exception that allows firmware to be shipped as pre-built blobs, but I'm thinking that BPF programs could not be considered as firmware. Has this been discussed before, if so can someone point to the results, as I'm not finding anything specific to BPF programs and Fedora packaging via Google. If there are no specialized Packaging Guidelines for something, then the general guidelines apply - so in this case, compiling from source is required, since Fedora packages MUST NOT ship precompiled binaries. It seems this case is a little more fuzzy in terms of what is considered "the preferred source" format, and whether the BPF program is in fact considered a "binary" at all. I think that the best would be if the "binary" was stripped from the guidelines. Because, you could somewhat argue about JARs or GEM and I believe, that this guideline should apply also to documentation as well as various grammar parsers generators etc. Vít There is the "dpdk" package in Fedora which is doing similar to what is being proposed for QEMU. In dpdk-stable-19.11.3/drivers/net/tap/tap_bpf_program.c there is BPF "source code". This was, at some point in time, apparently compiled using some undefined process to create a binary byte code blob. This binary blob was then processed, again by some undefined process, to extract just the BPF instructions, which are then turned back into "source code" as a static byte array in a C header file. The latter is what is actually compiled. This derived source is in the dpdk-stable-19.11.3/drivers/net/tap/tap_bpf_insns.h Given that dpdk is not providing any build process for going from tap_bpf_program.c to tap_bpf_insns.h, it looks to me like the upstream project effectively considers "tap_bpf_insns.h" to be their preferred source form, and tap_bpf_program.c as just a reference for its original creation. IOW, to me it looks like this embedded BPF program shouldn't be considered a binary from Fedora POV and can be used as is. Regards, Daniel ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
Hi, > QEMU ships various keymap data files which are auto-generated from data > provided by X11, but none of these are built "from source", because the > generated data files are considered the preferred source form. Well, the actual reason is the X11 dependency. qemu needs those files also on non-X11 platforms like Windows, but generating them on non-X11 platforms doesn't work. Also note that if we could easily gather the data needed to generate those keymap files on all platforms we would not need them in the first place ;) BTW: Starting with qemu 5.2 those files will be regenerated during the build if the required dependency (xkbcommon-devel) is available at build time. take care, Gerd ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On Thu, Nov 05, 2020 at 07:54:16AM -0500, Chuck Anderson wrote: > On Thu, Nov 05, 2020 at 12:03:43PM +, Daniel P. Berrangé wrote: > > Given that dpdk is not providing any build process for going from > > tap_bpf_program.c to tap_bpf_insns.h, it looks to me like the upstream > > project effectively considers "tap_bpf_insns.h" to be their preferred > > source form, and tap_bpf_program.c as just a reference for its > > original creation. > > > > IOW, to me it looks like this embedded BPF program shouldn't be > > considered a binary from Fedora POV and can be used as is. > > One example of a package doing something incorreclty doesn't mean it > is okay to continue that practice. I'd argue thet dpdk needs to be > fixed to provide the original source code and build scripts for the > BPF program. The point is that this doesn't feel much different from many other scenarios that applications often do wrt code generation. Any non trivial application will have source files whose contents are auto-generated, and the used as is, instead of building from source every time. Every single autotools based application is shipping a pre-generated configure script, whose content may or may not match what is expressed in the configure.ac script. Fedora doesn't require "configure" to be re-generated from source every time, nor validate that the "configure.ac" actually is the corresponding source. There are certainly other cases where programs ship sources which were originally generated from some other data file, but don't even include the rules to reproduce that. QEMU ships various keymap data files which are auto-generated from data provided by X11, but none of these are built "from source", because the generated data files are considered the preferred source form. Libvirt ships various test cases where the source is auto-generated from data extracted from QEMU. These generated files are used as is, and not re-created by QEMU during Fedora builds, because Fedora doesn't have all 30+ historical versions of QEMU available in the build root. The embedded static arrays in source containing BPF instruction are not conceptually that different from other cases where we build using generated sources and don't re-generated during Fedora builds. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On Thu, Nov 05, 2020 at 12:03:43PM +, Daniel P. Berrangé wrote: > Given that dpdk is not providing any build process for going from > tap_bpf_program.c to tap_bpf_insns.h, it looks to me like the upstream > project effectively considers "tap_bpf_insns.h" to be their preferred > source form, and tap_bpf_program.c as just a reference for its > original creation. > > IOW, to me it looks like this embedded BPF program shouldn't be > considered a binary from Fedora POV and can be used as is. One example of a package doing something incorreclty doesn't mean it is okay to continue that practice. I'd argue thet dpdk needs to be fixed to provide the original source code and build scripts for the BPF program. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On Tue, Nov 03, 2020 at 11:58:54AM +0100, Fabio Valentini wrote: > On Tue, Nov 3, 2020 at 11:49 AM Daniel P. Berrangé > wrote: > > > > In QEMU there's a desire to make use of BPF programs for implementing > > some networking features. The current patches are proposing adding > > prebuilt BPF byte code to the QEMU repo, with source available, but > > not actually building from source during a build. > > > > I was wondering if we had any specific guidance or rules covering the > > shipping BPF programs in particular ? > > > > To me it feels like BPF programs should fall under normal Fedora > > practice that expects everything to be built from master source. > > > > We do have the exception that allows firmware to be shipped as > > pre-built blobs, but I'm thinking that BPF programs could not > > be considered as firmware. > > > > Has this been discussed before, if so can someone point to the > > results, as I'm not finding anything specific to BPF programs and > > Fedora packaging via Google. > > If there are no specialized Packaging Guidelines for something, then > the general guidelines apply - so in this case, compiling from source > is required, since Fedora packages MUST NOT ship precompiled binaries. It seems this case is a little more fuzzy in terms of what is considered "the preferred source" format, and whether the BPF program is in fact considered a "binary" at all. There is the "dpdk" package in Fedora which is doing similar to what is being proposed for QEMU. In dpdk-stable-19.11.3/drivers/net/tap/tap_bpf_program.c there is BPF "source code". This was, at some point in time, apparently compiled using some undefined process to create a binary byte code blob. This binary blob was then processed, again by some undefined process, to extract just the BPF instructions, which are then turned back into "source code" as a static byte array in a C header file. The latter is what is actually compiled. This derived source is in the dpdk-stable-19.11.3/drivers/net/tap/tap_bpf_insns.h Given that dpdk is not providing any build process for going from tap_bpf_program.c to tap_bpf_insns.h, it looks to me like the upstream project effectively considers "tap_bpf_insns.h" to be their preferred source form, and tap_bpf_program.c as just a reference for its original creation. IOW, to me it looks like this embedded BPF program shouldn't be considered a binary from Fedora POV and can be used as is. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On 03.11.2020 11:48, Daniel P. Berrangé wrote: We do have the exception that allows firmware to be shipped as pre-built blobs, but I'm thinking that BPF programs could not be considered as firmware. According to Fedora packaging guidelines, all packages must be built from sources. No exceptions. BPF is not a firmware. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On Tue, Nov 03, 2020 at 11:13:53AM +, Daniel P. Berrangé wrote: > On Tue, Nov 03, 2020 at 11:58:54AM +0100, Fabio Valentini wrote: > > On Tue, Nov 3, 2020 at 11:49 AM Daniel P. Berrangé > > wrote: > > > > > > In QEMU there's a desire to make use of BPF programs for implementing > > > some networking features. The current patches are proposing adding > > > prebuilt BPF byte code to the QEMU repo, with source available, but > > > not actually building from source during a build. > > > > > > I was wondering if we had any specific guidance or rules covering the > > > shipping BPF programs in particular ? > > > > > > To me it feels like BPF programs should fall under normal Fedora > > > practice that expects everything to be built from master source. > > > > > > We do have the exception that allows firmware to be shipped as > > > pre-built blobs, but I'm thinking that BPF programs could not > > > be considered as firmware. > > > > > > Has this been discussed before, if so can someone point to the > > > results, as I'm not finding anything specific to BPF programs and > > > Fedora packaging via Google. > > > > > > Regards, > > > Daniel > > > > If there are no specialized Packaging Guidelines for something, then > > the general guidelines apply - so in this case, compiling from source > > is required, since Fedora packages MUST NOT ship precompiled binaries. > > > > Side note: Regarding BPF programs - I seem to remember that recent > > kernel security features (the Lockdown patches?) restricted and/or > > disabled the ability to run BPF programs at all. Have you considered > > that by default, those BPF programs might not be able to run under the > > Fedora default configuration? > > Yes, that is one of the issues raised upstream by other people. BPF appears to only be restricted if lockdown is running in "confidentiality mode", not "integrity mode", with the latter used in Fedora now according to https://bugzilla.redhat.com/show_bug.cgi?id=1815571#c3 Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
* Daniel P. Berrangé: > On Tue, Nov 03, 2020 at 11:58:32AM +0100, Florian Weimer wrote: >> * Daniel P. Berrangé: >> >> > In QEMU there's a desire to make use of BPF programs for implementing >> > some networking features. The current patches are proposing adding >> > prebuilt BPF byte code to the QEMU repo, with source available, but >> > not actually building from source during a build. >> > >> > I was wondering if we had any specific guidance or rules covering the >> > shipping BPF programs in particular ? >> >> Is this even technically feasible? I don't think the kernel presents a >> stable interface to BPF programs. > > I'm not sure to be honest, but I'll raise that as a question to the > patch submitter. I'm not very familiar with BPF, just saw the suggestion > to ship pre-built BPF bytecode and was concerned this would not be > acceptable for Fedora. Is this about ebpf/rss.bpf.c? It looks more like an attempt to bypass the Fedora kernel module policy. Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On Tue, Nov 03, 2020 at 11:58:32AM +0100, Florian Weimer wrote: > * Daniel P. Berrangé: > > > In QEMU there's a desire to make use of BPF programs for implementing > > some networking features. The current patches are proposing adding > > prebuilt BPF byte code to the QEMU repo, with source available, but > > not actually building from source during a build. > > > > I was wondering if we had any specific guidance or rules covering the > > shipping BPF programs in particular ? > > Is this even technically feasible? I don't think the kernel presents a > stable interface to BPF programs. I'm not sure to be honest, but I'll raise that as a question to the patch submitter. I'm not very familiar with BPF, just saw the suggestion to ship pre-built BPF bytecode and was concerned this would not be acceptable for Fedora. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On Tue, Nov 03, 2020 at 11:58:54AM +0100, Fabio Valentini wrote: > On Tue, Nov 3, 2020 at 11:49 AM Daniel P. Berrangé > wrote: > > > > In QEMU there's a desire to make use of BPF programs for implementing > > some networking features. The current patches are proposing adding > > prebuilt BPF byte code to the QEMU repo, with source available, but > > not actually building from source during a build. > > > > I was wondering if we had any specific guidance or rules covering the > > shipping BPF programs in particular ? > > > > To me it feels like BPF programs should fall under normal Fedora > > practice that expects everything to be built from master source. > > > > We do have the exception that allows firmware to be shipped as > > pre-built blobs, but I'm thinking that BPF programs could not > > be considered as firmware. > > > > Has this been discussed before, if so can someone point to the > > results, as I'm not finding anything specific to BPF programs and > > Fedora packaging via Google. > > > > Regards, > > Daniel > > If there are no specialized Packaging Guidelines for something, then > the general guidelines apply - so in this case, compiling from source > is required, since Fedora packages MUST NOT ship precompiled binaries. > > Side note: Regarding BPF programs - I seem to remember that recent > kernel security features (the Lockdown patches?) restricted and/or > disabled the ability to run BPF programs at all. Have you considered > that by default, those BPF programs might not be able to run under the > Fedora default configuration? Yes, that is one of the issues raised upstream by other people. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
On Tue, Nov 3, 2020 at 11:49 AM Daniel P. Berrangé wrote: > > In QEMU there's a desire to make use of BPF programs for implementing > some networking features. The current patches are proposing adding > prebuilt BPF byte code to the QEMU repo, with source available, but > not actually building from source during a build. > > I was wondering if we had any specific guidance or rules covering the > shipping BPF programs in particular ? > > To me it feels like BPF programs should fall under normal Fedora > practice that expects everything to be built from master source. > > We do have the exception that allows firmware to be shipped as > pre-built blobs, but I'm thinking that BPF programs could not > be considered as firmware. > > Has this been discussed before, if so can someone point to the > results, as I'm not finding anything specific to BPF programs and > Fedora packaging via Google. > > Regards, > Daniel If there are no specialized Packaging Guidelines for something, then the general guidelines apply - so in this case, compiling from source is required, since Fedora packages MUST NOT ship precompiled binaries. Side note: Regarding BPF programs - I seem to remember that recent kernel security features (the Lockdown patches?) restricted and/or disabled the ability to run BPF programs at all. Have you considered that by default, those BPF programs might not be able to run under the Fedora default configuration? Fabio ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Packaging rules for build from source vs BPF byte code ?
* Daniel P. Berrangé: > In QEMU there's a desire to make use of BPF programs for implementing > some networking features. The current patches are proposing adding > prebuilt BPF byte code to the QEMU repo, with source available, but > not actually building from source during a build. > > I was wondering if we had any specific guidance or rules covering the > shipping BPF programs in particular ? Is this even technically feasible? I don't think the kernel presents a stable interface to BPF programs. Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Packaging rules for build from source vs BPF byte code ?
In QEMU there's a desire to make use of BPF programs for implementing some networking features. The current patches are proposing adding prebuilt BPF byte code to the QEMU repo, with source available, but not actually building from source during a build. I was wondering if we had any specific guidance or rules covering the shipping BPF programs in particular ? To me it feels like BPF programs should fall under normal Fedora practice that expects everything to be built from master source. We do have the exception that allows firmware to be shipped as pre-built blobs, but I'm thinking that BPF programs could not be considered as firmware. Has this been discussed before, if so can someone point to the results, as I'm not finding anything specific to BPF programs and Fedora packaging via Google. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
