Bug#948041: impossible to update libbpf without updating the kernel
On Sun, 19 Jan 2020 22:32:22 -0800 Andrii Nakryiko wrote: > Libbpf releases are only loosely synchronized with kernel releases, > and it's only due to current convention we established within > community. There is nothing preventing multiple releases of libbpf > between two releases of kernel. So having Linux distributions use > Github's release tags ensures that all of them are building from > exactly the same source code version. > > > "No ties to any specific kernel, transparent handling of older kernels." > > > > What? Either they are upstream for it or patch the source. But this > > point means it can't be a direct mirror. So what is it? > > It is mostly a direct mirror, with some extra Github-specific stuff > beside (like Travis CI testing, etc). All the libbpf source code fixes > do go through bpf-next tree, though. > > But the point here was that libbpf itself is not built with any > particular kernel version in mind. It is by design possible to use the > very latest libbpf against a pretty old kernel version and this will > work: libbpf will degrade some of the features (e.g, BTF type info > associations), but everything will keep working otherwise (unless user > requests feature that requires latest kernel version, of course: in > that case BPF program will fail to load and libbpf will return > corresponding error code). +1 Having libbpf packaged from GH would spare Linux user space developers from headache of supporting different versioning across distributions, e.g. in Fedora and Debian. Libbpf releases are independent from kernel releases and mapping from kernel to libbpf versions will confuse and complicate developers. Libbpf from GH should be a common denominator across distributions, so Debian doesn't create a corner case.
Bug#948041: impossible to update libbpf without updating the kernel
On Sun, 12 Jan 2020 09:51:55 +0100 Bastian Blank wrote: > But updating independently does not provide any benefit. As the kernel > repo is still upstream, all upstream chnages will flow into the Debian > archive. An extra package will always lag behind, as changes needs to > flow from kernel to external repo. > > > It might be better if we decide now what we want to do and tell them > > accordingly. > > Why should we? If the upstream developers decide to maintain it > independently, aka don't use the kernel repo as true source, or better, > remove the source from it, then we have something to do. > > Bastian Hi Bastian, Why should we switch from kernel sources to GH is a frequently asked question so the reason was explained in libbpf README [1]. There is another bug report on libbpf packaging filed earlier by me [2], feel merge it with this one. [1] https://github.com/libbpf/libbpf#distributions [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=942903
Bug#942903: libbpf library is from kernel source
Package: libbpf-dev Version: 5.3.7-1 Severity: wishlist Hi, libbpf-dev is currently packaged from kernel sources. Please consider switching to GH mirror [1] so there are following advantages: - Versioning is consistent with ABI - GH mirror has CI tests - Be consistent w/ other distros, e.g. Fedora which has already switched to GH mirror in [2] The whole rationale is in [3] Thanks! [1] https://github.com/libbpf/libbpf [2] https://bugzilla.redhat.com/show_bug.cgi?id=1745478 [3] https://lore.kernel.org/netdev/3fbec3f8-5c3c-40f9-af6e-c355d8f62...@fb.com/
Re: libbpf distro packaging
Hi Jiri,
systemd folks published libbpf CentOS 7 package in systemd corp repo: [1],
so guess that proves that deps from other repo are fine.
Rebuild is fairly simple: [2]
[1]
https://copr.fedorainfracloud.org/coprs/mrc0mmand/systemd-centos-ci/build/1053694/
[2] https://github.com/systemd/systemd/pull/13744#issuecomment-541168076
On 10/8/19, 12:40 AM, "Jiri Olsa" wrote:
>
> On Mon, Oct 07, 2019 at 12:25:51AM +, Julia Kartseva wrote:
> >
> > I wonder what are the steps to make libbpf available for CentOS {7|8} as
> > well?
> > One (likely the quickest) way to do that is to publish it to Fedora's EPEL
> > [1].
> >
> > I have a little concern about dependencies, namely elfutils-libelf-devel
> > and
> > elfutils-devel are sourced directly by CentOS repos, e.g. [2], not sure if
> > dependencies from another repo are fine.
> >
> > Thoughts? Thanks!
>
> I think that should be ok, I'll ask around and let you know
>
> jirka
Re: libbpf distro packaging
On 9/30/19, 4:13 AM, "Jiri Olsa" wrote:
> heya,
> FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
>
> jirka
Hi Jiri,
I wonder what are the steps to make libbpf available for CentOS {7|8} as well?
One (likely the quickest) way to do that is to publish it to Fedora's EPEL [1].
I have a little concern about dependencies, namely elfutils-libelf-devel and
elfutils-devel are sourced directly by CentOS repos, e.g. [2], not sure if
dependencies from another repo are fine.
Thoughts? Thanks!
[1] https://fedoraproject.org/wiki/EPEL
[2] http://mirror.centos.org/centos/7/os/x86_64/Packages/
Re: libbpf distro packaging
Hi Jiri, v0.0.5 is out: [1] which brings questions regarding further maintenance of rpm. Who'll maintain the most recent version of rpm up-to-date? Are you looking into making the procedure automated and do you need any help from the side of libbpf devs if so? In particular, we can have the *.spec file in GH mirror synchronized with the most recent tag so you can take it from the mirror along with tarball. Thanks! [1] https://github.com/libbpf/libbpf/releases/tag/v0.0.5 On 9/30/19, 11:18 AM, "Julia Kartseva" wrote: > Thank you Jiri, that's great news. > > > On 9/30/19, 4:13 AM, "Jiri Olsa" wrote: > > > > heya, > > FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32 > > I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
Re: libbpf distro packaging
Thank you Jiri, that's great news. Adding Marco D'Itri. Marco, I wonder if you are OK with the rationale for libbpf packaging from GH mirror? Can we proceed with switching Debian package as well just like we discussed offline at ASG? The bug report for Fedora: [1] Thank you [1] https://bugzilla.redhat.com/show_bug.cgi?id=1745478 > On 9/30/19, 4:13 AM, "Jiri Olsa" wrote: > > heya, > FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32 > I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
Re: libbpf distro packaging
On 8/25/19, 11:42 PM, "Jiri Olsa" wrote: > On Fri, Aug 23, 2019 at 04:00:01PM +, Alexei Starovoitov wrote: > > > > Technically we can bump it at any time. > > The goal was to bump it only when new kernel is released > > to capture a collection of new APIs in a given 0.0.X release. > > So that libbpf versions are synchronized with kernel versions > > in some what loose way. > > In this case we can make an exception and bump it now. > > I see, I dont think it's worth of the exception now, > the patch is simple or we'll start with 0.0.3 PR introducing 0.0.5 ABI was merged: https://github.com/libbpf/libbpf/commit/476e158 Jiri, you'd like to avoid patching, you can start w/ 0.0.5. Also if you're planning to use *.spec from libbpf as a source of truth, It may be enhanced by syncing spec and ABI versions, similar to https://github.com/libbpf/libbpf/commit/d60f568
Re: libbpf distro packaging
On 8/19/19, 11:08 AM, "Julia Kartseva" wrote: On 8/13/19, 11:24 AM, "Andrii Nakryiko" wrote: On Tue, Aug 13, 2019 at 5:26 AM Jiri Olsa wrote: > > On Mon, Aug 12, 2019 at 07:04:12PM +, Julia Kartseva wrote: > > I would like to bring up libbpf publishing discussion started at [1]. > > The present state of things is that libbpf is built from kernel tree, e.g. [2] > > For Debian and [3] for Fedora whereas the better way would be having a > > package built from github mirror. The advantages of the latter: > > - Consistent, ABI matching versioning across distros > > - The mirror has integration tests > > - No need in kernel tree to build a package > > - Changes can be merged directly to github w/o waiting them to be merged > > through bpf-next -> net-next -> main > > There is a PR introducing a libbpf.spec which can be used as a starting point: [4] > > Any comments regarding the spec itself can be posted there. > > In the future it may be used as a source of truth. > > Please consider switching libbpf packaging to the github mirror instead > > of the kernel tree. > > Thanks > > > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.iovisor.org_g_iovisor-2Ddev_message_1521=DwIBaQ=5VD0RTtNlTh3ycd41b3MUw=zUrDY_Sp_5PqcGtRQPNeDA=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ=dYAc2jLhFg0wtCZ_ms2HF5bWANoHzA3UMug5TNCeBtE= > > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__packages.debian.org_sid_libbpf4.19=DwIBaQ=5VD0RTtNlTh3ycd41b3MUw=zUrDY_Sp_5PqcGtRQPNeDA=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ=lq1MpF-bt6y6ZEtFc57eT-BO_wMBx8uUBACJooWbUYk= > > [3] https://urldefense.proofpoint.com/v2/url?u=http-3A__rpmfind.net_linux_RPM_fedora_devel_rawhide_x86-5F64_l_libbpf-2D5.3.0-2D0.rc2.git0.1.fc31.x86-5F64.html=DwIBaQ=5VD0RTtNlTh3ycd41b3MUw=zUrDY_Sp_5PqcGtRQPNeDA=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ=NoolYHL57G2KhzE768iWdy6v5LD2GfJQyqPmtjy196E= > > [4] https://github.com/libbpf/libbpf/pull/64 > > hi, > Fedora has libbpf as kernel-tools subpackage, so I think > we'd need to create new package and deprecate the current > > but I like the ABI stability by using github .. how's actually > the sync (in both directions) with kernel sources going on? Sync is always in one direction, from kernel sources into Github repo. Right now it's triggered by a human (usually me), but we are using a script that automates entire process (see https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh). It cherry-pick relevant commits from kernel, transforms them to match Github's file layout and re-applies those changes to Github repo. There is never a sync from Github back to kernel, but Github repo contains some extra stuff that's not in kernel. E.g., the script I mentioned, plus Github's Makefile is different, because it can't rely on kernel's kbuild setup. Hi Jiri, I'm curious if you have any comments regarding sync procedure described By Andrii. Or if there is anything else you'd like us to address so Fedora can be switched to libbpf built from the github mirror? Thank you > > thanks, > jirka
libbpf distro packaging
I would like to bring up libbpf publishing discussion started at [1]. The present state of things is that libbpf is built from kernel tree, e.g. [2] For Debian and [3] for Fedora whereas the better way would be having a package built from github mirror. The advantages of the latter: - Consistent, ABI matching versioning across distros - The mirror has integration tests - No need in kernel tree to build a package - Changes can be merged directly to github w/o waiting them to be merged through bpf-next -> net-next -> main There is a PR introducing a libbpf.spec which can be used as a starting point: [4] Any comments regarding the spec itself can be posted there. In the future it may be used as a source of truth. Please consider switching libbpf packaging to the github mirror instead of the kernel tree. Thanks [1] https://lists.iovisor.org/g/iovisor-dev/message/1521 [2] https://packages.debian.org/sid/libbpf4.19 [3] http://rpmfind.net/linux/RPM/fedora/devel/rawhide/x86_64/l/libbpf-5.3.0-0.rc2.git0.1.fc31.x86_64.html [4] https://github.com/libbpf/libbpf/pull/64
