Re: build profile proposal: nogir (second try)
On Wed, Jan 17, 2024 at 11:38:09PM +, Simon McVittie wrote:
> On Wed, 17 Jan 2024 at 23:15:03 +0100, Matthias Geiger wrote:
> > Does this mean we should should split out the .gir XML files from existing
> > source packages into a separate gir1.2-foo-dev (in the long run) ?
>
> That's a good question, and I don't have an easy answer for it. The
> tradeoff is:
>
> - having the GIR XML in libfoo-dev means fewer binary packages and
> therefore smaller Packages metadata;
>
> - having a separate (non-virtual) gir1.2-foo-1-dev means we can "cleanly"
> turn off GIR/typelibs in cases when they're not needed, and means
> libfoo-dev is a bit smaller and with fewer dependencies
Really, I think the main advantage of splitting them out into real
packages is the additional QA that we get. With the Provides-mechanism,
consumers will often miss the additional gir1.2-*-dev build dependency
that is required and adding those back will be a permanent duty of cross
build porters.
> It's analogous to the choice between one big -dev package (libcairo2-dev,
> libwayland-dev) or multiple smaller -dev packages (libportal*-dev) for a
> source package with more than one shared library.
The QA aspect is different there.
> The larger, more widely-used and lower-level the library is, the more I
> would be inclined to opt for the approach with extra binary packages
> - for example splitting out gir1.2-gtk-4.0-dev from libgtk-4-dev
> seems more desirable than splitting out gir1.2-shumate-1.0-dev from
> libshumate-dev. Separating out the GIR XML is more interesting for
> packages that are involved in bootstrapping, or for packages that someone
> will frequently want to cross-compile, particularly the ones you'll want
> to cross-compile early in the development of a new port when tools like
> qemu-user might not be able to target it.
In essence, you are arguing for deciding on a case-by-case way and I
concur with that. The provides mechanism seems easier for maintainers
and so I'd recommend doing that, then changing to the split mechanism
where we deem it useful.
> In the case where the GIR XML is in libfoo-dev, asking for it to have
> Provides: gir1.2-foo-1-dev means that dependent packages can depend on the
> systematic gir1.2-foo-1-dev name, and then will work correctly either way.
The real question becomes how we can continuously ensure that packages
correctly depend on these virtual facilities. I fear the simplest way is
actually splitting the binary packages. Does anyone have a better idea?
> The only package where I'm sure that I intend to separate out the GIR
> XML in the short term is src:glib2.0, where for historical reasons
> gir1.2-glib-2.0 has been built by src:gobject-introspection until
> now. I'm most of the way through preparing a version of glib2.0
> 2.79.x for experimental that takes over gir1.2-glib-2.0{,-dev}
> from src:gobject-introspection, and I definitely don't want to fold
> gir1.2-glib-2.0-dev into libglib2.0-dev, because GLib's position at the
> bottom of the GNOME stack makes it particularly important that we can
> still bootstrap and cross-compile it.
Thank you. How annoying would it actually be to split this to a
different source package? glib2.0 is involved with bootstrap at this
time and that works fully automatically *because* it is not involved
with gir. When you add gir, builders have to add the nogir profile (and
thus manually order glib2.0). If you were to split this into two
distinct source packages, you'd remove the need for applying a build
profile and thus automatic bootstrapping continues to work. Of course, I
cannot tell how that impacts the implementation, but given that it
formerly was part of src:gobject-introspection, it cannot be unworkable.
Quite definitely, such a split is not a requirement though.
Helmut
Re: build profile proposal: nogir (second try)
On Wed, Jan 24, 2024 at 06:30:02PM +, Alberto Garcia wrote: > - Are packages that ship gobject-introspection files supposed to have >in the relevant build dependencies (gir1.2-*-dev, > gobject-introspection ?), or is the build profile handling this > automatically? This is not automatic. Please annotate relevant Build-Depends manually. > - Packages using dh_install may have a line with usr/share/gir-1.0 in > their debian/libfoo-dev.install. This will fail if the .gir files > are not generated. What's the recommended way to handle this? There is no silver bullet. Options: * You may use dh-exec. When doing so, you may annotate lines with build profiles. For example, samba's debian/winbind.install uses this approach. * You may conditionally run dh_install from debian/rules passing affected files as arguments. * You may split the affected files into a separate binary package to avoid this annoyance. Helmut
Re: build profile proposal: nogir (second try)
Hi Simon,
On Sun, Jan 21, 2024 at 03:24:25PM +, Simon McVittie wrote:
> > How annoying would it actually be to split this to a
> > different source package?
>
> Really quite annoying. [...]
You gave more than sufficient reason. I won't argue.
> If porters are interested in making bootstrap automatic despite cycles
> like this one, I think a better route would be to be able to have
> a list of suggested bootstrap steps and build-order considerations,
> either centralized in some sort of cross infrastructure or distributed
> among packages. I'd be fine with adding something like this to glib2.0,
> for example, if it helped:
>
> Bootstrap-Before: dbus, gobject-introspection
> Bootstrap-Build-Profiles: nogir, nocheck, noinsttest
We effectively tried the approach of encoding bootstrap-info into
individual packages with stage profiles and that was a bad idea. What
stages are needed can (and does) change. For instance, we no longer need
glibc's stage1 profile and go to stage2 directly. Hence, we try to more
and more use profiles that change a particular aspect of a package in an
obvious and isolated way and externally maintain how these are to be
combined into a successful bootstrap.
> Or, if we separated the nogir build profile that I'm proposing here into
> two, something like this:
>
> nogir-changing-content
> can change content: Y ("unreproducible"/"unsafe" profile)
> can change package set: Y
> nogir
> can change content: N ("reproducible"/"safe" profile)
> can change package set: Y
>
> would that allow automatic bootstrapping infrastructure to figure out
> that it was both safe and desirable to build glib2.0 with nogir?
I've considered this option for other profiles already and did not find
it appealing. Often times, you are interested in enabling the profile
without caring about whether it changes package contents, but such a
split would require you to figure out which of the profiles you need (or
simply both?).
More and more I think that merely documenting which instances of these
profiles are reproducible would be a better approach. I've had this
float as a vague idea since a while:
XS-Reproducible-Profiles: nogir
It's a promise that a source package can issue about a subset of the
profiles it supports. It bears some similarity to "Multi-Arch: foreign"
in the sense that both are promises on how the interface behaves. In
particular, such a declaration would be machine-checkable. We could
simply run an autobuilder that verifies whether such declarations are
practically correct (on amd64).
Bootstrappers do not really need that separation into two different
profile names that you propose. Having the information of which profiles
are reproducible in which source packages (and which packages get
disabled when enabling the profile), is what is needed.
So this is what I prefer, but it still comes at a cost. We're up for
changing lots of packages to declare these headers. And we're up for
setting QA to verify these. I fear I cannot provide the capacity to do
all of this and hence I have not pushed this forward.
Manually ordering glib2.0 in the bootstrap tooling may be annoying, but
that's about it. It still is way less work than any of the alternatives.
> (I infer that there must be some sort of infrastructure that knows that
> it's safe to build packages with "nocheck,noinsttest", otherwise glib2.0
> and dbus are already in a cyclic dependency for their test suites.)
Not really. nocheck and noinsttest are issued by default and simply
assumed to do the right thing in all cases.
> [...] I'm sorry if that's
> causing extra work for your use-case.
Yes, that's causing extra work on my side, but that extra work is really
low compared to the extra work on your side for the alternative. That
makes the choice rather obvious to me. Also having this advance warning
further lowers the cost on my side. You answered my question in way more
detail than expected. Thank you.
Helmut
