Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
On Thu, 04 Jan 2024 at 09:54:52 +0100, Helmut Grohne wrote: > I am not sure that you are the one who should express a qemu dependency. Discussion of whether gobject-introspection has the ideal dependencies seems off-topic for a release.d.o bug, so I've sent a reply to #1030223, dropping the release and apt teams from cc (but keeping -cross). Please follow up there with any further g-i-specifics. The part of this issue that was relevant for release.d.o was: given the current metadata of g-i_1.78.1-9, was britney2 correct to think its deps are satisfiable? And the answer was: no, it was a bug that britney2 disagreed with apt on this. According to the experimental pseudo-excuses, it seems that Paul's fix for that bug was successful, and the fixed version of britney2 is now happy to (pretend to) migrate g-i_1.78.1-9 if its autopkgtests pass. Thanks, smcv
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
On Sat, Jan 06, 2024 at 01:20:11PM +0100, Paul Gevers wrote: > We're not there yet, so please hold your horses. Although I tend to think we > should allow this too for the use cases you describe. So unless it's really > the intent of a (source) package or small (source) package set to be meant > to be used in a multi architecture environment I think we should demand that > dependencies are not be exclusively fulfilled by packages from another > architecture (:any is OK, :$arch is not). So indeed, each should require > manual review. While writing this that *could* mean that britney2 grows > support for cross-architecture dependencies while still blocking them if not > forced. I second this. I think we are already issuing a little too many :native and :any annotations that occasionally fire back (when changing M-A:no to M-A:foreign or M-A:allowed to M-A:same). Allowing :$arch for a reviewed set enables a few useful corner cases and avoids use where it is not appropriate. Before we drop -$arch-cross packages, we should consult with Matthias though. I think he has more reasons for them than britney2. One of them is that we can perform basic cross compilation to non-release architectures using only release-architecture packages. If we were to drop them, I'm not sure how gcc-$VER-cross-ports could exist. Helmut
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Quoting Paul Gevers (2024-01-06 13:20:11) > Thanks for being elaborate in your reply, it matches what I was thinking. (I > wasn't aware of the other examples though). there are certainly more examples. For example I maintain the package box64 which allows running amd64 binaries on arm64 but requires amd64 libc to operate. Because pkg:amd64 doesn't work for britney, I used this: Depends: libgcc-s1:amd64 | libgcc-s1-amd64-cross, libstdc++6:amd64 | libstdc++6-amd64-cross I had to patch the software to also look into the paths that libgcc-s1-amd64-cross and libstdc++6-amd64-cross use to make this work. Those two packages are Architecture:all but they do contain amd64 shared libraries. Helmut probably has a much better idea whether, in an ideal world, Arch:all packages like libgcc-s1-amd64-cross should go away and be replaced by corresponding architecture-qualified dependencies on the architecture-specific libraries. Thanks! cheers, josch signature.asc Description: signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Hi Simon On 06-01-2024 12:48, Simon McVittie wrote: On Sat, 06 Jan 2024 at 10:16:28 +0100, Paul Gevers wrote: If an explicit dependency on steam-libs:i386 would be valid, I'd be happy to use that, and remove the steam-libs-i386 binary package as redundant. We're not there yet, so please hold your horses. Although I tend to think we should allow this too for the use cases you describe. So unless it's really the intent of a (source) package or small (source) package set to be meant to be used in a multi architecture environment I think we should demand that dependencies are not be exclusively fulfilled by packages from another architecture (:any is OK, :$arch is not). So indeed, each should require manual review. While writing this that *could* mean that britney2 grows support for cross-architecture dependencies while still blocking them if not forced. packages being blocked for useful use cases (that we could hint through, but that britney2 would consider non-installable, so not protected from then on) and ... I think this bug report is one of only a couple over the years that requested anything on this front I specifically ment these sentences in the broader discussion we started having. This bug #1059929 involving gobject-introspection_1.78.1-9 is different from things like steam-installer and nss-mdns: Ack. I consider that just a bug in britney2: if apt, dpkg and dose3 allow this, so should britney2. My previous message was about the more generic case (including :$arch qualifiers instead of only :any (this bug being about :any on virtual packages)). in the steam-installer case I had to ask the release team (a while ago) to apply some force to work around a known limitation in britney2, but in the gobject-introspection case, my hope is that it can be resolved (possibly by a bug fix in britney2, or possibly by changing gobject-introspection) without forcing the installability check to be ignored. Absolutely. Thanks for being elaborate in your reply, it matches what I was thinking. (I wasn't aware of the other examples though). Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
On Sat, 06 Jan 2024 at 10:16:28 +0100, Paul Gevers wrote: > I guess there are exceptions we could accept like from > src:steam-installer (which doesn't use :i386 or :amd64 at the moment if I'm > correct) src:steam-installer avoids using :i386 in dependencies because I was under the impression that explicitly naming an architecture like that wasn't supported/allowed. Instead, steam-installer:amd64 Depends on steam-libs-i386, which only exists in the i386 Packages file (and is M-A: foreign so that it can satisfy the dependency from an amd64 package). I thought this was the standard workaround for something in the stack (apt? dpkg? the multiarch spec?) not allowing saying what I actually mean, which is: steam-installer:amd64 Depends on both steam-libs (implicitly :amd64) and steam-libs:i386. nss-mdns and the NVIDIA drivers both used this technique in the past, and Wine still does (it's called wine32 rather than wine-i386 but the principle is the same). If an explicit dependency on steam-libs:i386 would be valid, I'd be happy to use that, and remove the steam-libs-i386 binary package as redundant. Because it currently uses a lockstep dependency, I think we'd have to relax it to >=, and then keep it as a transitional package until after trixie. > packages being blocked for useful use cases (that we could hint > through, but that britney2 would consider non-installable, so not protected > from then on) I agree that explicit cross-architecture dependencies like the ones in steam-installer, nss-mdns and nvidia-graphics-drivers are quite rare, and it seems fine for them to need some manual intervention. The only use cases I know of are for proprietary i386 binaries that we can't just recompile as pure amd64 (like Steam and whatever Windows program you want to run via wine32), or for packages that support those (wine32 itself, graphics drivers, NSS plugins and so on). Maybe if cross-architecture dependencies were less of a special case, we might see a bit more use of this when cross-compiling (gcc-aarch64-linux-gnu Depends libc6-dev:arm64, making libc6-dev-arm64-cross unnecessary?) or for firmware for coprocessors (like if your x86 machine has a peripheral with a riscv64 processor that can run ordinary riscv64 code). > I think this bug report is one of only a couple over the years > that requested anything on this front This bug #1059929 involving gobject-introspection_1.78.1-9 is different from things like steam-installer and nss-mdns: in the steam-installer case I had to ask the release team (a while ago) to apply some force to work around a known limitation in britney2, but in the gobject-introspection case, my hope is that it can be resolved (possibly by a bug fix in britney2, or possibly by changing gobject-introspection) without forcing the installability check to be ignored. Yes, the dependencies are meant to be cross-satisfiable (and the package would be a lot simpler if that wasn't the goal), but they are also meant to be more trivially satisfiable in a single-architecture scenario. It shouldn't matter for this particular use-case whether you can *actually* cross-compile using gobject-introspection, because the scenario that I'm asking britney2 to evaluate when it considers migrating gobject-introspection is whether it's installable within a limited packaging universe that contains only :amd64 and :all packages - which is something that apt and dpkg are happy with. smcv
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Hi, On 06-01-2024 08:21, Johannes Schauer Marin Rodrigues wrote: I think it's a bit more complicated than that. Currently, the tool creates 8624 package relationships. If I remember correctly, britney is unable to analyze cross-architecture relationships? At least by lack of implementation. But thinking about pure cross-architecture relations (I mean those that are *only* satisfied using multiple architectures) a bit, what is it that we'd want at the archive level? I guess there are exceptions we could accept like from src:steam-installer (which doesn't use :i386 or :amd64 at the moment if I'm correct) or src:wine (which only uses it in alternatives and IIUC could equally well use :any), but do we really want to allow pure cross-architecture relations in general? I don't think so. What kind of complexity would that add for architecture qualification and efforts to remove an architecture from the archive later on? (steam-installer if it were using architecture qualifiers now would probably be handled somewhat because it could be accepted once manually and then afterwards it's accepted by britney2 because the non-installability doesn't go up). In that case that number goes down to 2352. One could reduce that number even further and only check those cases where apt, dpkg and dose3 agree on a solution but that would then rather be a documentation of the status quo than a list of the intended ground truth. Maybe it would make sense to analyze the archive and only add those cases that currently exist as real package relationships? As far as I can tell (I checked testing/main/source, testing/main/(amd64|i386) and testing/contrib/(amd64|i386) for (:i386|:amd64)) there is no package that does this for Depends or Build-Depends (excluding alternatives in src:wine and src:build-essential). So I think we can reduce it to :any in Depends and :any and :native in Build-Depends. Not sure how far your number goes down then. What the tool never received since its inception was somebody to look over these cases and write down what the ground-truth actually is supposed to look like. For the britney tests you likely want some kind of ground-truth and I fear that tool can give you the status quo but not necessarily the truth as intended by the spec. Ack. If that is fine for you, do you still want to add thousands of test-cases? Or do you want to hand-pick them? It depends on the route we take and what we envision as useful cases to support. What I want to avoid is two things, highest prio first: 1) something that we don't want to migrate migrates (I think the current *lack* of support achieves that mostly already) albeit *maybe* my current fix for this bug is going to change that unintentionally in the wrong direction. 2) (lots of) packages being blocked for useful use cases (that we could hint through, but that britney2 would consider non-installable, so not protected from then on) I think this bug report is one of only a couple over the years that requested anything on this front (I think all were from Simon), so I wonder how many (legitimate) use cases there are that people would want to use and don't because of lack of support on our side. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Quoting Paul Gevers (2024-01-05 20:15:22) > Thanks for reaching out. Thank Helmut for poking me in #debian-apt :) > For britney2, the Sources stanza would also be needed; then we could use this > to generate britney2 testcases. I created 10 of those yesterday by hand [1]. > > The simplest for of tests is a tree with > var/data/unstable/Sources > # i386 is the default archive of the testsuite, which can be overruled > # if that makes sense > var/data/unstable/Packages_i386 > var/data/testing/Sources (may be empty) > var/data/testing/Packages_i386 > expected > > expected is in Heidi format (if I understand correctly) of what britney2 > will allow to be in testing after the run, it will only migrate packages > that are installable. > > Would it make sense to you to generate a branch in that archive with a bunch > of tests that you know the answer too? I think it's a bit more complicated than that. Currently, the tool creates 8624 package relationships. If I remember correctly, britney is unable to analyze cross-architecture relationships? In that case that number goes down to 2352. One could reduce that number even further and only check those cases where apt, dpkg and dose3 agree on a solution but that would then rather be a documentation of the status quo than a list of the intended ground truth. Maybe it would make sense to analyze the archive and only add those cases that currently exist as real package relationships? What the tool never received since its inception was somebody to look over these cases and write down what the ground-truth actually is supposed to look like. For the britney tests you likely want some kind of ground-truth and I fear that tool can give you the status quo but not necessarily the truth as intended by the spec. If that is fine for you, do you still want to add thousands of test-cases? Or do you want to hand-pick them? Thanks! cheers, josch signature.asc Description: signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Control: tags -1 pending Hi, On 03-01-2024 20:40, Paul Gevers wrote: On 03-01-2024 20:22, Simon McVittie wrote: I think all of those are correct? I think that if apt allows you to install it, chances are that it's a britney2 bug. I'll try to debug it tomorrow. I have a first proposal for a fix in https://salsa.debian.org/release-team/britney2/-/merge_requests/89 Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Hi, Thanks for reaching out. On 05-01-2024 07:45, Johannes Schauer Marin Rodrigues wrote: It would generate the following two stubs (shortened here for brevity): Package: pkga Version: 1 Architecture: amd64 Depends: pkgc:any Multi-Arch: no Package: pkgb Version: 1 Architecture: amd64 Provides: pkgc Multi-Arch: allowed For britney2, the Sources stanza would also be needed; then we could use this to generate britney2 testcases. I created 10 of those yesterday by hand [1]. The simplest for of tests is a tree with var/data/unstable/Sources # i386 is the default archive of the testsuite, which can be overruled # if that makes sense var/data/unstable/Packages_i386 var/data/testing/Sources (may be empty) var/data/testing/Packages_i386 expected expected is in Heidi format (if I understand correctly) of what britney2 will allow to be in testing after the run, it will only migrate packages that are installable. Would it make sense to you to generate a branch in that archive with a bunch of tests that you know the answer too? Paul [1] https://salsa.debian.org/debian/britney2-tests/-/commit/5ab98de685e15a654227e9b188a48e44857f9d11 OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Hi, On Thu, 4 Jan 2024 21:04:57 +0100 Paul Gevers wrote: > [20:21:54] agreed, but britney2 doesn't handle :any on virtual > packages in any way (neither binary dep nor build dep) > [20:22:10] (and I'm not sure whether that's legal in any way) > [20:23:08] agreed > [20:23:28] which I'm fixing right now > [20:23:39] apparently there's builds with :native > [20:23:43] so I'll support that > [20:24:04] but I wanted to know what happens with :any (and > virtual B-D) > [20:25:07] well, this is not something that existed in the > archive before gobject-introspection, so we're about to extend what is > defined > [20:25:15] according to smcv this is legal to dpkg and apt > [20:25:35] but we know that resolvers > (dpkg/apt/dose/britney2/...) disagree on corner cases > [20:25:50] and I'm guilty of having written yet another > resolver in dumat > [20:26:26] britney2 just has to follow what dpkg and apt happen > to agree on > [20:26:45] or maybe even, what apt says > [20:27:01] if that works back in 2015 I wrote a small tool which is able to create artificial dependency situations involving two real packages and (optionally) a third virtual packages to make sure that the multi-arch implementations of apt, dpkg and dose3 agree with each other in all cases (spoiler: they do not). https://gitlab.mister-muffin.de/josch/deb-m-a-dep-check/ The tool is meant to make mass comparisons (it generates 8624 test cases) and thus its interface is a bit clunky but if what you want to do is to see whether apt, dose3 and dpkg agree on a situation where one package depends on a virtual package with :any which is provided by a real m-a: allowed package, then you would write this: ./check.sh binary pkgc amd64 amd64 no allowed depends pkgc:any It would generate the following two stubs (shortened here for brevity): Package: pkga Version: 1 Architecture: amd64 Depends: pkgc:any Multi-Arch: no Package: pkgb Version: 1 Architecture: amd64 Provides: pkgc Multi-Arch: allowed And yes, all three tools agree on this situation: it is satisfiable. If you make pkgb m-a:no, then all tools agree that the situation is unsatisfiable. We can do the same checks for pkga being a source package: ./check.sh source pkgc none amd64 none allowed depends pkgc:any Again, all three tools agree that this situation is satisfiable. Thanks! cheers, josch signature.asc Description: signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Hi Simon, Thanks for your work on gobject-introspection cross building! On Wed, Jan 03, 2024 at 07:22:26PM +, Simon McVittie wrote: > - gobject-introspection-little-endian:any, a virtual package provided > by g-i-bin, which is Multi-Arch: allowed > (experimentally, apt and dpkg both seem to be happy to assume that > this makes the gobject-introspection-little-endian virtual package > behave as though it was also Multi-Arch: allowed) Let me guess that this is the culprit. I also Cc apt maintainers for their input. > Or do I need to make gobject-introspection-bin Multi-Arch: foreign, > drop the :any from gobject-introspection-little-endian:any, and > replace the gobject-introspection-bin | qemu-user | qemu-user-static > dependency by python3 | qemu-user | qemu-user-static or similar? I am not sure that you are the one who should express a qemu dependency. When we reason about dependencies, we care about how they behave assuming that you can run them. Whether you can run an executable from a package or not is something that is not expressed in our package relationships. It's also rather difficult. Consider a few corner cases: * Some amd64 can run i386. * Most arm64, but not all, can run armhf. * You may operate in a chroot with some external qemu-binfmt and thus execute any arch. * You cannot run hurd-i386 on amd64 even in the presence of qemu-user. I probably have caused this in our discussion back in Cambridge where I suggested to you that having a dependency on qemu could be ok. Given the above, qemu quite likely needs more thought. > My goal here is that you can install gobject-introspection:amd64 on an > amd64 machine, or on any other little-endian machine that will be able to > cross-compile amd64 binaries and then run them by explicitly invoking them > via qemu-user, as discussed with Helmut Grohne at the recent Cambridge > miniDebconf. (It has to be little-endian because g-ir-inspect and similar > tools don't currently support byte-swapping fields in binary typelibs.) When we considered whether cross building should imply disabling tests, we went for "no, but yes by default". When you cross build a package for i386 on amd64, sbuild and pbuilder will automatically add nocheck to DEB_BUILD_OPTIONS and DEB_BUILD_PROFILES. However, you can opt out of this behaviour to really run tests despite performing a cross build. I think we need a similar mechanism for qemu integration. When we talked about this, I was having in mind (but probably didn't express this explicitly) that such qemu dependencies would happen in Build-Depends only. This is different from what you do here and has multiple implications: * Your satisfiability problem with britney2 probably goes away. * Every package that uses gobject-introspection needs to be modified for this. * We can annotate such qemu dependencies with a build profile e.g. . By default, such dependencies would only become active for cross builds, but the second profile enables you to skip them when you know that no emulator is required. Other than this, let me note that M-A:allowed always seemed a little annoying to me, because it makes an implementation detail visible to consumers. Whenever you think you need M-A:allowed, you may instead introduce a layer of indirection. In principle, you could add a real binary packages: gobject-introspection-any-endian with Arch:any M-A:foreign Depends:gobject-introspection-bin and architecture-dependent provides. Then you can just depend on gobject-introspection-little-endian without thinking about whether you have to add :any. Let me also note that the way you have gobject-introspection (the binary package) now fills a similar role to pkgconf/pkg-config and qt5-qmake as well as binutils-for-host and hopefully soon also gcc-for-host. That pattern seems to work out rather well. This probably isn't the final solution, but I hope my feedback moves you forward in some way. Helmut
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Hi Simon, On 03-01-2024 20:22, Simon McVittie wrote: I think all of those are correct? I think that if apt allows you to install it, chances are that it's a britney2 bug. I'll try to debug it tomorrow. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency
Package: release.debian.org Severity: normal User: release.debian@packages.debian.org Usertags: britney X-Debbugs-Cc: gobject-introspect...@packages.debian.org, debian-cr...@lists.debian.org gobject-introspection in experimental has this in https://release.debian.org/britney/pseudo-excuses-experimental.html#gobject-introspection: gobject-introspection (1.78.1-6 to 1.78.1-9) Migration status for gobject-introspection (1.78.1-6 to 1.78.1-9): BLOCKED: Rejected/violates migration policy/introduces a regression Issues preventing migration: gobject-introspection/amd64 has unsatisfiable dependency gobject-introspection/arm64 has unsatisfiable dependency Additional info: uninstallable on arch amd64, not running autopkgtest there uninstallable on arch arm64, not running autopkgtest there The gobject-introspection binary package *is* installable, and in fact I have it installed locally. Taking the amd64 version as an example, it depends on: - binutils-x86-64-linux-gnu:any, a real Multi-Arch: allowed package - gcc-x86-64-linux-gnu, a virtual package provided by gcc:amd64 - gobject-introspection-bin | qemu-user | qemu-user-static, where g-i-bin is a Multi-Arch: allowed package from the same source - gobject-introspection-little-endian:any, a virtual package provided by g-i-bin, which is Multi-Arch: allowed (experimentally, apt and dpkg both seem to be happy to assume that this makes the gobject-introspection-little-endian virtual package behave as though it was also Multi-Arch: allowed) - pkgconf, a real package - python3:any, a real Multi-Arch: allowed package I think all of those are correct? Or do I need to make gobject-introspection-bin Multi-Arch: foreign, drop the :any from gobject-introspection-little-endian:any, and replace the gobject-introspection-bin | qemu-user | qemu-user-static dependency by python3 | qemu-user | qemu-user-static or similar? My goal here is that you can install gobject-introspection:amd64 on an amd64 machine, or on any other little-endian machine that will be able to cross-compile amd64 binaries and then run them by explicitly invoking them via qemu-user, as discussed with Helmut Grohne at the recent Cambridge miniDebconf. (It has to be little-endian because g-ir-inspect and similar tools don't currently support byte-swapping fields in binary typelibs.) Thanks, smcv
