On Sat, 4 Nov 2023 at 02:02, Guillem Jover <[email protected]> wrote:
> Hi! > > On Thu, 2023-11-02 at 15:27:54 +0000, Michael Hudson-Doyle wrote: > > On Tue, 31 Oct 2023 at 09:21, Guillem Jover wrote: > > > This can be used among other things to set up foreign chroots, by > > > running the host tools, so disallowing installation could be > > > problematic. Even though I guess there could be a warning about this, > > > or maybe it could be controlled through a force option, although both > > > seems like they could be disruptive. > > > Of course in such cases dpkg knows that something funny is going on and > > could suppress the warning itself. > > Right, also true. > > > I spent a few minutes trying to think hard about this and I honestly > don't > > think I can predict whether trying to prevent installation of > incompatible > > packages is worth it (after all one of the ways users could get into > > trouble would be moving an installed system to a different CPU and having > > binaries start to fail and obviously dpkg can't help there). > > > > One result of this thinking was: I had been thinking/assuming the issue > of > > which variants to consider would be apt configuration, but maybe dpkg > > configuration would make more sense (after all, --add-architecture is a > > parameter to dpkg). And in this case, dpkg could object when installing a > > variant that has not been configured. > > Yes, the "plan" has been to add support for run-time CPU attributes, > so that when adding a new arch, for example you can specify whether > that arch is runnable, which could help dpkg decide whether to allow > by default to install M-A:foreign packages. > Ah. Would this be a change to /var/lib/dpkg/arch or an additional file or ...? > I guess this is similar, so such future interface should probably take > this into account as something to support too. Will check where this > is tracked and add a note to it. > Did you find this place? :) > And of course that is fine as a guardrail, but if a user hit that out > of running a frontend, then that would already be too late, which to > me means that frontends need to be aware of this too (and not pass > packages that dpkg would/could/might refuse to install), when deciding > what to pass to dpkg. > Good point. > But in any case, as you say, this currently would not be worse than > configuring a foreign arch, installing some foreign package and trying > to run it, but it might make it potentially more common. And as > mentioned above the effecting layer this needs to be decided up seems > higher anyway (even if dpkg could provide the infra for it). > > > > If the only change in the package filename format is in the <arch> part > > > where we'd use a name which would otherwise be valid as an arch name > (so, > > > no weird symbols, or «-» separators that are not intended to split <os> > > > and <cpu> or similar), then using a name for the variant/ISA would be > > > fine. > > > Right. I think that (when possible pretending e.g. "amd64v3" is a > distinct > > architecture will generally make things easier. E.g. I think britney > > wouldn't need to know about the relationship between "amd64" and > "amd64v3". > > I guess that depends on whether the intention is to create a full > optimized archive, or just a partial overlay one. In the latter case > then it might need to know to be able to satisfy dependencies. > Maybe! Depends on details I think. > > > That would be one solution yes, which could give automatic bijective > > > mappings, although ideally with a machine-readable way to get at it, > > > which I'm not sure we have currently. > > > I think "gcc -Q --help=target | grep -e '^\s*-march'" is about as machine > > readable as it gets currently, for better or worse (mostly worse). > > That does not look very satisfactory, though. Agreed! > And llvm/clang does not support it. :/ > Ah I did not know that. > > > For example code in dpkg-dev > > > already runs «$CC -dumpmachine» to infer the host architecture to use > > > during builds. > > > > > > While using a triplet variation could be a way to do that, that would > > > require such triplet support for each variant/ISA, which tends to be > > > very painful to introduce if it's not there already, so I'd not > > > consider this specific way a viable option. > > > > I admit I'm not an expert on triplet intricacies but I think a new > triplet > > is not appropriate here (a bit like a new Debian architecture for a > > variant/ISA choice is not the right concept). > > We have i386 or arm (?) as (bad IMO) examples where the triplet can > define the arch baseline. The problem is that this requires updating > the GNU config.git upstream, and then getting that to trickle down into > every package that might be using autotools and not using autoreconf > at build time, or to even update triplet matches in configure scripts > and similar, which might be "acceptable" for a new arch, but seems > disproportionate for a new ISA, so yes, as mentioned I agree it's not > viable. > OK. Let's stop worrying about that then :) > > > On Thu, 2023-09-21 at 14:43:42 +1200, Michael Hudson-Doyle wrote: > > > > * Should the ISA influence the toolchain via toolchain defaults or > > > > dpkg-buildflags? > > > > * How is the default ISA for a buildd chroot selected? > > > > > > So the clear downsides of either modifying the default toolchain or > > > having to provide an additional one is that this seems pretty heavy > > > weight. Also because people might want to build optimized variants > > > locally w/o having to mess with their already existing toolchains. > > > (I'm not sure whether something going along the lines of > > > <https://git.hadrons.org/cgit/debian/fakecross.git> could be an > > > option, although as mentioned above, if that would imply new triplets, > > > then probably not.) > > > > > > So the easiest way might indeed be by controlling this via an envvar, > > > > DEB_HOST_ARCH_ISA? > > Yeah, that works, and follows the current DPKG_*_ARCH_ABI lead for > example. > > > > which dpkg-buildpackage could also setup internally via a new option, > > > say --arch-isa=amd64v3 or similar > > > --host-arch-isa would be more coherent I think. > > Ah absolutely! For some reason had --arch in mind as a valid option > (I only see it now in dpkg-scanpackages :D, or maybe I was thinking > about --host-isa :). > > > I guess one could add support for --target-host-arch-isa to build a > > toolchain that defaults to a particular ISA. But well. > > Yes, the ISA support in dpkg should be extensive enough (so that if > this needs to be supported in the toolchain, then it is possible). > > > So to summarise, here are the generic changes that I think need to be > made > > to src:dpkg to support variant ISAs as a thing: > > > > * add get_host_arch_isa() to Dpkg::Arch > > Yes (perhaps as mentioned below also just get_host_isa()). > > > * dpkg-gencontrol records DEB_HOST_ARCH_ISA into DEBIAN/control as > > ArchitectureIsa > > Probably better Architecture-Isa, otherwise the current automatic > case folding would make it end up as Architectureisa. > Heh OK. > > * dpkg-architecture emits DEB_HOST_ARCH_ISA and grows --host-arch-isa > flag > > Also DEB_BUILD_ARCH_ISA and DEB_TARGET_ARCH_ISA, and also grows a > --target-arch-isa (but I'm thinking whether the shorter --host-isa would > be nicer, for example the --match-bits are not spelled --match-arch-bits, > which would seem also a bit redundant). > > > * dpkg-buildpackage passes --host-arch-isa to dpkg-architecture > > Yes, but only when not the baseline. > I think what I meant here was that if you pass one of the --*-arch-isa flags dpkg-buildpackage, it gets passed along to dpkg-architecture (as --host-arch etc are). > * dpkg-genchanges should record the ISA in the changes file somehow I > > guess? > > Yes, also dpkg-genbuildinfo. Oh yeah. Although on a quick look dpkg-genbuildinfo gets the architecture from the filename... > This could be done either from the > envvars, or perhaps through the debian/files attributes support. But > given that this is supposedly build global (I think it would be rather > weird to end up with a .changes including say _amd64.deb and > _amd64v3.deb file references from the same build), Ah yes that's true. > then probably using > the envvar might be the better way. > > > * dpkg-deb records the ISA in the file name > > Yes. > > > Have I missed anything? > > Nothing else comes to mind right now (except what I might have already > added). > Well of course I wrote this question in before going back and adding the stuff about having this all be driven by dpkg --add-archiitecture isa amd64v3 or anything like that. So those changes probably need to be hashed out too? > > (Hmm does anything need to reject unknown values > > found in DEB_HOST_ARCH_ISA / --host-arch-isa? Probably...) > > I guess it indeed makes sense to define what ISAs are supported, and > either error out or warn and ignore such values. So there might be a > need to add something like a new data/isatable. > I notice that --add-architecture doesn't seem to do any checking. > > Conceptually slightly separately, it might make sense to add a build > > "feature" to Dpkg::Vendor::Debian to allow setting -march (and -mtune?) > > > > Then when we want to add support to an ISA, we add a little thing to > > set_build_features (in either Vendor::Debian or Vendor::Ubuntu or > wherever) > > that maps get_host_arch_isa() to values for the march-influencing > feature. > > Hmm, right, how to hook this. I'm not sure the current interface is good > enough to describe this via build flags features, because such new feature > area would expose arch-specific features. I have been thinking through > the build flags and will try to send a proposal/RFC to revamp parts of > it during the weekend. Did that happen? I didn't see if if so. > But I think the ISA stuff is better just handled > (at leas for now) directly by injecting whatever flags when the requested > ISA is different to the baseline. > Well that's easy enough too. Cheers & thanks for the continued thoughts, mwh
