Bug#894965: dpkg-architecture should stop warning about unset CC
On Thu, Apr 12, 2018 at 01:18:57PM +0200, Guillem Jover wrote:
> So the intention here has been, AFAICT, to try to make detecting
> cross-building:
>
> 1) kind of automatic, something just needs to set CC to the correct
> cross-toolchain and then dpkg would figure out the host arch.
That must have predated my involvement. Ever since I worked on cross
building the canonical way to specify the host architecture has been
through dpkg-architecture -a$arch.
> 2) perform some kind of consistency and sanity check, to make
> sure the compiler architecture matches the one requested.
We don't even have widespread use of the CC variable. The clang people
essentially gave up on using it and simply divert gcc for their test
builds. Most of the working cross builds never set CC explicitly, but
the build system figures it out by other means (e.g. ./configure derives
it from $ac_tool_prefix, which comes from --host, which comes from
${DEB_HOST_GNU_TYPE}). In fact dpkg-architecture is happy with CC=true,
so any sanity check that is performed is not in fact sane.
> It is certainly true what you point out, and also that there are many
> other compilers, so the relationship between a C compiler and the
> resulting object installed might not match at all. Aslo even if dpkg
> itself set those envvars, there is no guarantee the debian/rules would
> use them. And I think the common way to request cross-compilation is
> via the dpkg-buildpackage -a argument nowadays.
>
> I guess given that cross-compilation has always been a bit rough on the
> edges, there's not been a clear direction how to expose cross-building
> interfaces until the above helpers gained support for it.
I must have been late to the party to the point where dpkg-architecture
has become the defacto interface.
> Removing the warning feels in a way like removing a safety check though,
> because while doing dpkg-buildpackage -afoo will emit it and it will
> be wrong, if the user specified an unmatched CC and -a, then bad
> things can happen. But if people cross-building have gotten used to
> ignore the warning anyway, then I'm not sure there's much point about
> it anyway.
There is no safety gained through this warning. It is emitted in the
common case, often worked around, it's like an annoying popup and the
user is trained to ignore it.
> If we'd go with removing the warning, I'd also have to question whether
> getting the architecture from CC or gcc is the correct thing to do, and
> whether we should instead expect the user to supply the arch, and
> simply validate that.
I don't think setting the architecture from CC is a good idea. If you
cross build with clang, you set "CC=clang -target
x86_64-unknown-linux-gnu". How do you parse it then?
> Something else that is related to this, and which I was pondering about
> and was kind of a blocker for fixing this or not, was whether to make
> dpkg-buildpackage set the buildtools variables itself, which would remove
> the warning altogether. This was tied in with the thread on d-d about
> the canonical entry point, and whether we'd still support debian/rules
> as that user entry point.
We have around 2000 source packages that build without dpkg setting it.
I don't see a compelling reason to change that interface. The present
way works. You can "dpkg-architecture -a$somearch -c ./debian/rules
binary-arch" and expect it to work.
> Another option, that might be more robust, would be to add some checks
> in dpkg somewhere, for whether the binary objects match the built
> arch (this is partially done in dpkg-shlibdeps, but I don't think it's
> complete or robust enough). This would of course trip over the abuse
> done by multilib packages where the contained objects do not match the
> .deb arch.
In practise, dh_strip performs that check. Quite a number of debhelper
packages just build for the build architecture and fail dh_strip, so we
already have this sanity check for the vast majority of packages. My
cross buildd performs the validation outside the build and rarely
catches a non-debhelper package.
To me, all of this sounds like we should just remove this warning
immediately with no replacement. It does no good, only harm.
Helmut
Bug#894965: dpkg-architecture should stop warning about unset CC
Control: forcemerge 644664 -1 [ More of a brain dump than anything else. ] Hi, On Thu, 2018-04-05 at 17:46:18 +0200, Helmut Grohne wrote: > Package: dpkg-dev > On Thu, Apr 05, 2018 at 04:28:48PM +0200, to...@tuxteam.de wrote: > > - Need to set CC > > > > Yes, the warning was clear enough, but I was a bit surprised > > I had to set CC like so: > > The warning you are talking about likely is this one: > > | dpkg-architecture: warning: specified GNU system type $DEB_HOST_GNU_TYPE > does not match CC system type $DEB_BUILD_GNU_TYPE, try setting a correct CC > environment variable > > I think this warning is obsolete as we assume that the packaging sets up > CC. For most cases, it actually happens automatically: > * dh_auto_build passes it to make. > * If you include /usr/share/dpkg/buildtools.mk in debian/rules, it'll >be correct. > * For many build systems, setting CC is not necessary and >dh_auto_configure does the right thing anyway. > > Telling the builder to set up CC is not needed at all, at least for 2000 > source packages and most of the failing ones don't fail due to missing > CC. > > Heck in many cases, I've seen and introduced workarounds like "CC=true > dpkg-architecture ..." to make it stop being noisy. > > Guillem, can we just remove the warning? So the intention here has been, AFAICT, to try to make detecting cross-building: 1) kind of automatic, something just needs to set CC to the correct cross-toolchain and then dpkg would figure out the host arch. 2) perform some kind of consistency and sanity check, to make sure the compiler architecture matches the one requested. It is certainly true what you point out, and also that there are many other compilers, so the relationship between a C compiler and the resulting object installed might not match at all. Aslo even if dpkg itself set those envvars, there is no guarantee the debian/rules would use them. And I think the common way to request cross-compilation is via the dpkg-buildpackage -a argument nowadays. I guess given that cross-compilation has always been a bit rough on the edges, there's not been a clear direction how to expose cross-building interfaces until the above helpers gained support for it. Removing the warning feels in a way like removing a safety check though, because while doing dpkg-buildpackage -afoo will emit it and it will be wrong, if the user specified an unmatched CC and -a, then bad things can happen. But if people cross-building have gotten used to ignore the warning anyway, then I'm not sure there's much point about it anyway. If we'd go with removing the warning, I'd also have to question whether getting the architecture from CC or gcc is the correct thing to do, and whether we should instead expect the user to supply the arch, and simply validate that. Something else that is related to this, and which I was pondering about and was kind of a blocker for fixing this or not, was whether to make dpkg-buildpackage set the buildtools variables itself, which would remove the warning altogether. This was tied in with the thread on d-d about the canonical entry point, and whether we'd still support debian/rules as that user entry point. Another option, that might be more robust, would be to add some checks in dpkg somewhere, for whether the binary objects match the built arch (this is partially done in dpkg-shlibdeps, but I don't think it's complete or robust enough). This would of course trip over the abuse done by multilib packages where the contained objects do not match the .deb arch. Thanks, Guillem
Bug#894965: dpkg-architecture should stop warning about unset CC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, Apr 11, 2018 at 05:31:15PM +0100, Wookey wrote: > On 2018-04-06 11:54 +0200, to...@tuxteam.de wrote: [buildtools.mk] > > Hm. I can't find that file. Apt-file (and packages.debian.org) can't > > either? > > It's part of dpkg-dev (but was typoed: /usr/share/dpkg/buildflags.mk ) Thanks. This one shows on my radar :-) > > > * For many build systems, setting CC is not necessary and > > >dh_auto_configure does the right thing anyway. > > > > Yes, but how do I find out why it is failing in my case? > > Is it actually failing if you don't set CC: or is it just spitting out > a confusing message? I somehow dreamt it failed: it didn't (I already retracted in another mail). > Setting DH_VERBOSE can help, or using make -d (or see man make '-debug' for > more details). To see when the message is being issued. But thanks for that: duly noted. > > the build system *was* using the wrong compiler, i.e. x86_64 instead > > of armhf. > > OK, so that's a useful error then? Is the problem that it's not gone > away now that you are in fact setting the right one? It should do (and > just did in my quick test). But I guess if the arch is set first, and > CC later (as is probably normal in most rules files driven by > dpkg-buildpackage -a) then you will always see this warning, where > it's not really very helpful. No: the "wrong compiler" problem disappeared, so I assume I was doing something wrong. As far as I am concerned, the error isn't useful anymore. Thanks a lot for your help! Cheers - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlrOWs4ACgkQBcgs9XrR2kas4gCfRLUioB8pBu+bCtQ0QDeri3Cw V/8AnA0p1ahMGTBiF1QJRORQzfM0XaED =4dfQ -END PGP SIGNATURE-
Bug#894965: dpkg-architecture should stop warning about unset CC
On 2018-04-06 11:54 +0200, to...@tuxteam.de wrote: > On Thu, Apr 05, 2018 at 05:46:18PM +0200, Helmut Grohne wrote: > > > I think this warning is obsolete as we assume that the packaging sets up > > CC. For most cases, it actually happens automatically: > > * dh_auto_build passes it to make. > > * If you include /usr/share/dpkg/buildtools.mk in debian/rules, it'll > >be correct. > > Hm. I can't find that file. Apt-file (and packages.debian.org) can't > either? It's part of dpkg-dev (but was typoed: /usr/share/dpkg/buildflags.mk ) > > * For many build systems, setting CC is not necessary and > >dh_auto_configure does the right thing anyway. > > Yes, but how do I find out why it is failing in my case? Is it actually failing if you don't set CC: or is it just spitting out a confusing message? Setting DH_VERBOSE can help, or using make -d (or see man make '-debug' for more details). To see when the message is being issued. > the build system *was* using the wrong compiler, i.e. x86_64 instead > of armhf. OK, so that's a useful error then? Is the problem that it's not gone away now that you are in fact setting the right one? It should do (and just did in my quick test). But I guess if the arch is set first, and CC later (as is probably normal in most rules files driven by dpkg-buildpackage -a) then you will always see this warning, where it's not really very helpful. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#894965: dpkg-architecture should stop warning about unset CC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, Apr 06, 2018 at 11:54:22AM +0200, to...@tuxteam.de wrote: > On Thu, Apr 05, 2018 at 05:46:18PM +0200, Helmut Grohne wrote: [...] > > I think this warning is obsolete as we assume that the packaging sets up > > CC. For most cases, it actually happens automatically: > > * dh_auto_build passes it to make. > > * If you include /usr/share/dpkg/buildtools.mk in debian/rules, it'll > >be correct. [...] > Yes, but how do I find out why it is failing in my case? I take that back. It is not failing in my case. The warning does actually seem obsolete, as Helmut says. Sorry for the noise - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlrHjBwACgkQBcgs9XrR2kY4PQCeJH/p2TRCi3wLE7Dr/MkWBQpb RV0An2iqUsQThlLt5tkSvUkX0xm5YWB4 =JATf -END PGP SIGNATURE-
Bug#894965: dpkg-architecture should stop warning about unset CC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Apr 05, 2018 at 05:46:18PM +0200, Helmut Grohne wrote: Hi, Helmut -- and sorry for taking the CC off the bug: if it feels rude to you, double sorry, but I felt queasy to spam the bug itself with my newbie questions: I already did, perhaps once too much :) > Package: dpkg-dev > > On Thu, Apr 05, 2018 at 04:28:48PM +0200, to...@tuxteam.de wrote: > > - Need to set CC > > > > Yes, the warning was clear enough, but I was a bit surprised > > I had to set CC like so: > > The warning you are talking about likely is this one: > > | dpkg-architecture: warning: specified GNU system type $DEB_HOST_GNU_TYPE > does not match CC system type $DEB_BUILD_GNU_TYPE, try setting a correct CC > environment variable Yes, exactly. > I think this warning is obsolete as we assume that the packaging sets up > CC. For most cases, it actually happens automatically: > * dh_auto_build passes it to make. > * If you include /usr/share/dpkg/buildtools.mk in debian/rules, it'll >be correct. Hm. I can't find that file. Apt-file (and packages.debian.org) can't either? > * For many build systems, setting CC is not necessary and >dh_auto_configure does the right thing anyway. Yes, but how do I find out why it is failing in my case? I'm reading now https://wiki.debian.org/Multiarch, that will take a while anyway, but FWIW, such an error message can be helpful -- it was in my case [1] (if it doesn't interfere with the normal course of things: you seem to suggest it does). Cheers & thanks [1] the build system *was* using the wrong compiler, i.e. x86_64 instead of armhf. - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlrHQ84ACgkQBcgs9XrR2kaqzACfUnd+ISZbpD8LXNir9Zzrtg0b EpYAnj+r+FZsjGDwuS2bjHFAsFp3fVmW =kA50 -END PGP SIGNATURE-
Bug#894965: dpkg-architecture should stop warning about unset CC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Apr 05, 2018 at 05:46:18PM +0200, Helmut Grohne wrote: > Package: dpkg-dev > > On Thu, Apr 05, 2018 at 04:28:48PM +0200, to...@tuxteam.de wrote: > > - Need to set CC > > > > Yes, the warning was clear enough, but I was a bit surprised > > I had to set CC like so: > > The warning you are talking about likely is this one: > > | dpkg-architecture: warning: specified GNU system type $DEB_HOST_GNU_TYPE > does not match CC system type $DEB_BUILD_GNU_TYPE, try setting a correct CC > environment variable Yes, this one exactly (variables instantiated, of course). > I think this warning is obsolete as we assume that the packaging sets up > CC. For most cases, it actually happens automatically: > * dh_auto_build passes it to make. > * If you include /usr/share/dpkg/buildtools.mk in debian/rules, it'll >be correct. > * For many build systems, setting CC is not necessary and >dh_auto_configure does the right thing anyway. > > Telling the builder to set up CC is not needed at all, at least for 2000 > source packages and most of the failing ones don't fail due to missing > CC. This would be a pity: in this case, it helped me to work around the issue (caused by my own blunder, most probably, but hey...) and tipped me off in the right direction. Without this message, I'd perhaps still be barking up wrong trees... > Heck in many cases, I've seen and introduced workarounds like "CC=true > dpkg-architecture ..." to make it stop being noisy. Yes, if this warning shows up even with a correct debian/rules it should be silenced, I get that. I'll try to understand a bit more before I come back, but ideally such a message might still be there to catch poor souls who don't know exactly what they are doing (as me). Cheers - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlrHHlEACgkQBcgs9XrR2kYb9wCff7uFZEH/zJ0kYYsJedAI5z1y z6EAn3NVTRB3sIydOhFiL0ihPE9v+5cs =E5Ht -END PGP SIGNATURE-
Bug#894965: dpkg-architecture should stop warning about unset CC
Package: dpkg-dev On Thu, Apr 05, 2018 at 04:28:48PM +0200, to...@tuxteam.de wrote: > - Need to set CC > > Yes, the warning was clear enough, but I was a bit surprised > I had to set CC like so: The warning you are talking about likely is this one: | dpkg-architecture: warning: specified GNU system type $DEB_HOST_GNU_TYPE does not match CC system type $DEB_BUILD_GNU_TYPE, try setting a correct CC environment variable I think this warning is obsolete as we assume that the packaging sets up CC. For most cases, it actually happens automatically: * dh_auto_build passes it to make. * If you include /usr/share/dpkg/buildtools.mk in debian/rules, it'll be correct. * For many build systems, setting CC is not necessary and dh_auto_configure does the right thing anyway. Telling the builder to set up CC is not needed at all, at least for 2000 source packages and most of the failing ones don't fail due to missing CC. Heck in many cases, I've seen and introduced workarounds like "CC=true dpkg-architecture ..." to make it stop being noisy. Guillem, can we just remove the warning? Helmut
