Re: Long-term mass bug filing for crossbuild support
Scribit Neil Williams dies 13/11/2007 hora 17:02: If you want to build an ARM toolchain to crossbuild for amd64 I'm not going to stop you but don't expect me to debug it!! But do your tools make it already possible for me to just ask for the build of toolchains for an arbitrary list of target architectures, from an arbitrary host architecture? I can well understand why you don't want the N*N-N packages or sets of packages in the official Debian archive, but it would be great if the tools make it possible to build them. There's also the fact that it could be useful for ports not in the released ones. In any case, thanks to all developers from Embdebian! That's a great project. Curiously, Pierre -- [EMAIL PROTECTED] OpenPGP 0xD9D50D8A signature.asc Description: Digital signature
Re: Long-term mass bug filing for crossbuild support
Pierre THIERRY wrote: Scribit Neil Williams dies 13/11/2007 hora 17:02: If you want to build an ARM toolchain to crossbuild for amd64 I'm not going to stop you but don't expect me to debug it!! But do your tools make it already possible for me to just ask for the build of toolchains for an arbitrary list of target architectures, from an arbitrary host architecture? No. Building a toolchain relies on various levels of support packages which may or may not be available for your choice. You would have to build those first. The tools make no promises about whether the choice is sensible, reasonable or actually going to build. Quite often, it will not build - you can't blame the tools if the underlying packages don't work in some weird combo. There is a vast gap between allowing the tools to support the theory and actually supporting real toolchain builds in practice. I don't think that it is worthwhile trying to attain the kind of arbitrary selections that you describe. I do want to actually get things done with the toolchains that we already have, I'm not going to spend the rest of my life building every permutation of every architecture combination for every compiler version and the rest. I'm not going to promise support for all possible permutations because that implies debugging bizarre versions. The tools don't rule it out, that's about the best I can offer. Not all permutations of toolchains will ever be possible to build, the tools cannot do anything about that. So, in theory, all architectures supported by dpkg-architecture (which is already far more than supported by Debian itself) can be used with the tools. In practice, the only reasonable choices are to install toolchains on i386, amd64 or powerpc. Even then, some toolchain packages will simply fail to build for certain host selections. Throw in requests for older compilers and even the regular selections will fail. I can well understand why you don't want the N*N-N packages or sets of packages in the official Debian archive, but it would be great if the tools make it possible to build them. Possible - yes. Practical ? maybe not. Supported? Definitely NO. There's also the fact that it could be useful for ports not in the released ones. Even porters don't need every possible permutation. Some combinations of --build and --host are just more practical than others. I see no point in extending the tools to support these in anything more than theory. In any case, thanks to all developers from Embdebian! That's a great project. You're welcome. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ signature.asc Description: OpenPGP digital signature
Re: Long-term mass bug filing for crossbuild support
Scribit Neil Williams dies 11/11/2007 hora 12:44: Emdebian supports amd64, i386 and powerpc as --build. Why aren't all architectures supported by Debian supported? Curiously, Pierre -- [EMAIL PROTECTED] OpenPGP 0xD9D50D8A signature.asc Description: Digital signature
Re: Long-term mass bug filing for crossbuild support
On Tue, 13 Nov 2007 17:34:45 +0100 Pierre THIERRY [EMAIL PROTECTED] wrote: Scribit Neil Williams dies 11/11/2007 hora 12:44: Emdebian supports amd64, i386 and powerpc as --build. Why aren't all architectures supported by Debian supported? ? Because I'm referring to --build, not --host ? The main point of cross building is to use powerful machines to build for less powerful machines. Trying to cross-build for amd64 on ARM is just being perverse. Building for ARM on amd64 or i386 or powerpc is fine. Also, cross-compiling means building toolchains for each supported permutation. By supporting three separate toolchain sets, we keep the workload of maintaining the toolchains within reasonable limits - i.e. 3*N where N is the number of architectures supported by Debian instead of N*N-N. All architectures supported by Debian are supported as --host as well as a few that are *not* supported by Debian (like armel and armeb) and ones that Debian used to support (like m68k) - various people are working on uclibc support for ARM as well as other architectures. Emdebian can support all those. Emdebian can support building for any architecture supported by dpkg-architecture and for which someone can build a toolchain. If you want to build an ARM toolchain to crossbuild for amd64 I'm not going to stop you but don't expect me to debug it!! :-) -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpUIxSi7ULTM.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
Hello Pierre, Emdebian supports amd64, i386 and powerpc as --build. Why aren't all architectures supported by Debian supported? Why would you want an ARM/MIPS/M68K build host if you can cross compile for them? Anyway, i believe all arches could be supported, but does not make much sense to support some arches and as we lack hardware for those arches and we lack human resources, we do not support them. Regards -- Héctor Orón
Re: Long-term mass bug filing for crossbuild support
On Sun, 11 Nov 2007 00:19:41 +0100 Hendrik Sattler [EMAIL PROTECTED] wrote: Am Samstag 10 November 2007 schrieb Neil Williams: emdebian-tools will support any build system that can be used in debian/rules. The build system should configure itself using only the data provided by dpkg-architecture but can use that information in whatever manner is needed to get the right values into the right variables. How do you determine the root dir of the target? ? What do you mean ? / is / 'target' is a misleading term in cross compiling - it is probably best to reserve the keyword 'target' as referring solely to the TARGET variable specified to gcc when building the cross compiler. 'target' in that context, has no specific meaning during the cross build itself. Emebian uses the terminology: --build == Big or Desktop (Build or builD) depending on your point of view. --host == handheld Emdebian supports amd64, i386 and powerpc as --build. My primary --host is ARM but others are working on other objectives, like mips, and other architectures are supported. The cross built package will install into /usr/ just as before because it is expected to be installed on the host device where / is correct. When using dpkg-cross, the library is rebuilt to install into: /usr/$triplet/lib/ and -dev packages into: /usr/$triplet/include/ where $triplet = `dpkg-architecture -qDEB_HOST_GNU_TYPE` When dpkg-buildpackage is called with the --arch option, dpkg-architecture is called to set the appropriate values. That makes the --host libfoo.so available to the cross compiler running on the --build by a mechanism of paths and symlinks. It is this support that is being rewritten during the migration of dpkg-cross back into dpkg. This /usr/$triplet/ is only used to provide the shared objects during the cross build - the final package looks in /usr/lib/ when loading the dynamic links at runtime. Currently, 'dpkg-buildpackage -a' is not fully functional. Emdebian uses wrapper code in 'emdebuild' that implements a temporary solution until bug 439979 is closed. It's likely not / so it can be about anywhere in the tree. Or does the build happens in a chroot automatically. Please read http://www.emdebian.org/ and then transfer this thread to [EMAIL PROTECTED] How and which environment variables are set to which values? man dpkg-buildpackage doesn't tell anything about those. man dpkg-architecture -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpmnWxjiUcP2.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
On Tue, Nov 06, 2007 at 10:50:51PM +0100, Mike Hommey wrote: AFAIK there's nothing to be fixed here. If you specify both --build and --host, autoconf sees this as a request for cross-compiling. Of course, invoking the native compiler as a cross-compiler should /generally/ give the same results as invoking it as gcc, so it's entirely possible that it doesn't cause problems on your packages. The autoconf documentation says only if you enter both and they're different will it enter cross compiling mode. FWIW, the dh_make samples set both. And it also has a bug against it to be changed: http://bugs.debian.org/423707 Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
Am Dienstag 06 November 2007 schrieb Neil Williams: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide Does Emdebian support upcoming cmake-2.6 cross-compile support or is this planned? See: http://www.cmake.org/Wiki/CMake_Cross_Compiling So when creating a cross-compiler, a corresponding cmake toolchain file should also be created. The right one should then noted somewhere (e.g. in an environment variable) when cross-compiling a debian packge so that the debian/rules file can give cmake the proper -DCMAKE_TOOLCHAIN_FILE= option. HS -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Sat, 10 Nov 2007 15:02:31 +0100 Hendrik Sattler [EMAIL PROTECTED] wrote: Am Dienstag 06 November 2007 schrieb Neil Williams: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide Does Emdebian support upcoming cmake-2.6 cross-compile support or is this planned? See: http://www.cmake.org/Wiki/CMake_Cross_Compiling emdebian-tools will support any build system that can be used in debian/rules. The build system should configure itself using only the data provided by dpkg-architecture but can use that information in whatever manner is needed to get the right values into the right variables. The crossbuild should require no more information than would be provided by running dpkg-buildpackage. At present, there is some wrapper code but that is destined to be removed. After all, the only difference between a normal build and a crossbuild is that the normal build uses: dpkg-buildpackage ... and the cross build uses: dpkg-buildpackage -a $arch ... That should be all that the package (or build system) needs to know - apart from architecture-specific flags like little-endian or big-endian etc. which are provided by dpkg-cross (the config files themselves are soon to migrate into dpkg-dev itself). So when creating a cross-compiler, a corresponding cmake toolchain file should also be created. Providing and updating the toolchain is already sufficiently complex. There are no plans to complicate things further by adding support for any build-system-specific toolchain anything. IMHO CMake should be configured (by the package or within itself) in such a way that it uses the dpkg-architecture data to provide all the data it needs, just like all the other build systems. A file in the home directory is not likely to be relevant or useful, rather I would expect such a hack to be a source of yet more bugs. Besides, there would appear to be nothing useful to put into such a file - the toolchain binaries are all in the $PATH and I see no apparent need for a lot of the extras that CMake tries to add. If a package doesn't build with 'dpkg-buildpackage -a' then I will consider that a bug in the package and if the bug is reproducible in other packages using the same build system, reassign to the build system. (Hence the current crossbuilt bug against CDBS). The right one should then noted somewhere (e.g. in an environment variable) when cross-compiling a debian packge so that the debian/rules file can give cmake the proper -DCMAKE_TOOLCHAIN_FILE= option. emdebian-tools is build-system agnostic. It should not and does not care about what is used. Any build system specifics need to be configured either in the build system itself or in the package debian/rules, just like every other package. Note that all build systems must also work within a chroot without any user-specific configuration (without a /home at all) and in this respect, Emdebian is no different to the autobuilders. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgp5zdBqBMdVe.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
Am Samstag 10 November 2007 schrieb Neil Williams: On Sat, 10 Nov 2007 15:02:31 +0100 Hendrik Sattler [EMAIL PROTECTED] wrote: Am Dienstag 06 November 2007 schrieb Neil Williams: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide Does Emdebian support upcoming cmake-2.6 cross-compile support or is this planned? See: http://www.cmake.org/Wiki/CMake_Cross_Compiling emdebian-tools will support any build system that can be used in debian/rules. The build system should configure itself using only the data provided by dpkg-architecture but can use that information in whatever manner is needed to get the right values into the right variables. How do you determine the root dir of the target? It's likely not / so it can be about anywhere in the tree. Or does the build happens in a chroot automatically. How and which environment variables are set to which values? man dpkg-buildpackage doesn't tell anything about those. HS -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
Neil Williams wrote: [snip] As noted elsewhere in this thread, --build can be specified alone but is usually only used for specialist builds for i686 on i386 etc. I fail to see the merit of proposing that packages add --build to the normal Debian build for no reason. One reason is that a guess via uname will lead to e.g. a mips64-linux build attempt on a 32 bit userland, just because the kernel is 64 bit. Generally speaking, such guesses are unreliable in multi-ABI environments. Thiemo -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, 06 Nov 2007 17:18:57 -0800 Russ Allbery [EMAIL PROTECTED] wrote: Neil Williams [EMAIL PROTECTED] writes: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) Depending on whether the package uses ./configure: ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) else CROSS= endif This is a bad suggestion and was apparently made without referring to the existing Debian instructions for how to handle packages that use Autoconf and friends. Your assumption is wrong, I'm afraid. For one reason or another, I spend a lot of time in the autofoo documentation. Must be crazy, yes, yes Please see the autotools-dev instructions, which are more detailed and which correctly recommend: DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) ifeq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) SYSTEM = --build $(DEB_HOST_GNU_TYPE) else SYSTEM = --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) endif # ... ./configure $(SYSTEM) Actually, /usr/share/doc/autotools-dev/README.Debian.gz differs from that by specifying $(confflags) instead of SYSTEM and using 'export'. I don't see what benefit arises from the only other difference, ?= instead of =. When cross-compiling, it isn't sensible to allow the value to be overridden elsewhere. The value is set in the cross-compiling tools via dpkg-architecture and should be read into debian/rules as such. As noted elsewhere in this thread, --build can be specified alone but is usually only used for specialist builds for i686 on i386 etc. I fail to see the merit of proposing that packages add --build to the normal Debian build for no reason. I don't see why this makes the proposed change bad. Please explain why ?= is better than = in this specific instance and why I should use SYSTEM instead of CROSS (when the recommended one is $(confflags) anyway) or add --build to every package instead of leaving the variable as empty. AFAICT there is no material reason to use the autofoo recommended layout and no justification for calling my layout bad merely because of minor changes in variable names. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpaYKXJiKC1Q.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
On Wed, 07 Nov 2007 09:43:34 -0800 Russ Allbery [EMAIL PROTECTED] wrote: Neil Williams [EMAIL PROTECTED] writes: Russ Allbery [EMAIL PROTECTED] wrote: This is a bad suggestion and was apparently made without referring to the existing Debian instructions for how to handle packages that use Autoconf and friends. Your assumption is wrong, I'm afraid. For one reason or another, I spend a lot of time in the autofoo documentation. Must be crazy, yes, yes Well, then I'm even more mystified as to why you contradicted those instructions and told people not to pass --build to configure for non-cross-compile builds. So it is preferable for me to add --build to native builds even in a patch that is meant to only affect the cross build? I'm sure some maintainers will query why I'm setting --build outside the crossbuild. If that is how it is to be done then that is OK with me - I was just trying to keep the patch to the basic cross build requirements. After all, I see a mass bug filing as almost an NMU on each package - change as little as necessary. Actually, /usr/share/doc/autotools-dev/README.Debian.gz differs from that by specifying $(confflags) instead of SYSTEM and using 'export'. Sorry, you're entirely correct, and I shouldn't paste from one of my packages instead of from the actual documentation. (And I should go add export, as that was there for a reason. Mea culpa.) No problem. What is the reason for 'export' ? -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpOFBdDfDMXQ.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
On Wed, Nov 07, 2007 at 09:43:34AM -0800, Russ Allbery wrote: Neil Williams [EMAIL PROTECTED] writes: Russ Allbery [EMAIL PROTECTED] wrote: This is a bad suggestion and was apparently made without referring to the existing Debian instructions for how to handle packages that use Autoconf and friends. Your assumption is wrong, I'm afraid. For one reason or another, I spend a lot of time in the autofoo documentation. Must be crazy, yes, yes Well, then I'm even more mystified as to why you contradicted those instructions and told people not to pass --build to configure for non-cross-compile builds. Please see the autotools-dev instructions, which are more detailed and which correctly recommend: DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) ifeq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) SYSTEM = --build $(DEB_HOST_GNU_TYPE) else SYSTEM = --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) endif Actually, /usr/share/doc/autotools-dev/README.Debian.gz differs from that by specifying $(confflags) instead of SYSTEM and using 'export'. Sorry, you're entirely correct, and I shouldn't paste from one of my packages instead of from the actual documentation. (And I should go add export, as that was there for a reason. Mea culpa.) How about making the snippet available in a .mk file that could just be included, so that whenever there is an enhancement, everyone can take advantage of it (but then, the snippet would need extra care to stay compatible) ? Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Wed, 7 Nov 2007 12:05:55 -0500 Steve Langasek [EMAIL PROTECTED] wrote: No, it's used to prevent upstream packages from *wrongly* building for i686 on the i386 architecture, as I've now stated several times in this thread. But why should that be added as part of a fix for cross-building? It isn't relevant. If a package does not already use --build in a Debian build, why change it? It's not up to me. It's bad because it's worse for the *common* case that the existing autotools-dev recommendation. This is not a minor change in variable names, it's missing support for correct enforcement of the target architecture in the non-cross-building case that the autotools-dev recommendation has already addressed. It's not up to me to enforce the non-cross-building case. These bugs are about cross-building. Some packages use --build already and those are unchanged, I just add --build and --host when cross compiling. Other packages use neither --build nor --host and I set both only when cross-building. I don't see why a cross-building patch should specify --build for the native build. I'm sorry for failing to notice this lapse at first glance, I assumed you were at least taking advantage of the established best practices instead of inventing solutions in a vacuum. There isn't a vacuum, I'm quite familiar with the autotools docs - I just don't see why a cross-building patch should tinker with --build for the native build if the package does not use it already (and if it does, I just leave it in place). -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpePgVOfq9dU.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
Neil Williams [EMAIL PROTECTED] writes: Russ Allbery [EMAIL PROTECTED] wrote: This is a bad suggestion and was apparently made without referring to the existing Debian instructions for how to handle packages that use Autoconf and friends. Your assumption is wrong, I'm afraid. For one reason or another, I spend a lot of time in the autofoo documentation. Must be crazy, yes, yes Well, then I'm even more mystified as to why you contradicted those instructions and told people not to pass --build to configure for non-cross-compile builds. Please see the autotools-dev instructions, which are more detailed and which correctly recommend: DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) ifeq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) SYSTEM = --build $(DEB_HOST_GNU_TYPE) else SYSTEM = --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) endif Actually, /usr/share/doc/autotools-dev/README.Debian.gz differs from that by specifying $(confflags) instead of SYSTEM and using 'export'. Sorry, you're entirely correct, and I shouldn't paste from one of my packages instead of from the actual documentation. (And I should go add export, as that was there for a reason. Mea culpa.) I don't see what benefit arises from the only other difference, ?= instead of =. When cross-compiling, it isn't sensible to allow the value to be overridden elsewhere. The value is set in the cross-compiling tools via dpkg-architecture and should be read into debian/rules as such. Sometimes humans know better than tools, so I like to leave it open so that humans can override if they need to. As noted elsewhere in this thread, --build can be specified alone but is usually only used for specialist builds for i686 on i386 etc. I fail to see the merit of proposing that packages add --build to the normal Debian build for no reason. Do we really need to rehash this? This has been discussed at some length on debian-devel in the past. There are a wide variety of reasons why always being explicit about the build architecture is better, from upstream brokenness to upstream defaults that differ from how Debian handles architectures to getting the right canonical triplet encoded into things. The reason why these instructions are captured in autotools-dev is so that we don't have to go over this again each time this comes up. It's this problem that leads me to specifically call your instructions bad, based on the previous discussions on debian-devel. Depending on the version of the Autotools involved and the architecture involved, following your instructions rather than the instructions in autotools-dev may violate Policy 4.3 and 11.1 by encoding the wrong architecture triplet. The autotools-dev documentation points this out. As near as I can tell, the autotools-dev instructions already handle the cross-build requirements from the message at the start of this thread, and they've already previously been hashed out and gotten some consensus. I'd rather just stick with recommending them to people rather than coming up with a new and different recommendation and going through this again. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
Neil Williams wrote: As noted elsewhere in this thread, --build can be specified alone but is usually only used for specialist builds for i686 on i386 etc. I fail to see the merit of proposing that packages add --build to the normal Debian build for no reason. As also noted elsewhere on this thread, the other way around is also bad: optimizing for i686 when you should be targetting i386 (which I guess is the case on most x86 developer machines): Quoting Steve Langasek There *is* a need to specify --build when not cross-compiling, if your upstream build rules do clever things when the architecture is detected as i686 instead of i386. -- Felipe Sateler -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Wed, 7 Nov 2007 20:06:55 +0100 Mike Hommey [EMAIL PROTECTED] wrote: How about making the snippet available in a .mk file that could just be included, so that whenever there is an enhancement, everyone can take advantage of it (but then, the snippet would need extra care to stay compatible) ? Nice idea and I have thought about it - the natural place for a cross building .mk file is within emdebian-tools or dpkg-cross - but even after only building a core of 90 source packages, I can tell that there is sufficient variation that the snippet would be so short as to be a runt. It isn't sufficient to just set the section that we have been discussing (disputing?) here. The variables set by that section need to be implemented later in the rules file and tie in with other settings used by the package. In practice, this means individual patches. (When I said this problem was common, I did not mean uniform.) -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgp8R46LIRcst.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
On Wed, Nov 07, 2007 at 06:54:04PM +, Neil Williams wrote: So it is preferable for me to add --build to native builds even in a patch that is meant to only affect the cross build? I'm sure some maintainers will query why I'm setting --build outside the crossbuild. Provide two distinct patches: one that adds support for --build, and a second one that does whatever modifications are neccessary for cross-compiling. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Wed, Nov 07, 2007 at 08:18:12AM +, Neil Williams wrote: Actually, /usr/share/doc/autotools-dev/README.Debian.gz differs from that by specifying $(confflags) instead of SYSTEM and using 'export'. I don't see what benefit arises from the only other difference, ?= instead of =. When cross-compiling, it isn't sensible to allow the value to be overridden elsewhere. The value is set in the cross-compiling tools via dpkg-architecture and should be read into debian/rules as such. As noted elsewhere in this thread, --build can be specified alone but is usually only used for specialist builds for i686 on i386 etc. I fail to see the merit of proposing that packages add --build to the normal Debian build for no reason. No, it's used to prevent upstream packages from *wrongly* building for i686 on the i386 architecture, as I've now stated several times in this thread. I don't see why this makes the proposed change bad. Please explain why ?= is better than = in this specific instance and why I should use SYSTEM instead of CROSS (when the recommended one is $(confflags) anyway) or add --build to every package instead of leaving the variable as empty. AFAICT there is no material reason to use the autofoo recommended layout and no justification for calling my layout bad merely because of minor changes in variable names. It's bad because it's worse for the *common* case that the existing autotools-dev recommendation. This is not a minor change in variable names, it's missing support for correct enforcement of the target architecture in the non-cross-building case that the autotools-dev recommendation has already addressed. I'm sorry for failing to notice this lapse at first glance, I assumed you were at least taking advantage of the established best practices instead of inventing solutions in a vacuum. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
Neil Williams [EMAIL PROTECTED] writes: Russ Allbery [EMAIL PROTECTED] wrote: Well, then I'm even more mystified as to why you contradicted those instructions and told people not to pass --build to configure for non-cross-compile builds. So it is preferable for me to add --build to native builds even in a patch that is meant to only affect the cross build? I'm sure some maintainers will query why I'm setting --build outside the crossbuild. I would tend to do so, personally, with the note that people can leave that part off if they don't like it. But that way, it's just bringing the package in line with the autotools-dev recommendation, which puts somewhat more force behind the patch anyway for people who may not think that cross-compilation is interesting for their package. If that is how it is to be done then that is OK with me - I was just trying to keep the patch to the basic cross build requirements. Yes, that makes sense. No problem. What is the reason for 'export' ? It's not particularly significant and I'm not sure anything currently uses it, but it makes the variable available to other helper tools used by debian/rules, such as debhelper, in case they need to be able to detect that cross-compilation is going on for some reason. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, Nov 06, 2007 at 01:04:48PM +, Neil Williams wrote: Emdebian has currently built 91 Debian source packages in the preparation of the basic root filesystem and other testing of the toolset. All of those have involved some level of patches to support cross-building with CDBS packages needing the least. Which is FUD, insomuch as CDBS only handles cross-building for packages that use GNU autoconf and many cross-building fixes require patching upstream code. This thread will discuss the type of changes that are needed, which packages are affected and how these bugs will be identified. If you want that to be what the thread is about, kindly don't lead into it with trolling about CDBS. I would recommend every maintainer of a compiled package to consider adding this basic level of cross-building support if it does not exist already. I agree. If nothing else, this is relevant because packages which use autoconf may otherwise mis-optimize for targets other than the default target for each of our given architectures when not explicitly told the system type using the dpkg-architecture information. Note that CDBS is inherently *easier* to crossbuild than any other build system in Debian. Only one change is necessary: DEB_CONFIGURE_SCRIPT_ENV := Note that this is an inherently *non-obvious* change to have to make to support cross-building, which is why the opacity of CDBS makes it an incredibly sucky system if you want maintainers who are actually able to reason about what their packages are doing. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, Nov 06, 2007 at 01:04:48PM +, Neil Williams wrote: This is one of the changes sought by Emdebian to support using Debian on embedded devices where storage space is far from cheap and involves running counter to the current Debian default of install everything that works, every time, every package. http://lists.debian.org/debian-devel/2007/11/msg00116.html Emdebian has currently built 91 Debian source packages in the preparation of the basic root filesystem and other testing of the toolset. All of those have involved some level of patches to support cross-building with CDBS packages needing the least. This thread will discuss the type of changes that are needed, which packages are affected and how these bugs will be identified. The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) Depending on whether the package uses ./configure: ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) else CROSS= endif Shouldn't packages be using these --build and --host arguments already even if not cross-building ? If they aren't there are possibilities of i686 binaries to be built for what is supposed to be the i386 arch... Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, 6 Nov 2007 13:08:30 -0500 Steve Langasek [EMAIL PROTECTED] wrote: On Tue, Nov 06, 2007 at 01:04:48PM +, Neil Williams wrote: Emdebian has currently built 91 Debian source packages in the preparation of the basic root filesystem and other testing of the toolset. All of those have involved some level of patches to support cross-building with CDBS packages needing the least. Which is FUD, insomuch as CDBS only handles cross-building for packages that use GNU autoconf and many cross-building fixes require patching upstream code. So far, I've only come across a handful of packages that need upstream patches for cross-building but then I've only built a minimal set of packages. Sorry if my preference for CDBS comes across negatively. I just wanted to point out that some packages need less work than others and to give some hint about why the build system affects which packages are going to get bug reports. This thread will discuss the type of changes that are needed, which packages are affected and how these bugs will be identified. If you want that to be what the thread is about, kindly don't lead into it with trolling about CDBS. OK. Let's leave the build system issue behind. I would recommend every maintainer of a compiled package to consider adding this basic level of cross-building support if it does not exist already. I agree. If nothing else, this is relevant because packages which use autoconf may otherwise mis-optimize for targets other than the default target for each of our given architectures when not explicitly told the system type using the dpkg-architecture information. There are other reasons why packages may get a bug report for crossbuild support, the DEB_*_GNU_TYPE is just a common one. Note that CDBS is inherently *easier* to crossbuild than any other build system in Debian. Only one change is necessary: DEB_CONFIGURE_SCRIPT_ENV := Note that this is an inherently *non-obvious* change to have to make to support cross-building, which is why the opacity of CDBS makes it an incredibly sucky system if you want maintainers who are actually able to reason about what their packages are doing. It wasn't obvious, true. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpaiSVEyZhgi.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
On Tue, 6 Nov 2007 19:35:41 +0100 Mike Hommey [EMAIL PROTECTED] wrote: On Tue, Nov 06, 2007 at 01:04:48PM +, Neil Williams wrote: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) Depending on whether the package uses ./configure: ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) else CROSS= endif Shouldn't packages be using these --build and --host arguments already even if not cross-building ? No. There is no need to specify --build and --host if not cross-compiling. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpOnwPsPJPmB.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
On Tue, Nov 06, 2007 at 07:23:39PM +, Neil Williams wrote: On Tue, 6 Nov 2007 19:35:41 +0100 Mike Hommey [EMAIL PROTECTED] wrote: On Tue, Nov 06, 2007 at 01:04:48PM +, Neil Williams wrote: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) Depending on whether the package uses ./configure: ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) else CROSS= endif Shouldn't packages be using these --build and --host arguments already even if not cross-building ? No. There is no need to specify --build and --host if not cross-compiling. Er, no. There *is* a need to specify --build when not cross-compiling, if your upstream build rules do clever things when the architecture is detected as i686 instead of i386. This was the point in my previous comment. There is no need to specify --host when not cross-compiling, and specifying it will result in autoconf believing that the compiler should be invoked as a cross-compiler. Sometimes this has strange side effects as well. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, Nov 06, 2007 at 02:52:37PM -0500, Steve Langasek wrote: On Tue, Nov 06, 2007 at 07:23:39PM +, Neil Williams wrote: On Tue, 6 Nov 2007 19:35:41 +0100 Mike Hommey [EMAIL PROTECTED] wrote: On Tue, Nov 06, 2007 at 01:04:48PM +, Neil Williams wrote: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) Depending on whether the package uses ./configure: ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) else CROSS= endif Shouldn't packages be using these --build and --host arguments already even if not cross-building ? No. There is no need to specify --build and --host if not cross-compiling. Er, no. There *is* a need to specify --build when not cross-compiling, if your upstream build rules do clever things when the architecture is detected as i686 instead of i386. This was the point in my previous comment. There is no need to specify --host when not cross-compiling, and specifying it will result in autoconf believing that the compiler should be invoked as a cross-compiler. Sometimes this has strange side effects as well. Hadn't this been fixed quite some time ago ? Or wasn't it something with --target ? Because I've been using --build and --host for a long time with no problem. Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, Nov 06, 2007 at 03:13:58PM -0500, Steve Langasek wrote: On Tue, Nov 06, 2007 at 09:07:54PM +0100, Mike Hommey wrote: There is no need to specify --host when not cross-compiling, and specifying it will result in autoconf believing that the compiler should be invoked as a cross-compiler. Sometimes this has strange side effects as well. Hadn't this been fixed quite some time ago ? Or wasn't it something with --target ? Because I've been using --build and --host for a long time with no problem. AFAIK there's nothing to be fixed here. If you specify both --build and --host, autoconf sees this as a request for cross-compiling. Of course, invoking the native compiler as a cross-compiler should /generally/ give the same results as invoking it as gcc, so it's entirely possible that it doesn't cause problems on your packages. The autoconf documentation says only if you enter both and they're different will it enter cross compiling mode. FWIW, the dh_make samples set both. Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, Nov 06, 2007 at 09:07:54PM +0100, Mike Hommey wrote: There is no need to specify --host when not cross-compiling, and specifying it will result in autoconf believing that the compiler should be invoked as a cross-compiler. Sometimes this has strange side effects as well. Hadn't this been fixed quite some time ago ? Or wasn't it something with --target ? Because I've been using --build and --host for a long time with no problem. AFAIK there's nothing to be fixed here. If you specify both --build and --host, autoconf sees this as a request for cross-compiling. Of course, invoking the native compiler as a cross-compiler should /generally/ give the same results as invoking it as gcc, so it's entirely possible that it doesn't cause problems on your packages. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Long-term mass bug filing for crossbuild support
On Tue, 6 Nov 2007 14:52:37 -0500 Steve Langasek [EMAIL PROTECTED] wrote: On Tue, Nov 06, 2007 at 07:23:39PM +, Neil Williams wrote: Shouldn't packages be using these --build and --host arguments already even if not cross-building ? No. There is no need to specify --build and --host if not cross-compiling. Er, no. There *is* a need to specify --build when not cross-compiling, if your upstream build rules do clever things when the architecture is detected as i686 instead of i386. This was the point in my previous comment. Sorry, my reply wasn't clear. I should have added 'both': There is no need to specify both --build and --host if not cross-compiling. or There is no need to specify '--build and --host' if not cross-compiling. The patches that I will be submitting will set both at the same time (as I hope the original message showed). There is no need to specify --host when not cross-compiling, and specifying it will result in autoconf believing that the compiler should be invoked as a cross-compiler. Sometimes this has strange side effects as well. That is what I was trying to get at. If --build == --host then don't specify both - unless you are trying to build a cross compiler at which point this whole thing gets even more confusing. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpjGD1ZqzYdU.pgp Description: PGP signature
Re: Long-term mass bug filing for crossbuild support
Neil Williams [EMAIL PROTECTED] writes: The most common change is simply to retrieve the cross-building metadata from dpkg-architecture in debian/rules: http://wiki.debian.org/EmdebianGuide DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) Depending on whether the package uses ./configure: ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) else CROSS= endif This is a bad suggestion and was apparently made without referring to the existing Debian instructions for how to handle packages that use Autoconf and friends. Please see the autotools-dev instructions, which are more detailed and which correctly recommend: DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) ifeq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) SYSTEM = --build $(DEB_HOST_GNU_TYPE) else SYSTEM = --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) endif # ... ./configure $(SYSTEM) -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
