Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
On Thu, 29 Mar 2012, Wookey wrote: But you should not rely on this because calling debian/rules directly must be supported. Hmm, but if a package cannot use the variables set by dpkg-buildpackage and must set them itself, what is the point of dpkg-buildpackage setting them? To save the package exporting the variables? I see two main reasons (but I did not write the original code, so I don't know the initial motivation): 1/ this allows debian/rules to use ?= assignation resulting in no-op when the variables are set. This avoids multiple dpkg-architecture invocations since dpkg-buildpackage only calls it once. 2/ it is required for cross-compilation since the fact that DEB_HOST_ARCH is set to a value that is not equal to DEB_BUILD_ARCH is the canonical fact that defines cross-compilation of the debian package Well, perhaps I shouldn't (and indeed I'd like us to get to a point where we don't), but currently, in practice, non-native builds need other things setting in the environment anyway so even using dpkg-buildpackage isn't necessarily sufficient, whereas for a native build it always is. OK but this should ideally be integrated in common layers such as CDBS and debhelper. Now if everyone is happy that debian/rules remains the canonical interface even for cross-builds and that they should work without dpkg-buildpackage help then I should start testing on that basis. I have not done that to date. Honestly I have never seen anyone doing cross-builds and calling debian/rules manually. So while I believe that cross-build should not make different assumptions from native builds (i.e. we want to converge), I would also not bother testing this explicitly. dpkg-buildpackage -a is what people are using (and should be using). Cheers, -- Raphaël Hertzog ◈ Debian Developer Pre-order a copy of the Debian Administrator's Handbook and help liberate it: http://debian-handbook.info/liberation/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120330060656.gk12...@rivendell.home.ouaza.com
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
On Fri, 30 Mar 2012 08:06:56 +0200 Raphael Hertzog hert...@debian.org wrote: On Thu, 29 Mar 2012, Wookey wrote: Well, perhaps I shouldn't (and indeed I'd like us to get to a point where we don't), but currently, in practice, non-native builds need other things setting in the environment anyway so even using dpkg-buildpackage isn't necessarily sufficient, whereas for a native build it always is. OK but this should ideally be integrated in common layers such as CDBS and debhelper. The bits needed for cross-builds vary on a package-by-package basis, have little in common between packages and are only discovered when someone tries to cross-build that particular package. The list of values needed by a package or the ways in which the package can discover the values it needs can vary between package versions. Generally, there are two kinds of values: 0: architecture-specific extensions - values which are directly related to the specific architecture and change between architectures but which aren't sufficiently common or general to be provided as the default for all packages on that architecture. (Indeed, supplying these values to packages which do not normally check for them can cause build failures due to side-effects within the ./configure scripts.) These values are currently provided by dpkg-cross as package-specific extensions to the list of common values (like the size of an int or void pointer, endianness etc.) which were always part of dpkg-cross. [0] These values are typically picked up by the ./configure script and need to be provided for cross-builds solely because otherwise that specific package will try to discover the value by compiling something for the HOST architecture and then executing it, which is fatal for a cross-build. 1: package-breakage prevention values - things which can go into debian/rules or as patches to the upstream code to supply or prevent attempted detection of architecture-neutral values or conditionals around parts of the package build which need to be done using the build compiler instead of the cross compiler. These values or conditionals are the same whatever the HOST architecture, they just differ between native builds vs cross builds. i.e. these are perfect to go inside the ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) conditional in debian/rules. [0] http://packages.debian.org/sid/all/dpkg-cross/filelist So in the second case, it is clear that debian/rules can and should contain these conditionals and values and the only problem with that is when the code bit rots because it's not the default build of the package and the maintainer doesn't get to test that version when something upstream has changed how it works. The long term aim for the first case would be for -dev packages to provide files in a /path/cross.d/$arch/ directory which are generated during the build of the package for that architecture and which declare the values which that package had to discover by executing a compiled test binary. That could be automated, possibly as a filter on the full list of values logged by ./configure. There is a role here for a debhelper tool. Both of these changes would be much easier to design, implement, test and maintain once we have the combination of MultiArch -dev packages and cross-compilers built by Debian. (Then we could even look at a cross-piuparts infrastructure to report issues with these mechanisms.) Now if everyone is happy that debian/rules remains the canonical interface even for cross-builds and that they should work without dpkg-buildpackage help then I should start testing on that basis. I have not done that to date. Honestly I have never seen anyone doing cross-builds and calling debian/rules manually. ... only because it *always* fails I have longed for such support myself at times. It is incredibly frustrating to see a cross-build fail 90% of the way through and then be faced within having to run the clean target after putting in a test fix to an .install file or similar... Yes, once all the fixes are in, the actual build is done with dpkg-buildpackage. In the meantime, we do need support to be able to run individual debian/rules targets within the cross-build environment and without having to delete everything you've just cross-compiled. So while I believe that cross-build should not make different assumptions from native builds (i.e. we want to converge), I would also not bother testing this explicitly. dpkg-buildpackage -a is what people are using (and should be using). For build runs / packages for upload, yes - absolutely. For actual development and testing, no. Just as the buildd infrastructure must rely on dpkg-buildpackage but maintainers need to be able to run debian/rules targets manually sometimes. For exactly the same reasons, cross-build maintainers also need to be able to run specific debian/rules targets sometimes too. Except we can't and that should be fixable and should be
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
Wookey woo...@wookware.org writes: +++ Raphael Hertzog [2012-03-29 21:06 +0200]: Hi, On Thu, 29 Mar 2012, Wookey wrote: Anyone know when this happened and what if any, the limitations are? It's certainly true in wheezy, squeeze, precise and oineiric. This has always been the case ever since dpkg-architecture has been introduced. OK, shows how much I know :-) But you should not rely on this because calling debian/rules directly must be supported. Hmm, but if a package cannot use the variables set by dpkg-buildpackage and must set them itself, what is the point of dpkg-buildpackage setting them? To save the package exporting the variables? There is no point. Packages must use dpkg-architecture, dpkg-buildflags,... or the respective makefile sniplets in /usr/share/dpkg/ in case debian/rules is called directly, they must not rely of dpkg-architecture setting them. Hopefully dpkg-buildpackage will stop setting those varibales at some point so sources that wrongfully depend on the variables being set actualy break. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87398q9s9g.fsf@frosties.localnet
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
On Thu, Mar 29, 2012 at 09:58:41PM +0200, Julien Cristau wrote: On Thu, Mar 29, 2012 at 19:10:05 +0100, Wookey wrote: Should a package depending on this behaviour build-dep on a particular dpkg version? As it already works in build-essential in stable do the same rules apply as essential packages in stable (i.e no explicit dependency required)? That would be consistent. Maybe it's been doing it since forever? A package may not depend on this behaviour. The interface to build a package is still debian/rules, not dpkg-buildpackage. While this might be the formal policy, isn't dpkg-buildpackage the /de-facto/ interface? AFAIK we don't actually test that building via debian/rules alone works, while we do test that building via dpkg-buildpackage works since this is what most packages build for upload, and all packages built on the autobuilders, use. Would there not be some advantage to making dpkg-buildpackge the interface for building? (Not dropping the debian/rules interface, of course.) This would permit the automatic setting of all the host- and build-related variables without requiring every package to also set them by hand. It would of course still be possible to build using debian/rules directly, but it wouldn't be required to set the flags. Without commenting on whether or not this is a good idea, isn't this basically the status quo already, given that most packages don't test direct usage of debian/rules, and manual setting of the various arch- and build-related variables is inconsistent? Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linuxhttp://people.debian.org/~rleigh/ `. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools `-GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120330130833.gj30...@codelibre.net
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
On Mar 30, 2012 8:40 AM, Goswin von Brederlow goswin-...@web.de wrote: Hopefully dpkg-buildpackage will stop setting those varibales at some point so sources that wrongfully depend on the variables being set actualy break. Already happened in version 1.16.1 (#560070).
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
On Fri, 30 Mar 2012, Neil Williams wrote: Honestly I have never seen anyone doing cross-builds and calling debian/rules manually. ... only because it *always* fails I have longed for such support myself at times. It is incredibly frustrating to see a cross-build fail 90% of the way through and then be faced within having to run the clean target after putting in a test fix to an .install file or similar... Yes, once all the fixes are in, the actual build is done with dpkg-buildpackage. In the meantime, we do need support to be able to run individual debian/rules targets within the cross-build environment and without having to delete everything you've just cross-compiled. Since Peter Green also said something similar, let me point out that dpkg-buildpackage has a -nc option which avoids debian/rules clean and thus give similar results to calling debian/rules build (or binary) directly. Cheers, -- Raphaël Hertzog ◈ Debian Developer Pre-order a copy of the Debian Administrator's Handbook and help liberate it: http://debian-handbook.info/liberation/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120330131334.gh18...@rivendell.home.ouaza.com
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
+++ peter green [2012-03-29 20:06 +0100]: Now, you can build packages without using dpkg-buildpackage by calling rules directly, and in that case the rules file would need to call dpkg-architecture, but someone would have to convince me that that was an interface worth supporting for non-native builds The big reason it's worth supporting IMO is that with most packages you can resume after a failld build by manually running debian/rules build. When fixing compile errors in a large package I don't want to have to restart the build from scratch after every file I fix. That is indeed a very useful use case (and the only one I use rules for directly, apart from 'clean' quite often). But it's not hard to do 'dpkg-buildpackage -Tbuild' to get exactly the same effect but with the dpkg-buildpackage environement set-up done too. So if we decided that cross-building was only expected to work via dpkg-buildpackage -aarch, you wouldn't be unduly inconvenienced, would you? Is there some important subtle difference here that I am missing? Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120330130636.gb15...@dream.aleph1.co.uk
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
On Fri, Mar 30, 2012 at 14:08:33 +0100, Roger Leigh wrote: Would there not be some advantage to making dpkg-buildpackge the interface for building? (Not dropping the debian/rules interface, of course.) This would permit the automatic setting of all the host- and build-related variables without requiring every package to also set them by hand. It would of course still be possible to build using debian/rules directly, but it wouldn't be required to set the flags. I'm not sure how you reconcile make dpkg-buildpackage the interface for building and not dropping the debian/rules interface. If you say packages are allowed to use variables they don't set, then that means debian/rules build is allowed to fail, which would be a pain when working on a package, and effectively means dropping the debian/rules interface AFAICT. Cheers, Julien signature.asc Description: Digital signature
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
Hi Wookey, Wookey wrote: I recently noticed that when building with dpkg-buildpackage there is no need for the DEB_BUILD_GNU_TYPE := $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) DEB_HOST_GNU_TYPE := $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) Don't you mean ?=? [...] You can just do the test: ifeq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) and it works, so it looks like dpkg-buildpackage is now setting those vars for us. This is good and sensible, it's just that it's news to me. Anyone know when this happened and what if any, the limitations are? It happened years ago, but it's not guaranteed by policy. As you wrote: [...] Now, you can build packages without using dpkg-buildpackage by calling rules directly, and in that case the rules file would need to call dpkg-architecture, but someone would have to convince me that that was an interface worth supporting for non-native builds, because I have certainly always considered the minimal interface for cross package-building to be dpkg-buildpackage -aarch, and in practice there are other things you need to do for non-trivial packages (set CONFIG_SITE, set DEB_BUILD_OPTS=nocheck). I believe invoking debian/rules by hand is an interface worth supporting, even for cross builds, but the person calling debian/rules by hand is certainly responsible for setting DEB_BUILD_OPTIONS=nocheck! Hope that helps, Jonathan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120329190545.GF3147@burratino
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
Hi, On Thu, 29 Mar 2012, Wookey wrote: Anyone know when this happened and what if any, the limitations are? It's certainly true in wheezy, squeeze, precise and oineiric. This has always been the case ever since dpkg-architecture has been introduced. But you should not rely on this because calling debian/rules directly must be supported. dpkg-buildpackage(1) clearly states: | Even if dpkg-buildpackage exports some variables, debian/rules should | not rely on their presence and should instead use the respective | interface to retrieve the needed values. And this interface is dpkg-architecture itself in this case. Now, you can build packages without using dpkg-buildpackage by calling rules directly, and in that case the rules file would need to call dpkg-architecture, but someone would have to convince me that that was an interface worth supporting for non-native builds, because I have certainly always considered the minimal interface for cross package-building to be dpkg-buildpackage -aarch, and in practice there are other things you need to do for non-trivial packages (set CONFIG_SITE, set DEB_BUILD_OPTS=nocheck). (and ensure various things like toolchain and dpkg-cross are installed). And I don't think we want that stuff in every single rules file. I don't see why you're differentiating non-native builds from native builds (unless you assume different debian/rules files). In any case, if you're worried about the boilerplate code required to get the variables defined, you can now use include /usr/share/dpkg/architecture.mk to avoid it (but this requires dpkg-dev = 1.16.1). Cheers, -- Raphaël Hertzog ◈ Debian Developer Pre-order a copy of the Debian Administrator's Handbook and help liberate it: http://debian-handbook.info/liberation/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120329190625.ge9...@rivendell.home.ouaza.com
re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
Now, you can build packages without using dpkg-buildpackage by calling rules directly, and in that case the rules file would need to call dpkg-architecture, but someone would have to convince me that that was an interface worth supporting for non-native builds The big reason it's worth supporting IMO is that with most packages you can resume after a failld build by manually running debian/rules build. When fixing compile errors in a large package I don't want to have to restart the build from scratch after every file I fix. Of course I will do a proper build with dpkg-buildpackage at the end but only after i've fixed all the compile errors. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f74b2cd.5070...@p10link.net
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
On Thu, Mar 29, 2012 at 19:10:05 +0100, Wookey wrote: Should a package depending on this behaviour build-dep on a particular dpkg version? As it already works in build-essential in stable do the same rules apply as essential packages in stable (i.e no explicit dependency required)? That would be consistent. Maybe it's been doing it since forever? A package may not depend on this behaviour. The interface to build a package is still debian/rules, not dpkg-buildpackage. Cheers, Julien signature.asc Description: Digital signature
Re: dpkg-buildpackage now sets DEB_BUILD_HOST etc for you?
+++ Raphael Hertzog [2012-03-29 21:06 +0200]: Hi, On Thu, 29 Mar 2012, Wookey wrote: Anyone know when this happened and what if any, the limitations are? It's certainly true in wheezy, squeeze, precise and oineiric. This has always been the case ever since dpkg-architecture has been introduced. OK, shows how much I know :-) But you should not rely on this because calling debian/rules directly must be supported. Hmm, but if a package cannot use the variables set by dpkg-buildpackage and must set them itself, what is the point of dpkg-buildpackage setting them? To save the package exporting the variables? Now, you can build packages without using dpkg-buildpackage by calling rules directly, and in that case the rules file would need to call dpkg-architecture, but someone would have to convince me that that was an interface worth supporting for non-native builds, because I have certainly always considered the minimal interface for cross package-building to be dpkg-buildpackage -aarch, and in practice there are other things you need to do for non-trivial packages (set CONFIG_SITE, set DEB_BUILD_OPTS=nocheck). (and ensure various things like toolchain and dpkg-cross are installed). And I don't think we want that stuff in every single rules file. I don't see why you're differentiating non-native builds from native builds (unless you assume different debian/rules files). Well, perhaps I shouldn't (and indeed I'd like us to get to a point where we don't), but currently, in practice, non-native builds need other things setting in the environment anyway so even using dpkg-buildpackage isn't necessarily sufficient, whereas for a native build it always is. Although what you're telling me is that that's irrelevant because in fact just calling debian/rules should also always be sufficient. To make that true for cross-builds requires more than we are putting in rules files at the moment. So there is a difference. Now if everyone is happy that debian/rules remains the canonical interface even for cross-builds and that they should work without dpkg-buildpackage help then I should start testing on that basis. I have not done that to date. The obvious bits that will be missing in many packages are something to deal with autoconf cache variables (currently handled by dpkg-cross and setting CONFIG_SITE externally). And something to set cmake toolchain file variables (also currently in dpkg-cross, but I suspect it needs updating for multiarch). In any case, if you're worried about the boilerplate code required to get the variables defined, you can now use include /usr/share/dpkg/architecture.mk to avoid it (but this requires dpkg-dev = 1.16.1). OK. That's useful to know. And fine from here on in. I'll add that to the page. Keeping boilerplate in includable snippets seems like an excellent plan generally. Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120329203052.gu26...@dream.aleph1.co.uk
