Re: assumptions about the build environment.
Jakub Wilk writes (Re: assumptions about the build environment.): More questions about build env assumptions: My answers: Can you assume that /sbin and /usr/sbin are within PATH? No. Package builds are supposed to be done as the user; the rootness of the install target is just for file permissions and doesn't mean it should be using admin tools. Can you assume that the SHELL environment variable (_not_ the makefile variable) is set to something §10.4-compliant? No. SHELL might refer to ksh, csh or even emacs. Ian. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20594.62821.442938.293...@chiark.greenend.org.uk
Re: assumptions about the build environment.
More questions about build env assumptions: Can you assume that /sbin and /usr/sbin are within PATH? Can you assume that the SHELL environment variable (_not_ the makefile variable) is set to something §10.4-compliant? (My personal answers to these questions are: no and no.) -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121007085831.ga7...@jwilk.net
Re: Re: assumptions about the build environment.
While working on debian one thing I have not managed to find is documentation on what packages can and can't assume about the build environment. Does such documentation exist and if not should it be created. Some specific cases i'm wondering about: I just discovered that on my beagleboard XM (under armhf sid) nacl (which previously build on a debian experimental armhf buildd but not a debian unstable armhf buildd) will build if /sys is mounted but will not build if it is not mounted. Can packages assume that /sys will be mounted in the build environment or not? I consider /sys almost as essential as /proc. However I wonder what a build process would need it for. I will hijack this thread a bit and I know there is another one running on the subject, but assumption like this makes it impossible to cross compile most packages. I'm currently trying to compile armhf package for the rasberry pi on a amd64 machine and naively though it would be easy to do with multiarch. I screwed my machines(replaced the dynamic linkers, ftp and other tools by arm binaries although I followed the scarce available documentation). Natively compiling package as big as XBMC on the PI is a nightmare and current tools fails really short because you: 1) need a root filesystem for the machines. You can use debootstrap but will need many additionnal packages that are yet not build, 2) cannot install produced .deb in the root filesystem exept by running them on qemu which is.. Any hint? -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/50715160.5050...@free.fr
Re: assumptions about the build environment.
On Sun, Oct 07, 2012 at 10:58:31AM +0200, Jakub Wilk wrote:
More questions about build env assumptions:
Can you assume that /sbin and /usr/sbin are within PATH?
At which point(s) during the build?
For sbuild:
During any command run as the build user (*not* root), it will
default to
my %common_keys = (
'PATH' = {
DEFAULT =
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
},
(from Sbuild::ConfBase). They should be present when running commands
as root, though I'd have to double-check that.
Can you assume that the SHELL environment variable (_not_ the
makefile variable) is set to something §10.4-compliant?
It looks like we currently pass SHELL through, unless you configure it
otherwise. Maybe this should be revisited?
(My personal answers to these questions are: no and no.)
I think that this is correct (as the default behaviour; it could be
configured otherwise).
Regards,
Roger
--
.''`. Roger Leigh
: :' : Debian GNU/Linuxhttp://people.debian.org/~rleigh/
`. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools
`-GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20121007213942.gi18...@codelibre.net
Re: Re: assumptions about the build environment.
On Sun, Oct 07, 2012 at 11:54:40AM +0200, Eric Valette wrote: I'm currently trying to compile armhf package for the rasberry pi on a amd64 machine and naively though it would be easy to do with multiarch. I screwed my machines(replaced the dynamic linkers, ftp and other tools by arm binaries although I followed the scarce available documentation). Natively compiling package as big as XBMC on the PI is a nightmare and current tools fails really short because you: 1) need a root filesystem for the machines. You can use debootstrap but will need many additionnal packages that are yet not build, 2) cannot install produced .deb in the root filesystem exept by running them on qemu which is.. Any hint? schroot will let you run a non-native chroot with qemu, and you can use this with sbuild for package building. sbuild now also has initial support or multiarch cross building. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linuxhttp://people.debian.org/~rleigh/ `. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools `-GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121007214110.gj18...@codelibre.net
Re: assumptions about the build environment.
+++ Wouter Verhelst [2012-09-24 13:14 +0200]: On Sat, Sep 22, 2012 at 07:06:57PM +0200, Adam Borowski wrote: That it breaks multiarch builds. Multiarch builds are not done on our buildd machines. Yet. They very likely will be soon, for partial architectures and cross-toolchains. And we still want them to work, in cross-compiling configurations, too, at least for core stuff. [...] As for complex hard to patch build systems, could you give me an example? apache/apr :-) But the case of uname _is_ easy to deal with - use dpkg-architecture. And on the original point, anything checking /sys for HOST arch stuff is likely to be getting it wrong too - fine for BUILD arch checks. I can't quite think of a case where patching references to uname could be tricky. I can't give you an example of *this* particular issue; but given the amount of complex build systems I've seen, I don't think it's too unreasonable to assume they exist. I'm not saying we should change policy to mandate that i386 builds in i386 chroots should be done with linux32 running or something similar. But I also do happen to think that tuning buildd machines to do a bit more than what policy requires them to do is usually a good thing. For I used to keep debhelper installed in my buildd chroots, even though it's not part of build-essential; but since almost all packages use it in some form or another, it was being installed and deinstalled enough that I could gain some time by having it installed by default. The result was of course that packages who mistakenly failed to add debhelper to their build-depends would successfully build on my buildd hosts, but I didn't think that was much of a problem. Similarly, if adding linux32 to the environment on amd64 hosts building inside a chroot means the success rate for packages built will go up, I don't think we should refrain from doing so -- unless, of course, doing so would cause more problems than it would solve, which was the point of my question. Well, this is indeed exactly the point. If we specify what can be assumed by packages, but then actually don't test that, we test something slightly different (by putting extra stuff int he environment by default) then bugs will not be found. It's a simple tradeoff between rigour and build success/hassle. In general I like the rigour in Debian and have spent the last year or so being rigourous in cross-building/multiarch stuff which has found an awful lot of (expected) breakage. (You an gloss over a lot of stuff by installing qemu in chroots when cross-building, for example, and often it's the best thing to do) In general I'm in favour of agreeing what can be relied upon and then only providing that in buildds. But you are quite right that for the purposes of buildds doing native building this improves your success rate by glossing over potentially-dubious checks, with little cost. After all, the autobuilder network is not meant to do QA; we have resources for doing full-archive rebuilds for that purpose. Instead, the autobuilder network is meant to make sure we build as many packages as possible. If we can avoid some issues in building packages that isn't really worth spelling out in policy by just doing some minor configuration tweak on a buildd host, I think we should do so -- and this certainly qualifies, IMO. I only run cross-buildds, not native ones, so this isn't my call. Just bear in mind that every such choice of providing more than is strictly mandated will be hiding bugs in other circumstances. Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120927121449.gp24...@stoneboat.aleph1.co.uk
Re: assumptions about the build environment.
On Thu, 2012-09-27 at 13:14:49 +0100, Wookey wrote: But the case of uname _is_ easy to deal with - use dpkg-architecture. Well, not really if the patch is supposed to go upstream, given that dpkg-architecture is not a distribution-neutral interface. In most of the cases uname is wrong because the build system should be checking for features instead of hardcoded lists of architectures and because its output is not really normalized across different operating systems, or as mentioned on this thread the detection should really be delayed until run-time. But if the architecture is needed for whatever reason then using something like config.guess/sub's output and a way to distinguish between host and build architectures is probably better. And on the original point, anything checking /sys for HOST arch stuff is likely to be getting it wrong too - fine for BUILD arch checks. Right. regards, guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120927135557.ga...@gaara.hadrons.org
Re: assumptions about the build environment.
On Thu, Sep 27, 2012 at 01:14:49PM +0100, Wookey wrote: In general I'm in favour of agreeing what can be relied upon and then only providing that in buildds. But you are quite right that for the purposes of buildds doing native building this improves your success rate by glossing over potentially-dubious checks, with little cost. I guess the question here is whether we write policy on how to build a package in support of buildd machines, or whether things are the other way around, that is, we use buildd machines to verify packages' compliance to policy. If we were doing the latter, we should probably also do things like run lintian on a package after it built, or run it through piuparts, or do any number of other time-consuming things that would improve the quality of our distribution overall. We are, however, not doing that, because this is not the primary focus of the buildd network. Instead, the primary focus of the buildd network is to build our packages for their architecture in a timely manner. Now I'm not saying that I'm opposed, in principle, to such testing if it provided a net benefit; and to some extent, we are in fact already doing that (since maintainers are encouraged to enable test suites as part of the regular build of their packages). However, when the buildd network is backlogged because of broken or ageing hardware, or because of other issues not directly under the control of the buildd admins, these admins should have the ability to cut some corners (while still producing correct packages) so that they can cut down on the time required when things are getting tight. To be able to do this, the amount of things the hosts in the buildd network absolutely must do should be a list that is shorter rather than longer. In essense, the buildd network is not a QA resource; and while it is okay to use it as such if buildd hosts have time to spare, I believe we should ensure that our QA team's primary resources for archive-wide tests should be found elsewhere, so that buildd admins have the liberty to do what they need to do: ensure that an architecture can still be part of the next release, by building the packages that we need to build. -- The volume of a pizza of thickness a and radius z can be described by the following formula: pi zz a signature.asc Description: Digital signature
Re: assumptions about the build environment.
On Sat, Sep 22, 2012 at 07:06:57PM +0200, Adam Borowski wrote: On Sat, Sep 22, 2012 at 04:28:32PM +0200, Wouter Verhelst wrote: On Sat, Sep 22, 2012 at 12:23:36AM +0200, Kurt Roeckx wrote: They probably should try to use the output of dpkg-architecture to select the arch. Then should never check that output of uname -m. That's living on the assumption that there's never any upstream build system which is both complex (thus difficult to patch correctly) and relying on uname -m in one or more places. I'm not saying we should necessarily support such systems, but if running inside something akin to linux32 is not causing too many problems, it would make that easier. What's the harm? That it breaks multiarch builds. Multiarch builds are not done on our buildd machines. You'd need a separate chroot for every arch you want to be able to compile for. Buildd machines typically have only one chroot for each distribution they build, as they don't build for multiple architectures. [...] As for complex hard to patch build systems, could you give me an example? I can't quite think of a case where patching references to uname could be tricky. I can't give you an example of *this* particular issue; but given the amount of complex build systems I've seen, I don't think it's too unreasonable to assume they exist. I'm not saying we should change policy to mandate that i386 builds in i386 chroots should be done with linux32 running or something similar. But I also do happen to think that tuning buildd machines to do a bit more than what policy requires them to do is usually a good thing. For I used to keep debhelper installed in my buildd chroots, even though it's not part of build-essential; but since almost all packages use it in some form or another, it was being installed and deinstalled enough that I could gain some time by having it installed by default. The result was of course that packages who mistakenly failed to add debhelper to their build-depends would successfully build on my buildd hosts, but I didn't think that was much of a problem. Similarly, if adding linux32 to the environment on amd64 hosts building inside a chroot means the success rate for packages built will go up, I don't think we should refrain from doing so -- unless, of course, doing so would cause more problems than it would solve, which was the point of my question. After all, the autobuilder network is not meant to do QA; we have resources for doing full-archive rebuilds for that purpose. Instead, the autobuilder network is meant to make sure we build as many packages as possible. If we can avoid some issues in building packages that isn't really worth spelling out in policy by just doing some minor configuration tweak on a buildd host, I think we should do so -- and this certainly qualifies, IMO. -- The volume of a pizza of thickness a and radius z can be described by the following formula: pi zz a -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120924111419.gq17...@grep.be
Re: assumptions about the build environment.
Wouter Verhelst wou...@debian.org (24/09/2012): On Sat, Sep 22, 2012 at 07:06:57PM +0200, Adam Borowski wrote: You'd need a separate chroot for every arch you want to be able to compile for. Buildd machines typically have only one chroot for each distribution they build, as they don't build for multiple architectures. I guess “typically” was indeed warranted, s390/s390x come to mind (ditto on the porterbox side). Mraw, KiBi. signature.asc Description: Digital signature
Re: assumptions about the build environment.
On Sat, Sep 22, 2012 at 12:23:36AM +0200, Kurt Roeckx wrote: On Fri, Sep 21, 2012 at 08:26:24PM +0100, peter green wrote: Some time ago I found that a package (I think it was openjdk but I don't remember for sure) which relied on uname -r such that linux32 had to be used to build it in an i386 chroot on an amd64. However since then I'm pretty sure i've seen similar cases with other packages on other architectures being treated as bugs. They probably should try to use the output of dpkg-architecture to select the arch. Then should never check that output of uname -m. That's living on the assumption that there's never any upstream build system which is both complex (thus difficult to patch correctly) and relying on uname -m in one or more places. I'm not saying we should necessarily support such systems, but if running inside something akin to linux32 is not causing too many problems, it would make that easier. What's the harm? -- The volume of a pizza of thickness a and radius z can be described by the following formula: pi zz a -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120922142832.gk29...@grep.be
Re: assumptions about the build environment.
On Sat, Sep 22, 2012 at 04:28:32PM +0200, Wouter Verhelst wrote: On Sat, Sep 22, 2012 at 12:23:36AM +0200, Kurt Roeckx wrote: They probably should try to use the output of dpkg-architecture to select the arch. Then should never check that output of uname -m. That's living on the assumption that there's never any upstream build system which is both complex (thus difficult to patch correctly) and relying on uname -m in one or more places. I'm not saying we should necessarily support such systems, but if running inside something akin to linux32 is not causing too many problems, it would make that easier. What's the harm? That it breaks multiarch builds. You'd need a separate chroot for every arch you want to be able to compile for. Just like you can compile Windows crap by CC=i686-w64-mingw32-gcc, you should be able to CC=i486-linux-gnu-gcc, and, with a bit of effort that is supposed to be unnecessary in jessie, already are. So you can with CC=armel-linux-gnueabi-gcc, without any ugly hacks like scratchbox. xapt allowed this in squeeze already, adding a suffix -armel-cross where we add :armel in wheezy. It still doesn't work for every package, but for stuff YOU maintain, YOU can fix it. It's nice for whoever will be hacking on your code. And an interesting tidbit: checking for suffix of executables... .exe checking whether we are cross-compiling... no So the concept of cross-compiling is pretty fuzzy :) As for complex hard to patch build systems, could you give me an example? I can't quite think of a case where patching references to uname could be tricky. -- Copyright and patents were never about promoting culture and innovations; from the very start they were legalized bribes to give the king some income and to let businesses get rid of competition. For some history, please read https://en.wikipedia.org/wiki/Statute_of_Monopolies_1623 signature.asc Description: Digital signature
Re: assumptions about the build environment.
On Sat, 22 Sep 2012 19:06:57 +0200, Adam Borowski kilob...@angband.pl wrote: And an interesting tidbit: checking for suffix of executables... .exe checking whether we are cross-compiling... no So the concept of cross-compiling is pretty fuzzy :) That typically happens if you have wine and binfmt-support installed, since the cross-compiling test is can the system run an executable built using the chosen compiler! Regards, Stephen signature.asc Description: PGP signature
assumptions about the build environment.
While working on debian one thing I have not managed to find is documentation on what packages can and can't assume about the build environment. Does such documentation exist and if not should it be created. Some specific cases i'm wondering about: I just discovered that on my beagleboard XM (under armhf sid) nacl (which previously build on a debian experimental armhf buildd but not a debian unstable armhf buildd) will build if /sys is mounted but will not build if it is not mounted. Can packages assume that /sys will be mounted in the build environment or not? IIRC it is generally established that packages are not allowed to rely on an internet connection during build but if one is present are they allowed to assume it's non-broken. I recently came accross a package ( sslh ) which fails to build in the presense of nxdomain hijacking. Is that a bug? Some time ago I found that a package (I think it was openjdk but I don't remember for sure) which relied on uname -r such that linux32 had to be used to build it in an i386 chroot on an amd64. However since then I'm pretty sure i've seen similar cases with other packages on other architectures being treated as bugs. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/505cbf60.5020...@p10link.net
Re: assumptions about the build environment.
peter green wrote: Some time ago I found that a package (I think it was openjdk but I don't remember for sure) which relied on uname -r sorry I meam -m not -r -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/505cc36b.6010...@p10link.net
Re: assumptions about the build environment.
On Fri, Sep 21, 2012 at 08:26:24PM +0100, peter green wrote: While working on debian one thing I have not managed to find is documentation on what packages can and can't assume about the build environment. Does such documentation exist and if not should it be created. Some specific cases i'm wondering about: I just discovered that on my beagleboard XM (under armhf sid) nacl (which previously build on a debian experimental armhf buildd but not a debian unstable armhf buildd) will build if /sys is mounted but will not build if it is not mounted. Can packages assume that /sys will be mounted in the build environment or not? I consider /sys almost as essential as /proc. However I wonder what a build process would need it for. IIRC it is generally established that packages are not allowed to rely on an internet connection during build but if one is present are they allowed to assume it's non-broken. No, because a 'non-broken' connection would include some particular hosts being available and there is no way an auto-builder can ensure that is true. I recently came accross a package ( sslh ) which fails to build in the presense of nxdomain hijacking. Is that a bug? Yes. Some time ago I found that a package (I think it was openjdk but I don't remember for sure) which relied on uname -r such that linux32 had to be used to build it in an i386 chroot on an amd64. However since then I'm pretty sure i've seen similar cases with other packages on other architectures being treated as bugs. I think confusion between kernel vs userland architecture is so widespread that we should consider this a necessary part of doing a native build. (It should preferably be fixed upstream, to benefit users who build from either Debian or upstream source on a 32/64 environment. But I don't know a simple way to find the 'primary userland architecture' that is not distribution-specific.) Ben. -- Ben Hutchings We get into the habit of living before acquiring the habit of thinking. - Albert Camus -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120921200757.gf29...@decadent.org.uk
Re: assumptions about the build environment.
On Fri, Sep 21, 2012 at 09:07:57PM +0100, Ben Hutchings wrote: On Fri, Sep 21, 2012 at 08:26:24PM +0100, peter green wrote: Some time ago I found that a package (I think it was openjdk but I don't remember for sure) which relied on uname -r such that linux32 had to be used to build it in an i386 chroot on an amd64. However since then I'm pretty sure i've seen similar cases with other packages on other architectures being treated as bugs. I think confusion between kernel vs userland architecture is so widespread that we should consider this a necessary part of doing a native build. Please, don't. This breaks cross building. Any use of uname during build other than for logging purposes is a bug. (It should preferably be fixed upstream, to benefit users who build from either Debian or upstream source on a 32/64 environment. But I don't know a simple way to find the 'primary userland architecture' that is not distribution-specific.) There's more than just 32/64. Try this: amd64 kernel, i386 host (and compiler), armel chroot, multiarch with armhf. What should uname report? And this is not even an artificial example: you can't really buy a real i386 machine anymore, yet most semi-proprietary (maemo, android) SDKs ship only i386 binaries, if you want to develop a Debian guest you'd better use armhf instead of armel, and semi-emulated scratchbox style builds can take 44 seconds compared to 8 hours native[1]. I don't do android so never went 4-way, but played with wine on armhf earlier this very week... thanks to how wine is set up in wheezy, that's 3-way. The result of uname wouldn't be too meaningful. In a multiarch world, you can't speak of environment for more than a single executable[2]. One of portable ways to check your target platform is: $CC -dumpmachine (assuming a typical build scheme). [1]. A not so new amd64 box with make -j6 vs armel -j1, on C++ code that caused a swappeathon on the latter. [2]. And even that only because ELF doesn't support fat binaries. -- Copyright and patents were never about promoting culture and innovations; from the very start they were legalized bribes to give the king some income and to let businesses get rid of competition. For some history, please read https://en.wikipedia.org/wiki/Statute_of_Monopolies_1623 signature.asc Description: Digital signature
Re: assumptions about the build environment.
On Fri, Sep 21, 2012 at 08:26:24PM +0100, peter green wrote: I just discovered that on my beagleboard XM (under armhf sid) nacl (which previously build on a debian experimental armhf buildd but not a debian unstable armhf buildd) will build if /sys is mounted but will not build if it is not mounted. Can packages assume that /sys will be mounted in the build environment or not? By default, you get /proc, /dev/pts and /sys mounted. Unless the buildd admin specifically configured they system differently than the defaults (/etc/schroot/buildd/fstab). IIRC it is generally established that packages are not allowed to rely on an internet connection during build but if one is present are they allowed to assume it's non-broken. I recently came accross a package ( sslh ) which fails to build in the presense of nxdomain hijacking. Is that a bug? You are not supposed to rely on any network connectivity during a package build. If it's present, that's just happenstance; it's not guaranteed to be present and/or functional, and you should not be using it under any circumstances. Local loopback is OK though for e.g. unit testing services. Just for the record, I'm planning on adding support for unshare(CLONE_NEWNET) in schroot post wheezy. This will allow the buildd (sbuild) to request that networking be explicitly turned off (bar localhost) during a package build. This will break any buggy packages which are relying on networking during a build. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linuxhttp://people.debian.org/~rleigh/ `. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools `-GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120921211314.gb18...@codelibre.net
Re: assumptions about the build environment.
On Fri, Sep 21, 2012 at 08:26:24PM +0100, peter green wrote: While working on debian one thing I have not managed to find is documentation on what packages can and can't assume about the build environment. Does such documentation exist and if not should it be created. One thing that is at least documented in policy is that the only packages that it can rely on being installed are essential, build-essential and the packages's build-depends. Some specific cases i'm wondering about: I just discovered that on my beagleboard XM (under armhf sid) nacl (which previously build on a debian experimental armhf buildd but not a debian unstable armhf buildd) will build if /sys is mounted but will not build if it is not mounted. Can packages assume that /sys will be mounted in the build environment or not? As far as I know, on all the buildds /sys, /proc, /dev/pts and /dev/shm are available and we get complaints if some of them aren't there. At least /proc and /dev/pts will frequently results in errors, I think there are also at least some packages that require /dev/shm/. I don't remember anything about /sys. I think it would also be a good idea to have this documented in policy if it's not already. IIRC it is generally established that packages are not allowed to rely on an internet connection during build but if one is present are they allowed to assume it's non-broken. I recently came accross a package ( sslh ) which fails to build in the presense of nxdomain hijacking. Is that a bug? It basicly comes down to if they try to download something as source to be build. In that case it's clearly a violation of policy because the source is not in the archive. Some packages then fall back to the source file that's in the package, but they should have always used that in the first place. I know there are also some packages that have a test suite that connects to something on the internet. This doesn't result in changes to the binaries, it just checks that it works properly. You can argue wheter that should be allowed or not, or that the test server should run on localhost. In any case packages doing that shouldn't fail to build because of that and should just skip that test in case the network is down. Some time ago I found that a package (I think it was openjdk but I don't remember for sure) which relied on uname -r such that linux32 had to be used to build it in an i386 chroot on an amd64. However since then I'm pretty sure i've seen similar cases with other packages on other architectures being treated as bugs. They probably should try to use the output of dpkg-architecture to select the arch. Then should never check that output of uname -m. On the buildds we work around this by using linux32 because there were a lot of packages that were broken, and it was easier to work around it. Maybe we should stop doing that. Kurt -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120921222336.ga31...@roeckx.be
Re: assumptions about the build environment.
* peter green plugw...@p10link.net [120921 21:26]: I just discovered that on my beagleboard XM (under armhf sid) nacl (which previously build on a debian experimental armhf buildd but not a debian unstable armhf buildd) will build if /sys is mounted but will not build if it is not mounted. Can packages assume that /sys will be mounted in the build environment or not? I'm quite suprised to see /sys to be mounted in chroots. Wasn't one of the reasons to start /sys and not put the info there in /proc to not have to have it available in chroots? Shouldn't that information about hardware usually be kept away from chroots? Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120921232531.ga20...@client.brlink.eu
Bug#688363: Document assumptions about the build environment.
Package: debian-policy Severity: whishlist Version: 3.9.4.0 X-Debbugs-CC: debian-devel@lists.debian.org Le Sat, Sep 22, 2012 at 12:23:36AM +0200, Kurt Roeckx a écrit : I think it would also be a good idea to have this documented in policy if it's not already. I totally agree. I created a bug (see the number in the title) to track this suggestion. Any volunteer ? -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120922032033.gd8...@falafel.plessy.net
Re: assumptions about the build environment.
On Sat, 2012-09-22 at 01:25 +0200, Bernhard R. Link wrote: * peter green plugw...@p10link.net [120921 21:26]: I just discovered that on my beagleboard XM (under armhf sid) nacl (which previously build on a debian experimental armhf buildd but not a debian unstable armhf buildd) will build if /sys is mounted but will not build if it is not mounted. Can packages assume that /sys will be mounted in the build environment or not? I'm quite suprised to see /sys to be mounted in chroots. Wasn't one of the reasons to start /sys and not put the info there in /proc to not have to have it available in chroots? I've never heard that claimed. Shouldn't that information about hardware usually be kept away from chroots? Chroots aren't containers. A chrooted environment can use all CPUs and all network devices, and programs may expect to find information about them under sysfs. If you're concerned about leaking sensitive information to untrusted processes then procfs is a far, far bigger problem (somewhat mitigated by hidepid or pid namespaces). Ben. -- Ben Hutchings Once a job is fouled up, anything done to improve it makes it worse. signature.asc Description: This is a digitally signed message part
Re: assumptions about the build environment.
On Fri, 2012-09-21 at 23:06 +0200, Adam Borowski wrote: On Fri, Sep 21, 2012 at 09:07:57PM +0100, Ben Hutchings wrote: On Fri, Sep 21, 2012 at 08:26:24PM +0100, peter green wrote: Some time ago I found that a package (I think it was openjdk but I don't remember for sure) which relied on uname -r such that linux32 had to be used to build it in an i386 chroot on an amd64. However since then I'm pretty sure i've seen similar cases with other packages on other architectures being treated as bugs. I think confusion between kernel vs userland architecture is so widespread that we should consider this a necessary part of doing a native build. Please, don't. This breaks cross building. Any use of uname during build other than for logging purposes is a bug. Certainly build systems should support cross-building and should not rely on uname -m. And cross-building is useful for derivatives. But since not all build systems do that, and Debian does not attempt to cross-build its own packages, we should not compromise the reliability of the native build environments. An i386 chroot on an x86_64 kernel should look as much like a plain i386 environment as possible. [...] semi-emulated scratchbox style builds can take 44 seconds compared to 8 hours native[1]. [...] And scratchbox makes cross-build more reliable by presenting a native- looking environment... e.g. by making uname report whatever you want: http://scratchbox.org/documentation/user/scratchbox-1.0/html/installdoc.html#AEN707 Ben. -- Ben Hutchings Once a job is fouled up, anything done to improve it makes it worse. signature.asc Description: This is a digitally signed message part
