Bug#229357: Can we require build-arch/indep targets for lenny?
On Sat, 13 Oct 2007, Frank Lichtenheld wrote: 1) Build-Options field As pointed out this doesn't scale very well and there is no real way to make it default behaviour one day. This would be the way to go though if it only needs to be specified for few packages (either because we think that few packages actually need build-arch support or because of the solution I propose below, combining it with autodetection). IMO combining Build-Options and Standards-Version could be enough to get the best of both worlds. Use Build-Options to declare that the package supports some should rules of the policy but also auto-set some flags based on the version of the policy: if it's greater than the version which changed a rule in a must, then we define the corresponding flag. That way maintainers don't have to carry dozen of options indefinitely. I would be interested to gather some input from the interested persons regarding their exact position. I think the following questions should give us a good understanding or their position: (want == 'I want it and I also think it would be possible to do') 1) Do you want to change sbuild to actually respect policy? Yes. 1a) (SKIP if 'no' to 1) Before lenny's release? I don't mind. 3) Do you want for all packages to support build-arch in the nearer future (longest lenny+1) [== policy 'must']? 4) (SKIP if 'yes' to 3) In the farer future? I think it makes sense in the long term, yes. 5) Do you think autodetection can work and should be used? I must say I'm not very convinced by the various tricks tried and suggested. That said I wouldn't oppose all of them either. 6) (SKIP if 'yes' to 5) Do you think that autodetection can work and should be used, if packages would have the ability to override it if they know they get detected wrong? Seems reasonable, but you still have to let them declare if they support the target build-arch if they declare skip-auto-detection. :-) So basically the process would be: - if Build-Options contains build-arch, call build-arch/indep - unless Build-Options contains skip-auto-detection (or whatever name we come by), do an auto-detection and call build-arch/indep - otherwise you have to call build After considering all the arguments I believe that the best solution would be to try to implement autodetection _and_ support Build-Options to give maintainers the ability to override it. Build-Options is simply the easiest and best-working possibility, but for good behaving packages it should not be neccessary. And I strongly believe that after lenny's release dpkg-buildpackage should start to check the 'correct' build-dependencies according to policy (i.e. requiring b-d-i if build-arch is not supported). I would volunteer to implement the solution I propose (in the near future) if there are enough supporters. Ack from me at least. It's time to take a decision here. First step is to add the Build-Options support IMO. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
Bug#229357: Can we require build-arch/indep targets for lenny?
Hi, Frank Lichtenheld schrieb: 3) Autodetection My approach would be to have the autobuilders use build-arch, and if that fails within 60 seconds, clean and build. If build-arch is not implemented, it fails rather quickly, so we use build and make a note in the build log. Later, one can grep for that note. If it is implemented, but fails within 60 seconds, not much is lost. If it takes longer to fail, then it's a normal FTBFS. Simon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Sat, Sep 29, 2007 at 10:09:19PM +0200, Frank Lichtenheld wrote: On Mon, Jul 02, 2007 at 09:26:23PM -0700, Steve Langasek wrote: Attached is a patch to dpkg which implements a check for a 'build-arch' target using 'make -f debian/rules -qn build-arch'. Is there actually a defined semantic for make -qn ? The make info manual states: It is an error to use more than one of these three flags [-q, -t, -n] in the same invocation of `make'. No answer? I would like to work on this, but someone would need to answer my questions about it... (explicetly sending to vorlon, too, ignoring the M-F-T) Gruesse, -- Frank Lichtenheld [EMAIL PROTECTED] www: http://www.djpig.de/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Fri, Oct 12, 2007 at 10:49:13PM +0200, Frank Lichtenheld wrote: On Sat, Sep 29, 2007 at 10:09:19PM +0200, Frank Lichtenheld wrote: On Mon, Jul 02, 2007 at 09:26:23PM -0700, Steve Langasek wrote: Attached is a patch to dpkg which implements a check for a 'build-arch' target using 'make -f debian/rules -qn build-arch'. Is there actually a defined semantic for make -qn ? The make info manual states: It is an error to use more than one of these three flags [-q, -t, -n] in the same invocation of `make'. No answer? I would like to work on this, but someone would need to answer my questions about it... (explicetly sending to vorlon, too, ignoring the M-F-T) I have no answers for you about whether there are defined semantics for this use of make. :) Anyway, I thought this patch was ruled out in later discussion in the thread. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Fri, Oct 12, 2007 at 02:13:49PM -0700, Steve Langasek wrote: On Fri, Oct 12, 2007 at 10:49:13PM +0200, Frank Lichtenheld wrote: No answer? I would like to work on this, but someone would need to answer my questions about it... (explicetly sending to vorlon, too, ignoring the M-F-T) I have no answers for you about whether there are defined semantics for this use of make. :) Anyway, I thought this patch was ruled out in later discussion in the thread. Hmm, it was opposed by some but I don't see a consensus reached anywhere. Let's try to give a summary of the discussion. So far the proposed solutions for this problem are: 1) Build-Options field As pointed out this doesn't scale very well and there is no real way to make it default behaviour one day. This would be the way to go though if it only needs to be specified for few packages (either because we think that few packages actually need build-arch support or because of the solution I propose below, combining it with autodetection). I'm not particulary impressed with this. 2) Standards-Version, i.e. make it 'must' in policy This should work but it completly contradicts the past Debian standards process (according to Lucas' numbers, the new policy would currently only be satisfied by 2000 packages, i.e. somewhere in the 20% region) and would make a solution dependant on the policy team, which is currently somewhat MIA... It would be really nice to have a policy someday that acutally matches reality, though. 3) Autodetection Would be clearly the easiest solution if it works reliably. There are some problems: - Works only with GNU make - depends on a _should_ in policy, not a _must_ (error code) On the other hand, according to http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=229357#172 it mostly works, and most of the cases where it doesn't work seem to be the packages fault (i.e. the binary-arch target seems to depend on the build-indep target without actually declaring that dependency). BTW, the correct solution in any case would be to run checkbuilddeps again if we don't find build-arch support, since we need b-d-i, too. And *poof*, there go our buildds ;) which brings me to the last solution which I think wasn't actually proposed: 4) Adapt policy to sbuilds behaviour I.e. don't require b-d-i on build, but only on binary and binary-indep. Conclusion: I would be interested to gather some input from the interested persons regarding their exact position. I think the following questions should give us a good understanding or their position: (want == 'I want it and I also think it would be possible to do') 1) Do you want to change sbuild to actually respect policy? 1a) (SKIP if 'no' to 1) Before lenny's release? 2) (SKIP if 'yes' to 1) Do you want to change policy to declare sbuild's behaviour official? 3) Do you want for all packages to support build-arch in the nearer future (longest lenny+1) [== policy 'must']? 4) (SKIP if 'yes' to 3) In the farer future? 5) Do you think autodetection can work and should be used? 6) (SKIP if 'yes' to 5) Do you think that autodetection can work and should be used, if packages would have the ability to override it if they know they get detected wrong? My answers are: YN-NNNY After considering all the arguments I believe that the best solution would be to try to implement autodetection _and_ support Build-Options to give maintainers the ability to override it. Build-Options is simply the easiest and best-working possibility, but for good behaving packages it should not be neccessary. And I strongly believe that after lenny's release dpkg-buildpackage should start to check the 'correct' build-dependencies according to policy (i.e. requiring b-d-i if build-arch is not supported). I would volunteer to implement the solution I propose (in the near future) if there are enough supporters. Gruesse, -- Frank Lichtenheld [EMAIL PROTECTED] www: http://www.djpig.de/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Mon, Jul 02, 2007 at 09:26:23PM -0700, Steve Langasek wrote: Attached is a patch to dpkg which implements a check for a 'build-arch' target using 'make -f debian/rules -qn build-arch'. Is there actually a defined semantic for make -qn ? The make info manual states: It is an error to use more than one of these three flags [-q, -t, -n] in the same invocation of `make'. Gruesse, -- Frank Lichtenheld [EMAIL PROTECTED] www: http://www.djpig.de/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Wouter Verhelst [EMAIL PROTECTED] writes: On Fri, Jun 29, 2007 at 11:11:04PM +0200, Bill Allombert wrote: One of the issue is that tools like sbuild and pbuilder which want to take advantage of the Build-Depends-Indep split needs to know whether dpkg-buildpackage will call debian/rules build or build-arch. It needs to know no such thing. It just needs to know that if it runs dpkg-buildpackage -b, only .debs will be generated, and if it runs dpkg-buildpackage -B, all debs apart from the _all.debs will be generated. How exactly this happens is of no concern to sbuild. It does when Build-Depends is used as specified in policy and not like it is (brokenly) used now: Policy 7.6: | Build-Depends, Build-Conflicts | |The Build-Depends and Build-Conflicts fields must be satisfied |when any of the following targets is invoked: build, clean, |binary, binary-arch, build-arch, build-indep and binary-indep. | | Build-Depends-Indep, Build-Conflicts-Indep | | The Build-Depends-Indep and Build-Conflicts-Indep fields must be | satisfied when any of the following targets is invoked: build, | build-indep, binary and binary-indep. That means that is -b is used or if the package does NOT support build-arch then sbuild must install Build-Depends-Indep as well. If build-arch remains optional and policy for Build-Depends remains unchanged then sbuild must now. Making build-arch required or changing the meaning of Build-Depends to the broken use we have now resolves this. I favor making build-arch required. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Russ Allbery writes (Re: Can we require build-arch/indep targets for lenny?): Lo?c Minier [EMAIL PROTECTED] writes: Why not promote these to requirements in a particular policy version instead? I fear we will have to list 10 Build-Options in all packages in a couple of years. This is a much better idea. Currently, policy says that it's recommended (the weakest policy directive) to support noopt and nostrip. My main concern with increasing the strength of that directive is that, depending on how demented the upstream build system is, it can be difficult to support these options, and since neither is used for regular builds in Debian, they're not usually tested and aren't necessary for properly functioning packages. Surely we are planning to support these options in all packages eventually ? In a package where it is difficult to separate out the work for binary-indep, it would be acceptable to say: binary-indep: binary binary-arch: binary binary: build some stuff ? I'm tempted to suggest _just_ going by the package's Standards-Version. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Wouter Verhelst [EMAIL PROTECTED] writes: On Mon, Jul 02, 2007 at 03:28:05PM +0200, Simon Richter wrote: Running debian-rules can always have side effects and can actively rely on them so a --has-target can not be implemented cleanly in make. I am proposing hooking into the logic that ultimately decides that there is no such target in the Makefile and goes on to print Don't know how to make 'foo'. Stop.. $(shell ls temp-target-* rm temp-target-*): Yes, that's broken, but there are your side effects, and you'll have to run this code if you want to make your --has-target work. Or just a simple include debian/rules.gen while that is generated as needed. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Hi, Ian Jackson wrote: We want the package to _declare_ whether it supports this new target. Ideally, we'd want all packages to support the new target, and then turn that into policy, otherwise we'll end up supporting both for a very long time. The proposed -Options field will actually be very useful for any other enhancements we make to the source package format. I'm not disputing that, but I'm not sure we really want it in this case. Simon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Hi, Wouter Verhelst wrote: $(shell ls temp-target-* rm temp-target-*): Yes, that's broken, but there are your side effects, and you'll have to run this code if you want to make your --has-target work. Yes, that is exactly what I'm proposing. The same thing happens for -q mode now. Simon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On 02/07/07 at 21:26 -0700, Steve Langasek wrote: Lucas has agreed to doing a full archive rebuild with a modified dpkg-dev, for comparison with the previous test. A dpkg-dev binary including this change can be found at http://people.debian.org/~vorlon/dpkg-dev_1.14.4-0.1_all.deb. Hi, Here are some results: 7299 packages from unstable/main were rebuilt (that's all packages building non-arch:all packages). 1823 packages were built using 'debian/rules build-arch'. Of those 1823 packages, 31 packages failed to build. Logs can be found on http://people.debian.org/~lucas/logs/2007/07/04/ . Regarding build time, it's difficult to compare, because the most recent data I have was generated on a different set of cluster nodes, and the nodes I used for this have a slower network connection. Also, the mirror's disk is a bottleneck since I was using 55 nodes. But some packages seem to benefit from using build-arch despite that. See http://people.debian.org/~lucas/logs/2007/07/04/00impr_bt.txt . Previous and current build times are the 8th and 9th columns. There might be others, that /built/ faster, but that took a longer time to fetch build-deps because of the network/mirror. The full listing of the results is on http://people.debian.org/~lucas/logs/2007/07/04/00summary.txt , with the columns being: 1: package 2, 3, 4: previous, current version, and whether they differ 5, 6, 7: previous, current result, and whether they differ 8, 9, 10: previous, current result, and whether they differ (incl. margin of error) 11, 12, 13: previous, current reason for build failure, and whether they differ. So, to conclude: this change seems like a good idea, with only about 30 packages to fix (but they should probably be fixed anyway). Not so many packages benefit from it currently, but there was no reason until now for packages to implement that. -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Mon, Jul 02, 2007 at 09:26:23PM -0700, Steve Langasek wrote: I believe the attached patch has the following characteristics: - Behavior on systems where 'make' is not GNU make is undefined. Specifically, on such a system dpkg is likely to either conclude that /all/ packages support 'build-arch', or that /none/ of them support 'build-arch', depending on whether and how 'make -qn' fails. Too bad, both Solaris and BSD make fail the bad way, returning 1 on a bad target -- and neither has -v or --version. I haven't checked the rest, but it's likely they behave the same way. So, an idea: what about checking make -f /dev/null blah 2/dev/null first, for some portability? -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Tue, Jul 03, 2007 at 09:54:03AM +0200, Adam Borowski wrote: On Mon, Jul 02, 2007 at 09:26:23PM -0700, Steve Langasek wrote: I believe the attached patch has the following characteristics: - Behavior on systems where 'make' is not GNU make is undefined. Specifically, on such a system dpkg is likely to either conclude that /all/ packages support 'build-arch', or that /none/ of them support 'build-arch', depending on whether and how 'make -qn' fails. Too bad, both Solaris and BSD make fail the bad way, returning 1 on a bad target -- and neither has -v or --version. I haven't checked the rest, but it's likely they behave the same way. So, an idea: what about checking make -f /dev/null blah 2/dev/null first, for some portability? What 'blah' are you planning to use that's guaranteed to not have broken side-effects in some cases on Debian packages? -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Simon Richter writes (Re: Bug#229357: Can we require build-arch/indep targets for lenny?): The entire issue circles around not being able to reliably detect whether the target is present using a simple script. But who said it has to be a script? We want the package to _declare_ whether it supports this new target. The proposed -Options field will actually be very useful for any other enhancements we make to the source package format. Proposed alternative solution: 1. hack GNU make to support a --has-target option that returns an appropriate return code (hey, it's free software, after all!). Proposed return codes are 0 (yes), 1 (no) and 2 (error). OMG WTF BBQ (ie, this is a terrible suggestion) Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Steve Langasek writes (Re: Bug#229357: Can we require build-arch/indep targets for lenny?): Attached is a patch to dpkg which implements a check for a 'build-arch' target using 'make -f debian/rules -qn build-arch'. Why are we so resistant to the new debian/control field ? That doesn't require any of this messing about with make. Note that the current setup does not actually require debian/rules to be a makefile. I don't think we should introduce software which has a requirement if we can avoid it. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Tue, Jul 03, 2007 at 06:07:54PM +0100, Ian Jackson wrote: Steve Langasek writes (Re: Bug#229357: Can we require build-arch/indep targets for lenny?): Attached is a patch to dpkg which implements a check for a 'build-arch' target using 'make -f debian/rules -qn build-arch'. Why are we so resistant to the new debian/control field ? That doesn't require any of this messing about with make. But it does require the maintainer to keep three bits of information in sync: the new declarative Build-Options field, the build-arch target, and the Build-Depends field. That's added complexity which means an added opportunity for bugs, so if the complexity can be avoided I think it's worthwhile. If the dpkg maintainers feel that this autodetection isn't adequate, I do support implementing build-arch by way of Build-Options. The benefits would be realized more slowly, but they would be realized, and without the insanity of making 75% of our packages FTBFS in unstable first. Note that the current setup does not actually require debian/rules to be a makefile. I don't think we should introduce software which has a requirement if we can avoid it. This doesn't require debian/rules to be a makefile either (though Policy does), it just requires that debian/rules be a makefile *if* the package implements build-arch and uses the corresponding semantics for Build-Depends. Anyway, for the perverse, the following is a valid makefile and a valid shell script. ;) #!/bin/sh fakeout= build-arch: case $1 in build-arch) echo whee fun. ;; esac -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Tue, Jul 03, 2007, Steve Langasek wrote: So, an idea: what about checking make -f /dev/null blah 2/dev/null first, for some portability? What 'blah' are you planning to use that's guaranteed to not have broken side-effects in some cases on Debian packages? How about: blah$((uptime;ps aux) | sha1sum | cut -f1 -d-) Cheers, -- Sam. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Tue, Jul 03, 2007 at 10:01:47AM -0700, Steve Langasek wrote: On Tue, Jul 03, 2007 at 09:54:03AM +0200, Adam Borowski wrote: So, an idea: what about checking make -f /dev/null blah 2/dev/null first, for some portability? What 'blah' are you planning to use that's guaranteed to not have broken side-effects in some cases on Debian packages? Any 'blah' which is not found in /dev/null, I think. I'm not aware of any local files which can break 'make -f'. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Fri, Jun 29, 2007 at 11:11:04PM +0200, Bill Allombert wrote: One of the issue is that tools like sbuild and pbuilder which want to take advantage of the Build-Depends-Indep split needs to know whether dpkg-buildpackage will call debian/rules build or build-arch. It needs to know no such thing. It just needs to know that if it runs dpkg-buildpackage -b, only .debs will be generated, and if it runs dpkg-buildpackage -B, all debs apart from the _all.debs will be generated. How exactly this happens is of no concern to sbuild. -- Lo-lan-do Home is where you have to wash the dishes. -- #debian-devel, Freenode, 2004-09-22 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Mon, Jul 02, 2007 at 03:28:05PM +0200, Simon Richter wrote: Running debian-rules can always have side effects and can actively rely on them so a --has-target can not be implemented cleanly in make. I am proposing hooking into the logic that ultimately decides that there is no such target in the Makefile and goes on to print Don't know how to make 'foo'. Stop.. $(shell ls temp-target-* rm temp-target-*): Yes, that's broken, but there are your side effects, and you'll have to run this code if you want to make your --has-target work. -- Lo-lan-do Home is where you have to wash the dishes. -- #debian-devel, Freenode, 2004-09-22 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Hi, Andreas Metzler wrote: --- Somehow make dpkg-buildpackage -B make use of the build-arch target if present. Either by detecting it automatically or by something like #229357. --- The entire issue circles around not being able to reliably detect whether the target is present using a simple script. But who said it has to be a script? Proposed alternative solution: 1. hack GNU make to support a --has-target option that returns an appropriate return code (hey, it's free software, after all!). Proposed return codes are 0 (yes), 1 (no) and 2 (error). 2. make dpkg-buildpackage -B use this facility to determine whether the target is present. If yes, use the build-arch target to build; if no, use the build target after printing a warning. 3. grep the build logs for warnings about missing build-arch target, add an entry to the TODO list on the package overview page on qa.debian.org and to DDPO. The only remaining question is what should happen with build failures in packages that provide a non-functional build-arch target. In my opinion, this is a genuine bug, even if policy can be read in a way that makes it disagree. Simon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Simon Richter [EMAIL PROTECTED] writes: Hi, Andreas Metzler wrote: --- Somehow make dpkg-buildpackage -B make use of the build-arch target if present. Either by detecting it automatically or by something like #229357. --- The entire issue circles around not being able to reliably detect whether the target is present using a simple script. But who said it has to be a script? Proposed alternative solution: 1. hack GNU make to support a --has-target option that returns an appropriate return code (hey, it's free software, after all!). Proposed return codes are 0 (yes), 1 (no) and 2 (error). 2. make dpkg-buildpackage -B use this facility to determine whether the target is present. If yes, use the build-arch target to build; if no, use the build target after printing a warning. 3. grep the build logs for warnings about missing build-arch target, add an entry to the TODO list on the package overview page on qa.debian.org and to DDPO. The only remaining question is what should happen with build failures in packages that provide a non-functional build-arch target. In my opinion, this is a genuine bug, even if policy can be read in a way that makes it disagree. Simon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] There are two points to this: 1.) don't call build so build-indep is not called 2.) have Build-Depends only contain packages needed for build-arch and binary-arch The seconds part requires that tools like sbuild and pbuilder know beforehand if build or build-arch will be used. Running debian-rules can always have side effects and can actively rely on them so a --has-target can not be implemented cleanly in make. So you missed the point. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Hello, Goswin von Brederlow wrote: The seconds part requires that tools like sbuild and pbuilder know beforehand if build or build-arch will be used. For packages that do not implement build-arch, it is acceptable to call the build target with only B-D installed, because that is the way the autobuilders handle it now. So no problem there; packages that implement build-arch can move the dependencies not needed for building arch-dependent packages from B-D to B-D-I as soon as the autobuilders start using build-arch. Getting rid of unneeded build dependencies is mostly orthogonal to the issue at hand, though. Running debian-rules can always have side effects and can actively rely on them so a --has-target can not be implemented cleanly in make. I am proposing hooking into the logic that ultimately decides that there is no such target in the Makefile and goes on to print Don't know how to make 'foo'. Stop.. This means that Makefiles are rebuilt before that test is performed, we stop immediately before the point where we would go towards the first goal target. Yes, that means running commands that possibly have side effects. But we are going to run these commands anyway. Simon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Sun, Jul 01, 2007 at 05:22:31PM +0200, Bill Allombert wrote: On Sat, Jun 30, 2007 at 09:40:29AM +0200, Andreas Metzler wrote: I think that is just wrong. sbuild should not need to know anything about dpkg-buildpackage's internals and there is no need for change here. The currently used and proven interface is: 1. install Build-Depends for running dpkg-buildpackage -B The issue we are trying to fix is that the current combination of Debian policy and dpkg-buildpackage actually require Build-Depends-Indep to be installed when running dpkg-buildpackage -B. Indeed dpkg-buildpackage -B calls 'debian/rules build' which requires Build-Depends-Indep to be installed. Since buildd actually implement 1) this cause packages to FTBFS until they are 'fixed' by having 'build' not depending on 'build-indep' (or equivalently, the build-indep task being performed directly by binary-indep), which is against the spirit of policy (because build-indep will then be executed under sudo or fakeroot). So this interface is used and proven to be wrong. Attached is a patch to dpkg which implements a check for a 'build-arch' target using 'make -f debian/rules -qn build-arch'. I looked into using make -pn, but Guillem Jover pointed out that this doesn't work if the 'build-arch' target is implemented using patterns. '-qn' appears the most reliable option; this is the same basic technique that was attempted once before and reverted by Adam Heath, the dpkg maintainer at the time, so I spoke with him about the reasons for the revert and it appears the concerns are mostly about dpkg behavior on systems that do not conform to Debian policy. I don't think that's a good enough reason to stall innovation that benefits Debian, but perhaps the current dpkg maintainers do; I'll try to lay out the technical details impartially for their consideration so they can make an informed decision. I believe the attached patch has the following characteristics: - Any packages in Debian that currently have existing but /broken/ 'build-arch' targets will fail to build because of the failure of this target. - Any packages that have a debian/rules which is not a valid Makefile will continue to build as before (the new code will conclude that there is no valid 'build-arch' target). - Packages in Debian that have a 'build-arch' target will have this target used instead of 'build' when 'dpkg-buildpackage -B' is called. - Packages in Debian that do not have a 'build-arch' target will continue to have their 'build' target called by 'dpkg-buildpackage -B'. - Behavior on systems where 'make' is not GNU make is undefined. Specifically, on such a system dpkg is likely to either conclude that /all/ packages support 'build-arch', or that /none/ of them support 'build-arch', depending on whether and how 'make -qn' fails. This has the following further implications: - Barring any buggy 'build-arch' targets, all packages that currently build on Debian autobuilders should continue to build successfully and correctly. - Packages that do support 'build-arch' today will also build faster, because the indep build will now be skipped. - Packages that do not yet support 'build-arch' can adopt this target without any need for coordinated changes on the buildds. - Packages that include in their Build-Depends field build-dependencies which are only needed for the architecture-independent portion of the 'build' target can move these build-dependencies to Build-Depends-Indep as soon as they have a 'binary-arch' target that does not depend on the 'build' target, and a working 'build-arch' target that does not perform the related architecture-independent build operations. Packages that make this change to their Build-Depends field should also build-depend on the version of dpkg-dev introducing these semantics. Documenting this in policy should be straightforward if these changes to dpkg are accepted. Lucas has agreed to doing a full archive rebuild with a modified dpkg-dev, for comparison with the previous test. A dpkg-dev binary including this change can be found at http://people.debian.org/~vorlon/dpkg-dev_1.14.4-0.1_all.deb. Cheers, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ diff -Nru /tmp/MWAqJMfVFu/dpkg-1.14.4/debian/changelog /tmp/V6mOrMQZzb/dpkg-1.14.4/debian/changelog --- /tmp/MWAqJMfVFu/dpkg-1.14.4/debian/changelog 2007-05-24 09:30:41.0 -0700 +++ /tmp/V6mOrMQZzb/dpkg-1.14.4/debian/changelog 2007-07-02 12:28:01.0 -0700 @@ -1,3 +1,14 @@ +dpkg (1.14.4-0.1) unstable; urgency=low + + * Non-maintainer upload. + * Detect 'build-arch' target if present using make -f debian/rules -qn, and +use it instead of 'build' when called with -B. This allows maintainers +who have a build-arch target to use
Bug#229357: Can we require build-arch/indep targets for lenny?
On Sat, Jun 30, 2007 at 09:40:29AM +0200, Andreas Metzler wrote: I think that is just wrong. sbuild should not need to know anything about dpkg-buildpackage's internals and there is no need for change here. The currently used and proven interface is: 1. install Build-Depends for running dpkg-buildpackage -B The issue we are trying to fix is that the current combination of Debian policy and dpkg-buildpackage actually require Build-Depends-Indep to be installed when running dpkg-buildpackage -B. Indeed dpkg-buildpackage -B calls 'debian/rules build' which requires Build-Depends-Indep to be installed. Since buildd actually implement 1) this cause packages to FTBFS until they are 'fixed' by having 'build' not depending on 'build-indep' (or equivalently, the build-indep task being performed directly by binary-indep), which is against the spirit of policy (because build-indep will then be executed under sudo or fakeroot). So this interface is used and proven to be wrong. 2. install Build-Depends *and* Build-Indep-Indep for running dpkg-buildpackage differently (e.g without any modifier or with -b) If you insist to go that road, you need to: 1) Change policy to require build-arch is implemented anytime the field Build-Depends-Indep is provided. and 2) Change dpkg-buildpackage -B to call build-arch if the field Build-Depends-Indep is present. Please submit patches if you are interested. However this proposal will cause a number packages to FTBFS until they are fixed (but less than always using build-arch). See http://bugs.debian.org/218893 for any additional details. I am merely restating the issues. Cheers, -- Bill. [EMAIL PROTECTED] Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Bill Allombert wrote: On Sat, Jun 30, 2007 at 09:40:29AM +0200, Andreas Metzler wrote: I think that is just wrong. sbuild should not need to know anything about dpkg-buildpackage's internals and there is no need for change here. The currently used and proven interface is: 1. install Build-Depends for running dpkg-buildpackage -B The issue we are trying to fix is that the current combination of Debian policy and dpkg-buildpackage actually require Build-Depends-Indep to be installed when running dpkg-buildpackage -B. Hello, Policy does not reflect current reality in that respect. The buildds do run dpkg-buildpackage -B and they do not install Build-Depends-Indep. Packages requiring Build-Depends-Indep for dpkg-buildpackage -B will FTBFS. Since that makes the package unreleasable there are not many packages around that work like that. (Except for source packages that do not build any arch-any packages and therefore are not autobuilt.) [snip] 2. install Build-Depends *and* Build-Indep-Indep for running dpkg-buildpackage differently (e.g without any modifier or with -b) If you insist to go that road, you need to: 1) Change policy to require build-arch is implemented anytime the field Build-Depends-Indep is provided. and 2) Change dpkg-buildpackage -B to call build-arch if the field Build-Depends-Indep is present. [...] No, that is not necessary. What needs to happen is just: --- Somehow make dpkg-buildpackage -B make use of the build-arch target if present. Either by detecting it automatically or by something like #229357. --- Once that happens the current wording in policy matches reality for packages proving a build-arch target. cu andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
In article [EMAIL PROTECTED] (gmane.linux.debian.devel.general) you wrote: On Fri, Jun 29, 2007 at 12:41:04AM +0300, Guillem Jover wrote: [...] I've been pondering on what's the cleanest way to fix it for some time, and I tend to agree with Steve about using the make options to test for the existence of the targets. But as others have pointed out it's not clear why that change was reverted at the time. http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=345;att=0;bug=218893 | -q checks for up to dateness ... the next time you ran the test, it | would fail because build-stamp had been touched and dpkg-buildpackage | would incorrectly run build for you instead. One of the issue is that tools like sbuild and pbuilder which want to take advantage of the Build-Depends-Indep split needs to know whether dpkg-buildpackage will call debian/rules build or build-arch. So if you go that route, the exact criterium used by dpkg-buildpackage need to be published as an interface. I think that is just wrong. sbuild should not need to know anything about dpkg-buildpackage's internals and there is no need for change here. The currently used and proven interface is: 1. install Build-Depends for running dpkg-buildpackage -B 2. install Build-Depends *and* Build-Indep-Indep for running dpkg-buildpackage differently (e.g without any modifier or with -b) cu andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Fri, Jun 29, 2007 at 12:41:04AM +0300, Guillem Jover wrote: Obviously the dpkg developers are rather busy at the moment. I think that the right thing to do is to offer to NMU. Errr, what's the rush now to get this fixed? It's something that has been there like forever, the bug report proposes adding a new field which personally I don't like taking lightly. This field does not need to be exported to the Source file, so it does not have a large impact. I've been pondering on what's the cleanest way to fix it for some time, and I tend to agree with Steve about using the make options to test for the existence of the targets. But as others have pointed out it's not clear why that change was reverted at the time. One of the issue is that tools like sbuild and pbuilder which want to take advantage of the Build-Depends-Indep split needs to know whether dpkg-buildpackage will call debian/rules build or build-arch. So if you go that route, the exact criterium used by dpkg-buildpackage need to be published as an interface. Any additional detail is available at http://bugs.debian.org/218893 Cheers, -- Bill. [EMAIL PROTECTED] Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Hi, On Tue, 2007-06-26 at 14:33:26 +0100, Ian Jackson wrote: Bill Allombert writes (Re: Can we require build-arch/indep targets for lenny?): At least, I would feel less alone. FWIW, I agree with you. I think the proposed `Build-Options' source control field is a sensible addition and the bug should be implemented immediately. Obviously the dpkg developers are rather busy at the moment. I think that the right thing to do is to offer to NMU. Errr, what's the rush now to get this fixed? It's something that has been there like forever, the bug report proposes adding a new field which personally I don't like taking lightly. I've been pondering on what's the cleanest way to fix it for some time, and I tend to agree with Steve about using the make options to test for the existence of the targets. But as others have pointed out it's not clear why that change was reverted at the time. regards, guillem -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Tue, Jun 26, 2007, Joey Hess wrote: I think it would also be useful to include 'nostrip' and 'noopt' in the Build-Options field, as a way to indicate that the package implements those DEB_BUILD_OPTIONS. I also have some Evil Plans for other things that can go in Build-Options, but they're not ready yet and would be OT in this thread. Why not promote these to requirements in a particular policy version instead? I fear we will have to list 10 Build-Options in all packages in a couple of years. -- Loïc Minier
Bug#229357: Can we require build-arch/indep targets for lenny?
Loïc Minier [EMAIL PROTECTED] writes: On Tue, Jun 26, 2007, Joey Hess wrote: I think it would also be useful to include 'nostrip' and 'noopt' in the Build-Options field, as a way to indicate that the package implements those DEB_BUILD_OPTIONS. I also have some Evil Plans for other things that can go in Build-Options, but they're not ready yet and would be OT in this thread. Why not promote these to requirements in a particular policy version instead? I fear we will have to list 10 Build-Options in all packages in a couple of years. Currently, policy says that it's recommended (the weakest policy directive) to support noopt and nostrip. My main concern with increasing the strength of that directive is that, depending on how demented the upstream build system is, it can be difficult to support these options, and since neither is used for regular builds in Debian, they're not usually tested and aren't necessary for properly functioning packages. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/
Bug#229357: Can we require build-arch/indep targets for lenny?
Bill Allombert writes (Re: Can we require build-arch/indep targets for lenny?): In 3 years and a half, I had the time to try all of that... So I will try something new: an online petition: If you would like bug #229357 to get an answer, please send a signed email to the buglog. Please, this is no way to carry on. At least, I would feel less alone. FWIW, I agree with you. I think the proposed `Build-Options' source control field is a sensible addition and the bug should be implemented immediately. Obviously the dpkg developers are rather busy at the moment. I think that the right thing to do is to offer to NMU. While we are at it we should write a specification for Build-Options, something like: The Build-Options field appears (only) in the first stanza in debian/control. It gives a whitespace-separated list of options. The meanings of these options is defined in policy. Any package processing tool may act only on options which it recognises. Unknown tokens must be ignored. Currently only the following token is defined: * build-arch Declares that the package supports all of the following build targets: `build-indep', `build-arch', `binary-indep', `binary-arch'. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
On Tue, Jun 26, 2007 at 02:33:26PM +0100, Ian Jackson wrote: Bill Allombert writes (Re: Can we require build-arch/indep targets for lenny?): In 3 years and a half, I had the time to try all of that... So I will try something new: an online petition: If you would like bug #229357 to get an answer, please send a signed email to the buglog. Please, this is no way to carry on. Ironically, you are the only one to do that so far, the fact that you did not sign your post notwithstanding. At least, I would feel less alone. FWIW, I agree with you. I think the proposed `Build-Options' source control field is a sensible addition and the bug should be implemented immediately. Obviously the dpkg developers are rather busy at the moment. I think that the right thing to do is to offer to NMU. So I hereby offer to do a NMU by applying this patch. While we are at it we should write a specification for Build-Options, something like: The Build-Options field appears (only) in the first stanza in debian/control. It gives a whitespace-separated list of options. The meanings of these options is defined in policy. Any package processing tool may act only on options which it recognises. Unknown tokens must be ignored. Currently only the following token is defined: * build-arch Declares that the package supports all of the following build targets: `build-indep', `build-arch', `binary-indep', `binary-arch'. Note: binary-indep and binary-arch are already mandatory according to Debian policy 4.9. The specification are included in the patch to debian-policy in bug #218893, msgid [EMAIL PROTECTED], specifically +sect1 id=f-Build-Options +headingttBuild-Options/tt/heading +p + The syntax is a list of options separated by + commas that are implemented in the build process. + The following options are defined: + list + item ttbuild-arch/tt The optional targets build-arch + and build-indep are implemented by ttdebian/rules/tt + as defined in ref id=debianrules. /item + /list +/p +/sect1 Thanks for yours answer, -- Bill. [EMAIL PROTECTED] Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Bill Allombert wrote: + The syntax is a list of options separated by + commas that are implemented in the build process. + The following options are defined: If commas are used as delimiters, it should use , as the delimiter for consistency with other fields using commas as delimiters. Since debian/control has both space and comma-delimited fields, I have no real preference which is chosen. Also, I like Ian's language about all unknown fields being ignored. -- see shy jo signature.asc Description: Digital signature
Bug#229357: Can we require build-arch/indep targets for lenny?
Joey Hess [EMAIL PROTECTED] writes: I think it would also be useful to include 'nostrip' and 'noopt' in the Build-Options field, as a way to indicate that the package implements those DEB_BUILD_OPTIONS. parallel=n as well, while we're at it. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#229357: Can we require build-arch/indep targets for lenny?
Ian Jackson wrote: While we are at it we should write a specification for Build-Options, something like: The Build-Options field appears (only) in the first stanza in debian/control. It gives a whitespace-separated list of options. The meanings of these options is defined in policy. Any package processing tool may act only on options which it recognises. Unknown tokens must be ignored. Currently only the following token is defined: * build-arch Declares that the package supports all of the following build targets: `build-indep', `build-arch', `binary-indep', `binary-arch'. Funny, I'd forgotten this was ever proposed before, and was planning to propose adding a Build-Options field for entirely other, though fully compatible reasons. Which suggests that the name and format are well chosen. I think it would also be useful to include 'nostrip' and 'noopt' in the Build-Options field, as a way to indicate that the package implements those DEB_BUILD_OPTIONS. I also have some Evil Plans for other things that can go in Build-Options, but they're not ready yet and would be OT in this thread. -- see shy jo signature.asc Description: Digital signature
