Re: many packages fail to build twice in a row again
* Charles Plessy ple...@debian.org, 2011-12-21, 10:30: Personally I have given up making packages buildable twice in a row using “debian clean” for the packages I maintain with git, Thanks for the heads-up. I will avoid your packages when doing QA work. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111223092557.ga3...@jwilk.net
Re: many packages fail to build twice in a row again
On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Yes, I'd expect that the situation is much worse than it used to be. Not only because of dpkg-source changes, but also due to ubiquity of build tools that throw away build tree after build (pbuilder, sbuild, $VCS-buildpackage...). On 20/12/11 at 23:16 +, brian m. carlson wrote: If I'm fixing an RC bug, it's very convenient to be able to test a patch and then rebuild with a different patch if necessary. If the package doesn't build cleanly N times in a row, or if there are other problems with the packaging that make it difficult to fix, I usually go on to fix other bugs instead. Also, when I file a bug report, the harder you make it to fix your package, the less likely you are to receive a patch with that report, workaround or not. Same here. * Lucas Nussbaum lu...@lucas-nussbaum.net, 2011-12-21, 10:41: I'm not saying that it's not convenient. But on the other hand, several hundreds of packages probably need to be fixed, which will require a large amount of manpower Well, hopefully there are still some non-MIA maintainers left, that can take care of their own packages. And it's not like these bugs would be RC, or that we'd have a dead-line to fix them all. Also, such bugs would be useful even if they were not going to be fixed, because they'd act as a warning for people trying to modify the package. that could be used instead to fix problems affecting our users. You seem to assume that this manpower is somehow transferable to other tasks. I'm afraid that the result of telling people don't fix FOO, do fix BAR instead can be that neither FOO[0] nor BAR[1] is fixed. [0] Because you just demotivated a person willing to FOO. [1] Because the person wasn't interested in fixing BAR in the first place. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111223101634.ga4...@jwilk.net
Re: many packages fail to build twice in a row again
On Fri, Dec 23, 2011 at 11:16:34AM +0100, Jakub Wilk wrote: On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Yes, I'd expect that the situation is much worse than it used to be. Not only because of dpkg-source changes, but also due to ubiquity of build tools that throw away build tree after build (pbuilder, sbuild, $VCS-buildpackage...). There's an outstanding feature request for sbuild to do double builds to detect such problems. It wouldn't be too much trouble to implement; it's just not been a high priority. I've noticed problems over the last few weeks with the new sources formats causing dirty build trees more often than not. Not sure if these problems need fixing in each package, or if changes to the tools will correct the vast majority of them. The suggestion that git clean be a solution appears to have caused some level of outrage. However, at least for '3.0 (git)', all the sources are known to git, and 'git clean' is a reliable and simple solution to the problem. The alternative, manually reverting all the changes, is both complex and error-prone. I'm not sure I see the problem with what is an obvious improvement to the process. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111223104107.gn5...@codelibre.net
Re: many packages fail to build twice in a row again
On Fri, Dec 23, 2011 at 10:41:07AM +, Roger Leigh wrote: The suggestion that git clean be a solution appears to have caused some level of outrage. However, at least for '3.0 (git)', all the sources are known to git, and 'git clean' is a reliable and simple solution to the problem. Apparently git clean isn't enough: On Wed, 21 Dec 2011 10:30:53 +0100, Charles Plessy wrote: “git clean” and “git checkout” could be ran from the clean target of debian/rules I've spent time trying to debug a package that mysteriously reverted my changes before. It's definitely not compatible with the principle of least surprise. Steve -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111223114124.ga21...@s.cotton.clara.co.uk
Re: many packages fail to build twice in a row again
On Fri, Dec 23, 2011 at 10:41:07AM +, Roger Leigh wrote: The suggestion that git clean be a solution appears to have caused some level of outrage. However, at least for '3.0 (git)', all the sources are known to git, and 'git clean' is a reliable and simple solution to the problem. The alternative, manually reverting all the changes, is both complex and error-prone. I'm not sure I see the problem with what is an obvious improvement to the process. I'd favor a solution that avoids having to fix hundres, or thousands, of upstream packages. For example, instead of building directly in the working tree, we could export the sources to a temporary directory and build there. Voila: no build artifacts to clean up at all. -- Freedom-based blog/wiki/web hosting: http://www.branchable.com/ signature.asc Description: Digital signature
Re: many packages fail to build twice in a row again
On 23/12/11 at 11:16 +0100, Jakub Wilk wrote: On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Yes, I'd expect that the situation is much worse than it used to be. Not only because of dpkg-source changes, but also due to ubiquity of build tools that throw away build tree after build (pbuilder, sbuild, $VCS-buildpackage...). On 20/12/11 at 23:16 +, brian m. carlson wrote: If I'm fixing an RC bug, it's very convenient to be able to test a patch and then rebuild with a different patch if necessary. If the package doesn't build cleanly N times in a row, or if there are other problems with the packaging that make it difficult to fix, I usually go on to fix other bugs instead. Also, when I file a bug report, the harder you make it to fix your package, the less likely you are to receive a patch with that report, workaround or not. Same here. * Lucas Nussbaum lu...@lucas-nussbaum.net, 2011-12-21, 10:41: I'm not saying that it's not convenient. But on the other hand, several hundreds of packages probably need to be fixed, which will require a large amount of manpower Well, hopefully there are still some non-MIA maintainers left, that can take care of their own packages. And it's not like these bugs would be RC, or that we'd have a dead-line to fix them all. Also, such bugs would be useful even if they were not going to be fixed, because they'd act as a warning for people trying to modify the package. that could be used instead to fix problems affecting our users. You seem to assume that this manpower is somehow transferable to other tasks. I'm afraid that the result of telling people don't fix FOO, do fix BAR instead can be that neither FOO[0] nor BAR[1] is fixed. It is transferrable to some level: fails to build twice bugs are quite similar to fails to build bugs, and we have a large amount of such bugs (501 packages failing to build in my latest rebuild). Anyway, if people are willing to work on that, why not. I would prefer if those bugs were filed with severity:minor, and if that effort was not a release goal. I could do one test rebuild for that effort, too. For that to happen, I need a patched sbuild that does what you want to test. I don't care if the patch is dirty: the patched sbuild doesn't need to permit normal builds, so feel free to hack deep inside. Back in 2007, the required sbuild patch was about 10-lines long (patch can be found in collab-qa svn, in debcluster/configs/sbuild-twice.patch). Make sure the patch works by testing it against a reasonable amount of randomly chosen packages. Then I would provide the logs (but do not file bugs myself). Lucas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111223151405.ga24...@xanadu.blop.info
Re: many packages fail to build twice in a row again
Lars Wirzenius l...@liw.fi writes: On Fri, Dec 23, 2011 at 10:41:07AM +, Roger Leigh wrote: The suggestion that git clean be a solution appears to have caused some level of outrage. However, at least for '3.0 (git)', all the sources are known to git, and 'git clean' is a reliable and simple solution to the problem. The alternative, manually reverting all the changes, is both complex and error-prone. I'm not sure I see the problem with what is an obvious improvement to the process. I'd favor a solution that avoids having to fix hundres, or thousands, of upstream packages. For example, instead of building directly in the working tree, we could export the sources to a temporary directory and build there. Voila: no build artifacts to clean up at all. That sucks nearly as much as packages that unpack a upstream.tar.gz for every build. It makes editing the source, running make till it works and then building a new package a problem. What can usualy be done quite easily is out-of-tree builds. Pretty much the same effect with less inconvenience. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87wr9nyu6e.fsf@frosties.localnet
Re: many packages fail to build twice in a row again
Am Freitag, 23. Dezember 2011, 20:17:13 schrieb Goswin von Brederlow: Lars Wirzenius l...@liw.fi writes: On Fri, Dec 23, 2011 at 10:41:07AM +, Roger Leigh wrote: The suggestion that git clean be a solution appears to have caused some level of outrage. However, at least for '3.0 (git)', all the sources are known to git, and 'git clean' is a reliable and simple solution to the problem. The alternative, manually reverting all the changes, is both complex and error-prone. I'm not sure I see the problem with what is an obvious improvement to the process. I'd favor a solution that avoids having to fix hundres, or thousands, of upstream packages. For example, instead of building directly in the working tree, we could export the sources to a temporary directory and build there. Voila: no build artifacts to clean up at all. That sucks nearly as much as packages that unpack a upstream.tar.gz for every build. It makes editing the source, running make till it works and then building a new package a problem. What can usualy be done quite easily is out-of-tree builds. Pretty much the same effect with less inconvenience. And just as much work, actually. With e.g. autotools, there is a surprising possibility to not be done right by upstream. HS -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201112232303.17622.p...@hendrik-sattler.de
Re: many packages fail to build twice in a row again
Le Tue, Dec 20, 2011 at 10:01:44PM +0200, Peter Eisentraut a écrit : With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. Personally I have given up making packages buildable twice in a row using “debian clean” for the packages I maintain with git, as it is really easy to reset the repository with “git clean” and “git checkout”. Half of the source packages that declare a VCS are managed with Git. I think that instead of making complex workarounds, we would benefit of allowing the 3.0 (git) source format, where “git clean” and “git checkout” could be ran from the clean target of debian/rules, if there is agreement that it does not violate the principle of least surprise, as it also wipes out changes made in the debian directory. About acceptability of that format for our archive, I have not seen direct evidence that it is unsuitable, see http://bugs.debian.org/642801. Have a nice day, -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111221093053.GH18013@chouca.igloo
Re: many packages fail to build twice in a row again
On 20/12/11 at 23:16 +, brian m. carlson wrote: On Tue, Dec 20, 2011 at 09:36:47PM +0100, Lucas Nussbaum wrote: On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. Is it really worth it? There are many ways to work-around this, such as using a temporary git repo to be able to 'git clean' and return to a clean state before each build. If I'm fixing an RC bug, it's very convenient to be able to test a patch and then rebuild with a different patch if necessary. If the package doesn't build cleanly N times in a row, or if there are other problems with the packaging that make it difficult to fix, I usually go on to fix other bugs instead. Also, when I file a bug report, the harder you make it to fix your package, the less likely you are to receive a patch with that report, workaround or not. I'm not saying that it's not convenient. But on the other hand, several hundreds of packages probably need to be fixed, which will require a large amount of manpower that could be used instead to fix problems affecting our users. Maybe it's better to fix those bugs when they are encountered, rather than have a systematic effort to fix them all? If you are interested into mass bug filing about issues that affect our users, see http://lists.debian.org/debian-qa/2011/08/msg00091.html for a list of 1073 packages failing to install, remove, or upgrade from squeeze. Lucas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111221094135.ga14...@xanadu.blop.info
Re: many packages fail to build twice in a row again
Hi, Am Dienstag, den 20.12.2011, 21:36 +0100 schrieb Lucas Nussbaum: On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. Is it really worth it? There are many ways to work-around this, such as using a temporary git repo to be able to 'git clean' and return to a clean state before each build. I also find this a worthy goal. It is not always forseeable that a package will be affected, so if you forget to prepare (e.g. by creating a git repo), you can easily get into a situation where your changes are entangled with a bunch of non-intentional changes that are hard to fix manually and I’ve had to restart from scratch and manually picking my changes from the large diff a few times. I, for one, welcome new FTBTIAR bugs :-) Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
Re: many packages fail to build twice in a row again
Lucas Nussbaum lu...@lucas-nussbaum.net writes: On 20/12/11 at 23:16 +, brian m. carlson wrote: On Tue, Dec 20, 2011 at 09:36:47PM +0100, Lucas Nussbaum wrote: On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. Is it really worth it? There are many ways to work-around this, such as using a temporary git repo to be able to 'git clean' and return to a clean state before each build. If I'm fixing an RC bug, it's very convenient to be able to test a patch and then rebuild with a different patch if necessary. If the package doesn't build cleanly N times in a row, or if there are other problems with the packaging that make it difficult to fix, I usually go on to fix other bugs instead. Also, when I file a bug report, the harder you make it to fix your package, the less likely you are to receive a patch with that report, workaround or not. I'm not saying that it's not convenient. But on the other hand, several hundreds of packages probably need to be fixed, which will require a large amount of manpower that could be used instead to fix problems affecting our users. Maybe it's better to fix those bugs when they are encountered, rather than have a systematic effort to fix them all? I think it is beter to have them fixed before someone needs to write a patch for them. The way I prepare a patch for a debian package goes like this: 1. apt-get source (or git/svn) 2. debuild -us -uc -b (see if it actually builds before changes) 3. dch --nmu 4. edit files 5. debuild 6. dpkg -i test 7. go back to 4 till it works 8. debuild -us -uc -S 9. rename debain/patches/debian-changes-version or debdiff the dsc files 10. reportbug -A patch package If the package does not build N times then steps 4/5/6 are more work. But also step 9 often becomes much more work. The patch becomes a lot larger, usualy because auto* files are included, and then needs extra cleanup before being able to send it. If you are interested into mass bug filing about issues that affect our users, see http://lists.debian.org/debian-qa/2011/08/msg00091.html for a list of 1073 packages failing to install, remove, or upgrade from squeeze. How large is the intersection of those 1073 with the packages that don't build multiple times? Lets fix that subset first. Easier to fix 2 bugs with a single upload. :) Lucas MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87pqfiuiqq.fsf@frosties.localnet
Re: many packages fail to build twice in a row again
Charles Plessy ple...@debian.org writes: Le Tue, Dec 20, 2011 at 10:01:44PM +0200, Peter Eisentraut a écrit : With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. Personally I have given up making packages buildable twice in a row using âdebian cleanâ for the packages I maintain with git, as it is really easy to reset the repository with âgit cleanâ and âgit checkoutâ. Half of the source packages that declare a VCS are managed with Git. I think that instead of making complex workarounds, we would benefit of allowing the 3.0 (git) source format, where âgit cleanâ and âgit checkoutâ could be ran from the clean target of debian/rules, if there is agreement that it does not violate the principle of least surprise, as it also wipes out changes made in the debian directory. About acceptability of that format for our archive, I have not seen direct evidence that it is unsuitable, see http://bugs.debian.org/642801. Have a nice day, That totaly violates the principle of least surprise. debian/rules clean should never undo changes made by the user. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87liq6uinc.fsf@frosties.localnet
many packages fail to build twice in a row again
With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1324411304.8520.3.ca...@vanquo.pezone.net
Re: many packages fail to build twice in a row again
On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. Is it really worth it? There are many ways to work-around this, such as using a temporary git repo to be able to 'git clean' and return to a clean state before each build. Lucas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20111220203647.ga32...@xanadu.blop.info
Re: many packages fail to build twice in a row again
On Tue, Dec 20, 2011 at 09:36:47PM +0100, Lucas Nussbaum wrote: On 20/12/11 at 22:01 +0200, Peter Eisentraut wrote: With recent dpkg(-source) changes, many packages are again failing to build twice in a row, because of uncommitted upstream changes. Fixing this was a lenny release goal, maybe it should be one again?!? Most importantly, maybe someone who has access to one of those build grids can run the old tests again, because I feel by accidentally stumbling upon these problems we will not be able to find and fix many of them any time soon. Is it really worth it? There are many ways to work-around this, such as using a temporary git repo to be able to 'git clean' and return to a clean state before each build. If I'm fixing an RC bug, it's very convenient to be able to test a patch and then rebuild with a different patch if necessary. If the package doesn't build cleanly N times in a row, or if there are other problems with the packaging that make it difficult to fix, I usually go on to fix other bugs instead. Also, when I file a bug report, the harder you make it to fix your package, the less likely you are to receive a patch with that report, workaround or not. -- brian m. carlson / brian with sandals: Houston, Texas, US +1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187 signature.asc Description: Digital signature
