Re: The wider implications of debhelper/dbus breakage
On Wed, 22 Jul 2009, Cyril Brulebois wrote: Raphael Hertzog hert...@debian.org (17/07/2009): At least dpkg-dev has one and it's run at build-time. I thought the goal of dpkg-dev was to actually build other packages. I don't know how dpkg-dev developers see this, but maybe having a few packages rebuilt using the new dpkg-dev package would help spotting breakages? Well, I always run the git version on my machine. But maintaining dpkg-dev is enough of a job that I don't maintain many other packages... so I don't build many other packages except for QA work or similar (like investigation build failures with the new source format). I dunno, but especially eglibc and other important packages would be quite good candidates. Well, multiple hours compilation don't buy us much testing of dpkg-dev. That said the bigger packages are also those that make usage of the advanced features... I guess that a loop for getting the source packages, pushing them through sbuild once with the old dpkg-dev package and once with the new dpkg-dev packages, then debdiffing (and whatever interesting diff tests might come to mind) would help and wouldn't be very hard to implement. Are you interested in writing it? :-) I don't see anything that needs to be specific for each tool. Starting with a new sbuild option to auto-install a new .deb in the chroot (and restore the official one provided by apt) would be half the job. Deciding what difference is tolerated is the second (more annoying) half of the job. Cheers, -- Raphaël Hertzog -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Raphael Hertzog hert...@debian.org (17/07/2009): At least dpkg-dev has one and it's run at build-time. I thought the goal of dpkg-dev was to actually build other packages. I don't know how dpkg-dev developers see this, but maybe having a few packages rebuilt using the new dpkg-dev package would help spotting breakages? I dunno, but especially eglibc and other important packages would be quite good candidates. I guess that a loop for getting the source packages, pushing them through sbuild once with the old dpkg-dev package and once with the new dpkg-dev packages, then debdiffing (and whatever interesting diff tests might come to mind) would help and wouldn't be very hard to implement. That also holds for debhelper, cdbs, insert any critical package here. Of course the test was added when the bug got fixed (we're speaking of #536034 for those wondering). I know. Guess what, I checked the facts. ;p Speaking for dpkg-dev, any help is always welcome to enhance the test-suite but it's a huge job that is not so rewarding. See above for a trivial idea. Mraw, KiBi. signature.asc Description: Digital signature
Re: The wider implications of debhelper/dbus breakage
On 2009-07-21, Goswin von Brederlow goswin-...@web.de wrote: Philipp Kern tr...@philkern.de writes: On 2009-07-20, Stéphane Glondu st...@glondu.net wrote: For example, each OCaml transition involve rebuilding a lot of packages (about 139), with 6 levels of dependencies. So if some build takes 2 days or more (for the current transition, on some builds, it was even more than a week) to be uploaded, it means that transition will last at least 12 days (for the current transition, all packages were rebuilt/uploaded/installed after 21 days). But most of the builds are successful (and fast). If packages were automatically uploaded on success, a dependency level would be cleared at each dinstall run, meaning the whole recompilation would take less than 2 days. Haskell is even more intense. But it's not exactly true because we are autobuilding from accepted, so you do not need to wait for dinstall runs to complete but can get it done much quicker. Kind regards, Philipp Kern The long wait is the signing, not the dinstall run. Even without accepted dinstall runs 4 times a day now. But I have to say I'm totaly against unsigned uploads. The buildds are insecure enough as it is. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Tue, Jul 21, 2009 at 07:38:54AM +, Philipp Kern wrote: On 2009-07-21, Goswin von Brederlow goswin-...@web.de wrote: The long wait is the signing, not the dinstall run. Even without accepted dinstall runs 4 times a day now. But I have to say I'm totaly against unsigned uploads. The buildds are insecure enough as it is. MfG Goswin [...nothing...] You were saying? ;-) -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Wouter Verhelst a écrit : Ah, so this is about not interfering with testing migration, I guess? It's not only testing migration. As an example: If you have a large chain of binNMUs which all need some dep-wait on a package upper in the chain you get the effect that the whole thing takes several days just because each step of the chain first blocks on signing and uploading once a day to do the next one. How often does that happen? (serious question, I have no clue) I will speak with my OCaml maitainer's hat, but what I say is not really specific to OCaml. For example, each OCaml transition involve rebuilding a lot of packages (about 139), with 6 levels of dependencies. So if some build takes 2 days or more (for the current transition, on some builds, it was even more than a week) to be uploaded, it means that transition will last at least 12 days (for the current transition, all packages were rebuilt/uploaded/installed after 21 days). But most of the builds are successful (and fast). If packages were automatically uploaded on success, a dependency level would be cleared at each dinstall run, meaning the whole recompilation would take less than 2 days. Now, imagine that during this transition, another long transition (for another library) starts... The new transition can be entangled with the first one, delaying it even further. Cheers, -- Stéphane -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On 2009-07-20, Stéphane Glondu st...@glondu.net wrote: For example, each OCaml transition involve rebuilding a lot of packages (about 139), with 6 levels of dependencies. So if some build takes 2 days or more (for the current transition, on some builds, it was even more than a week) to be uploaded, it means that transition will last at least 12 days (for the current transition, all packages were rebuilt/uploaded/installed after 21 days). But most of the builds are successful (and fast). If packages were automatically uploaded on success, a dependency level would be cleared at each dinstall run, meaning the whole recompilation would take less than 2 days. Haskell is even more intense. But it's not exactly true because we are autobuilding from accepted, so you do not need to wait for dinstall runs to complete but can get it done much quicker. Kind regards, Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Philipp Kern tr...@philkern.de writes: On 2009-07-20, Stéphane Glondu st...@glondu.net wrote: For example, each OCaml transition involve rebuilding a lot of packages (about 139), with 6 levels of dependencies. So if some build takes 2 days or more (for the current transition, on some builds, it was even more than a week) to be uploaded, it means that transition will last at least 12 days (for the current transition, all packages were rebuilt/uploaded/installed after 21 days). But most of the builds are successful (and fast). If packages were automatically uploaded on success, a dependency level would be cleared at each dinstall run, meaning the whole recompilation would take less than 2 days. Haskell is even more intense. But it's not exactly true because we are autobuilding from accepted, so you do not need to wait for dinstall runs to complete but can get it done much quicker. Kind regards, Philipp Kern The long wait is the signing, not the dinstall run. Even without accepted dinstall runs 4 times a day now. But I have to say I'm totaly against unsigned uploads. The buildds are insecure enough as it is. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Sat, Jul 18, 2009 at 03:45:13PM +, Philipp Kern wrote: On 2009-07-18, Wouter Verhelst wou...@debian.org wrote: Ah, so this is about not interfering with testing migration, I guess? It's not only testing migration. As an example: If you have a large chain of binNMUs which all need some dep-wait on a package upper in the chain you get the effect that the whole thing takes several days just because each step of the chain first blocks on signing and uploading once a day to do the next one. How often does that happen? (serious question, I have no clue) Even if the once a day doesn't hold it still affects slow architectures. I see it like Luk that we want porters to care about logs of build failures. Oh, I totally agree there; but then I should also add that I think buildd maintainers should preferably be porters themselves. I don't see why anyone should get bothered by the huge bunch of successful logs. I think I explained sufficiently in my previous mails that I personally do not think it is something bothersome, and that I would prefer not losing them. Of course it can be scripted, but then, why are we even putting the human in between of this. If it's just about some simple regexp to highlight them we can also weed out known patterns on the buildd and pass them on for manual review. That requires you to keep a bunch of regexps in sync across a bunch of buildd machines. It's easier to have those highlights in one place in a mail client configuration, especially in case you want to add one when you're on the road and figure out you need one extra. Additionally, it is _not_ only about those regexps; as I explained, I find that the daily batch of success mails helps me stay aware of the fact that I'm handling a buildd machine. It also helps me in being aware of how well it's performing -- if there are 20 failure mails and 100 success mails, it's doing well. If there are 20 failure mails and just 1 success mail, I may want to investigate what the hell is going on. A daily 'summary' mail doesn't do that for me, mainly because people have a tendency to not look at that (how many of you actually _read_ cron output?) -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On 2009-07-18, Wouter Verhelst wou...@debian.org wrote: Ah, so this is about not interfering with testing migration, I guess? It's not only testing migration. As an example: If you have a large chain of binNMUs which all need some dep-wait on a package upper in the chain you get the effect that the whole thing takes several days just because each step of the chain first blocks on signing and uploading once a day to do the next one. Even if the once a day doesn't hold it still affects slow architectures. I see it like Luk that we want porters to care about logs of build failures. I don't see why anyone should get bothered by the huge bunch of successful logs. Of course it can be scripted, but then, why are we even putting the human in between of this. If it's just about some simple regexp to highlight them we can also weed out known patterns on the buildd and pass them on for manual review. Kind regards, Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Thu, Jul 16, 2009 at 10:57:25PM -0500, Peter Samuelson wrote: [Michael Biebl] Would it make sense to avoid the upload of obviously broken packages from buildds in the future. E.g. if lintian detects an error it would need some special inspection from the buildd uploader. Don't all buildd binary packages already need special inspection from a buildd uploader? I get somewhere between 30 and 100 mails success mails from my two buildds (voltaire and malo) on an average day. I do have a few mutt rules that highlight mails with obvious issues (so I can more closely inspect them before signing), but I seriously do *not* read all of them from start to end. I wouldn't be able to get any work done in that case. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html signature.asc Description: Digital signature
Re: The wider implications of debhelper/dbus breakage
Wouter Verhelst wrote: On Thu, Jul 16, 2009 at 10:57:25PM -0500, Peter Samuelson wrote: [Michael Biebl] Would it make sense to avoid the upload of obviously broken packages from buildds in the future. E.g. if lintian detects an error it would need some special inspection from the buildd uploader. Don't all buildd binary packages already need special inspection from a buildd uploader? I get somewhere between 30 and 100 mails success mails from my two buildds (voltaire and malo) on an average day. I do have a few mutt rules that highlight mails with obvious issues (so I can more closely inspect them before signing), but I seriously do *not* read all of them from start to end. I wouldn't be able to get any work done in that case. Wouter's comment aside, checks at buildd level would be too late. It should be the new queue that may perform a few checks, such that obviously broken packages are not even forwarded to the builders. The idea with lintian I seems just fine to me - it just needs to happen earlier. Steffen -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Fri, Jul 17, 2009 at 10:57:25AM +0200, Steffen Moeller steffen_moel...@gmx.de wrote: Wouter Verhelst wrote: On Thu, Jul 16, 2009 at 10:57:25PM -0500, Peter Samuelson wrote: [Michael Biebl] Would it make sense to avoid the upload of obviously broken packages from buildds in the future. E.g. if lintian detects an error it would need some special inspection from the buildd uploader. Don't all buildd binary packages already need special inspection from a buildd uploader? I get somewhere between 30 and 100 mails success mails from my two buildds (voltaire and malo) on an average day. I do have a few mutt rules that highlight mails with obvious issues (so I can more closely inspect them before signing), but I seriously do *not* read all of them from start to end. I wouldn't be able to get any work done in that case. Wouter's comment aside, checks at buildd level would be too late. It should be the new queue that may perform a few checks, such that obviously broken packages are not even forwarded to the builders. Except you wouldn't have detected the debhelper/dbus breakage at the new queue level. Mike -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Mike Hommey wrote: On Fri, Jul 17, 2009 at 10:57:25AM +0200, Steffen Moeller steffen_moel...@gmx.de wrote: Wouter Verhelst wrote: On Thu, Jul 16, 2009 at 10:57:25PM -0500, Peter Samuelson wrote: [Michael Biebl] Would it make sense to avoid the upload of obviously broken packages from buildds in the future. E.g. if lintian detects an error it would need some special inspection from the buildd uploader. Don't all buildd binary packages already need special inspection from a buildd uploader? I get somewhere between 30 and 100 mails success mails from my two buildds (voltaire and malo) on an average day. I do have a few mutt rules that highlight mails with obvious issues (so I can more closely inspect them before signing), but I seriously do *not* read all of them from start to end. I wouldn't be able to get any work done in that case. Wouter's comment aside, checks at buildd level would be too late. It should be the new queue that may perform a few checks, such that obviously broken packages are not even forwarded to the builders. Except you wouldn't have detected the debhelper/dbus breakage at the new queue level. Not? Was the originally uploaded package correct? Amazing. Hm. Then, it should be lintian errors that denote a build as a failure, indeed, and these should somehow be detected by the mechanism that uploads the packages ... not by the buildd admin. Cheers, Steffen -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Fri, Jul 17, 2009 at 11:19:50AM +0200, Steffen Moeller wrote: Not? Was the originally uploaded package correct? Amazing. Hm. Then, it should be lintian errors that denote a build as a failure, indeed, and these should somehow be detected by the mechanism that uploads the packages ... not by the buildd admin. It's impossible to catch every issue in an automated way. There are things that can be caught (such as, /maybe/, this), but you have to deal with the fact that some things will still slip through the cracks. I'm also not at all sure whether sbuild runs lintian during the build. Perhaps that would be good, though. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Wouter Verhelst wrote: On Fri, Jul 17, 2009 at 11:19:50AM +0200, Steffen Moeller wrote: Not? Was the originally uploaded package correct? Amazing. Hm. Then, it should be lintian errors that denote a build as a failure, indeed, and these should somehow be detected by the mechanism that uploads the packages ... not by the buildd admin. It's impossible to catch every issue in an automated way. There are things that can be caught (such as, /maybe/, this), but you have to deal with the fact that some things will still slip through the cracks. I'm also not at all sure whether sbuild runs lintian during the build. Perhaps that would be good, though. AFAIK the FTP Team is working on a system to prevent uploads which have lintian errors. The whole category of lintian errors has already been assessed and possible overrides are planned to arrive in the NEW queue at least once... Please do note that I'm talking about errors, not about warnings. Cheers Luk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Fri, Jul 17, 2009 at 11:44:54AM +0200, Luk Claes wrote: Wouter Verhelst wrote: On Fri, Jul 17, 2009 at 11:19:50AM +0200, Steffen Moeller wrote: Not? Was the originally uploaded package correct? Amazing. Hm. Then, it should be lintian errors that denote a build as a failure, indeed, and these should somehow be detected by the mechanism that uploads the packages ... not by the buildd admin. It's impossible to catch every issue in an automated way. There are things that can be caught (such as, /maybe/, this), but you have to deal with the fact that some things will still slip through the cracks. I'm also not at all sure whether sbuild runs lintian during the build. Perhaps that would be good, though. AFAIK the FTP Team is working on a system to prevent uploads which have lintian errors. The whole category of lintian errors has already been assessed and possible overrides are planned to arrive in the NEW queue at least once... Please do note that I'm talking about errors, not about warnings. Right. However, having sbuild run lintian would allow a buildd maintainer to assess issues with packages by looking at *warnings*, rather than 'just' errors. This isn't something an automated system can do. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On 2009-07-17, Wouter Verhelst wou...@debian.org wrote: AFAIK the FTP Team is working on a system to prevent uploads which have lintian errors. The whole category of lintian errors has already been assessed and possible overrides are planned to arrive in the NEW queue at least once... Please do note that I'm talking about errors, not about warnings. Right. However, having sbuild run lintian would allow a buildd maintainer to assess issues with packages by looking at *warnings*, rather than 'just' errors. This isn't something an automated system can do. While that's true (and would require having an up-to-date lintian on the buildd which does not emit false warnings for packages built in the testing/stable/oldstable queues), some people are working to get autosigning happen. This might drop review except for special cases. Of course we could put lintian into that set, though. And that might actually be a good compromise. Kind regards, Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Wouter Verhelst wrote: On Fri, Jul 17, 2009 at 11:44:54AM +0200, Luk Claes wrote: Wouter Verhelst wrote: On Fri, Jul 17, 2009 at 11:19:50AM +0200, Steffen Moeller wrote: Not? Was the originally uploaded package correct? Amazing. Hm. Then, it should be lintian errors that denote a build as a failure, indeed, and these should somehow be detected by the mechanism that uploads the packages ... not by the buildd admin. It's impossible to catch every issue in an automated way. There are things that can be caught (such as, /maybe/, this), but you have to deal with the fact that some things will still slip through the cracks. I'm also not at all sure whether sbuild runs lintian during the build. Perhaps that would be good, though. AFAIK the FTP Team is working on a system to prevent uploads which have lintian errors. The whole category of lintian errors has already been assessed and possible overrides are planned to arrive in the NEW queue at least once... Please do note that I'm talking about errors, not about warnings. Right. However, having sbuild run lintian would allow a buildd maintainer to assess issues with packages by looking at *warnings*, rather than 'just' errors. This isn't something an automated system can do. Right, though that's why we expect maintainers to look at them. Although there may be architecture specific lintian warnings, they should be really rare. Besides, we want to get some support for autosigning packages built on the buildds. So we improve the speed of buildd uploads and we make the job of buildd maintainer more attractive to porters so they could really investigate (architecture specific) build errors instead of spending time in downloading, checking and signing successful build logs. Cheers Luk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Steffen Moeller steffen_moel...@gmx.de (17/07/2009): Wouter's comment aside, checks at buildd level would be too late. Yes, sure. It'd rather be time for critical packages (say: dpkg-dev, debhelper, cdbs) to have proper non-regression testsuites. Mraw, KiBi. signature.asc Description: Digital signature
Re: The wider implications of debhelper/dbus breakage
On Fri, 17 Jul 2009, Cyril Brulebois wrote: Steffen Moeller steffen_moel...@gmx.de (17/07/2009): Wouter's comment aside, checks at buildd level would be too late. Yes, sure. It'd rather be time for critical packages (say: dpkg-dev, debhelper, cdbs) to have proper non-regression testsuites. At least dpkg-dev has one and it's run at build-time. Symbol file parsing/matching was covered by a test suite but the precise problem that happened was not catched. Of course the test was added when the bug got fixed (we're speaking of #536034 for those wondering). I also saw that Joey Hess added a test-suite for the recent debhelper problems with dh_install: debhelper (7.2.24) unstable; urgency=low * dh_install: Add test suite covering the last 5 bugs. Speaking for dpkg-dev, any help is always welcome to enhance the test-suite but it's a huge job that is not so rewarding. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Fri, Jul 17, 2009 at 12:10:56PM +0200, Luk Claes wrote: Wouter Verhelst wrote: Right. However, having sbuild run lintian would allow a buildd maintainer to assess issues with packages by looking at *warnings*, rather than 'just' errors. This isn't something an automated system can do. Right, though that's why we expect maintainers to look at them. Although there may be architecture specific lintian warnings, they should be really rare. They would still catch this kind of bug, though. Also, there *are* many system-specific warnings emitted by gcc, and those can easily be picked up by mutt highlighting. Besides, we want to get some support for autosigning packages built on the buildds. So we improve the speed of buildd uploads and we make the job of buildd maintainer more attractive to porters so they could really investigate (architecture specific) build errors instead of spending time in downloading, checking and signing successful build logs. Hmm. I'm not sure that's very useful, really. Due to scripts and mutt's GPG key passphrase caching, my daily buildd mail signing stuff never takes more than a minute[1], even on days with hundreds of logs that need to be signed (except if highlighting tells me that there's something that needs to be looked at, obviously). Perhaps such scripts could be shared, but other than that... Additionally, I personally dislike a buildd host that is silent in the usual case. The fact that there is routinely something to do forces me to continually think about it and not neglect the things I need to do to maintain it[2]. You'll note that there've been times when the powerpc dailies were broken for long amounts of time in a row, when I used to maintain it; this is mainly because the system's output would not be very different between 'nothing is working' and 'everything is working fine', so I just wouldn't notice when things were broken. In other words, I personally do not feel that, from a buildd maintainer's point of view, the disadvantages of having to sign mails (which is no work at all, really) outweigh the advantages (me being much, /much/ more aware of what's happening, and being able to take care of it that much better). I understand that the delay in uploading that's inherent in manual action isn't ideal from an RM's point of view, but then that shouldn't be more than 24 hours in the usual case anyway (and if it is, that's a sign that the buildd maintainer is getting bored with the job, or needs help, or some such, which shouldn't be the usual case anyway). [1] and a minute really is exceptional; the average is more like 10 seconds or so. [2] obviously there'll always be times when I'm too busy to look at that stuff for a few days in a row, but those are the exception rather than the rule. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Luk Claes l...@debian.org writes: AFAIK the FTP Team is working on a system to prevent uploads which have lintian errors. The whole category of lintian errors has already been assessed and possible overrides are planned to arrive in the NEW queue at least once... Please do note that I'm talking about errors, not about warnings. Lintian now supports being given a list of tags and then only reporting specifically those tags, ignoring anything else. I believe the intention for automatic rejects is to add specific tags that are both very serious and that we're extremely confident in, rather than using the coarse error/warning distinction. It seems like something similar could work here. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
Wouter Verhelst wrote: On Fri, Jul 17, 2009 at 12:10:56PM +0200, Luk Claes wrote: Wouter Verhelst wrote: Right. However, having sbuild run lintian would allow a buildd maintainer to assess issues with packages by looking at *warnings*, rather than 'just' errors. This isn't something an automated system can do. Right, though that's why we expect maintainers to look at them. Although there may be architecture specific lintian warnings, they should be really rare. They would still catch this kind of bug, though. Also, there *are* many system-specific warnings emitted by gcc, and those can easily be picked up by mutt highlighting. Besides, we want to get some support for autosigning packages built on the buildds. So we improve the speed of buildd uploads and we make the job of buildd maintainer more attractive to porters so they could really investigate (architecture specific) build errors instead of spending time in downloading, checking and signing successful build logs. Hmm. I'm not sure that's very useful, really. Due to scripts and mutt's GPG key passphrase caching, my daily buildd mail signing stuff never takes more than a minute[1], even on days with hundreds of logs that need to be signed (except if highlighting tells me that there's something that needs to be looked at, obviously). Perhaps such scripts could be shared, but other than that... Unless you have a light speed internet connection you're cheating here... Additionally, I personally dislike a buildd host that is silent in the usual case. The fact that there is routinely something to do forces me to continually think about it and not neglect the things I need to do to maintain it[2]. You'll note that there've been times when the powerpc dailies were broken for long amounts of time in a row, when I used to maintain it; this is mainly because the system's output would not be very different between 'nothing is working' and 'everything is working fine', so I just wouldn't notice when things were broken. There are still the admin messages which give a summary of what happened and the logs for the non successful builds. In other words, I personally do not feel that, from a buildd maintainer's point of view, the disadvantages of having to sign mails (which is no work at all, really) outweigh the advantages (me being much, /much/ more aware of what's happening, and being able to take care of it that much better). I understand that the delay in uploading that's inherent in manual action isn't ideal from an RM's point of view, but then that shouldn't be more than 24 hours in the usual case anyway (and if it is, that's a sign that the buildd maintainer is getting bored with the job, or needs help, or some such, which shouldn't be the usual case anyway). In the usual case at least one of the buildd maintainers takes more than 24 hours to process the log. [1] and a minute really is exceptional; the average is more like 10 seconds or so. You're only counting the time autosigning would need, not the time needed to download the log, process it and wait till the cronjob acts on the to be uploaded package. The latter could be done without autosigning, but as it's measured in hours (maximum), it doesn't make sense to optimise it currently... You seem to only look at the most optimal situation which is very rare in practice. [2] obviously there'll always be times when I'm too busy to look at that stuff for a few days in a row, but those are the exception rather than the rule. Though unfortunately they don't collide with times when other people are too busy and for a few days it's also not very comfortable to switch to someone else for processing the logs... Cheers Luk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: The wider implications of debhelper/dbus breakage
On Fri, Jul 17, 2009 at 09:05:29PM +0200, Luk Claes wrote: Wouter Verhelst wrote: On Fri, Jul 17, 2009 at 12:10:56PM +0200, Luk Claes wrote: Wouter Verhelst wrote: Right. However, having sbuild run lintian would allow a buildd maintainer to assess issues with packages by looking at *warnings*, rather than 'just' errors. This isn't something an automated system can do. Right, though that's why we expect maintainers to look at them. Although there may be architecture specific lintian warnings, they should be really rare. They would still catch this kind of bug, though. Also, there *are* many system-specific warnings emitted by gcc, and those can easily be picked up by mutt highlighting. Besides, we want to get some support for autosigning packages built on the buildds. So we improve the speed of buildd uploads and we make the job of buildd maintainer more attractive to porters so they could really investigate (architecture specific) build errors instead of spending time in downloading, checking and signing successful build logs. Hmm. I'm not sure that's very useful, really. Due to scripts and mutt's GPG key passphrase caching, my daily buildd mail signing stuff never takes more than a minute[1], even on days with hundreds of logs that need to be signed (except if highlighting tells me that there's something that needs to be looked at, obviously). Perhaps such scripts could be shared, but other than that... Unless you have a light speed internet connection you're cheating here... I don't know about you, but I don't download my mails interactively. Offlineimap is great. Additionally, I personally dislike a buildd host that is silent in the usual case. The fact that there is routinely something to do forces me to continually think about it and not neglect the things I need to do to maintain it[2]. You'll note that there've been times when the powerpc dailies were broken for long amounts of time in a row, when I used to maintain it; this is mainly because the system's output would not be very different between 'nothing is working' and 'everything is working fine', so I just wouldn't notice when things were broken. There are still the admin messages which give a summary of what happened and the logs for the non successful builds. That is absolutely not the same thing, as my experience with d-i dailies has shown. In other words, I personally do not feel that, from a buildd maintainer's point of view, the disadvantages of having to sign mails (which is no work at all, really) outweigh the advantages (me being much, /much/ more aware of what's happening, and being able to take care of it that much better). I understand that the delay in uploading that's inherent in manual action isn't ideal from an RM's point of view, but then that shouldn't be more than 24 hours in the usual case anyway (and if it is, that's a sign that the buildd maintainer is getting bored with the job, or needs help, or some such, which shouldn't be the usual case anyway). In the usual case at least one of the buildd maintainers takes more than 24 hours to process the log. Ah, so this is about not interfering with testing migration, I guess? If it does indeed happen often that testing migration is delayed because of long-delayed uploads, then I agree that that's a bad thing; but I'm not convinced that autosigning is the answer. I do consider it bad form for a buildd maintainer to often take more than one day to sign their uploads. That doesn't take a long time, and shouldn't be delayed. When it does happen for me that I don't sign my uploads for a long time, it's usually because I was somewhere for work where I couldn't reach my mail, and got home so late that I didn't have the courage to pick up my laptop anymore, or so; clearly, a rare situation. Thus, this should not in principle significantly delay testing migration, unless I'm missing something. [1] and a minute really is exceptional; the average is more like 10 seconds or so. You're only counting the time autosigning would need, No, I'm counting the time it takes for me to sign a package. not the time needed to download the log, Which is done in the background and does not take any of my time. process it Processing the mail is done by scripts, and takes less than a second per mail. In fact, it goes so fast that I can do several logs per second. This process, for all practical purposes, is semi-automated. and wait till the cronjob acts on the to be uploaded package. Which is done multiple times per hour. The latter could be done without autosigning, but as it's measured in hours (maximum), it doesn't make sense to optimise it currently... You seem to only look at the most optimal situation which is very rare in practice. I'm not quite sure what you mean by those two sentences. [2] obviously there'll always be times when I'm too busy to look at that stuff for a few days in a row, but
Re: The wider implications of debhelper/dbus breakage
Roger Leigh schrieb: Some people may have recently been bitten by #537125. This mail isn't about that bug in particular, though it did certainly expose the fragility of systems depending upon dbus. Blaming this on D-Bus is not fair. It could have happened to any other package. Heaven forbid this would e.g. have happened to libc6 (or for that matter ny other important library), and your system would be in an equally or even worse state. Simple truth is, it was extremely bad timing between me preparing, testing and uploading the package and the broken debhelper reaching the archive shortly afterwards. My thoughts are going into a different direction: Would it make sense to avoid the upload of obviously broken packages from buildds in the future. E.g. if lintian detects an error it would need some special inspection from the buildd uploader. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Re: The wider implications of debhelper/dbus breakage
[Michael Biebl] Would it make sense to avoid the upload of obviously broken packages from buildds in the future. E.g. if lintian detects an error it would need some special inspection from the buildd uploader. Don't all buildd binary packages already need special inspection from a buildd uploader? That was my understanding, anyway: a buildd package is not uploaded until someone reads the build log and takes special action, gpg-signing something or other. (And, yes, the build logs seem to include some 'dpkg --contents' output at the end, so the breakage should have been visible before upload.) -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
