Re: Why not to let all DDs to execute gb-command
On Tue, Jun 11, 2013 at 09:15:48AM +0800, Chow Loong Jin wrote: On Mon, Jun 10, 2013 at 10:43:57PM +0200, Philipp Kern wrote: Hi, On Mon, Jun 10, 2013 at 07:56:24PM +0200, Anton Gladky wrote: So, I think the developer should have a set of tools (including gb and even slight removal commands), which allow him to do the most of packaging work without worrying other teams/developers. And, of course, those tools should be relatively secure not to break others work and the whole archive. gb is a harmless in this case. it is not. If you rely on random successes of your build this is worse than not providing a build at all. If there's a security issue, people will be forced to spend time on the issue. Either the Security Team or by extension the Stable Release Team, to get it built to finally include it into a point release or leave it lingering forever in p-u-new because a test case fails. It's not always the case of relying on random successes of your build. There are valid cases -- for example, if a build-dep, or a dep of a build-dep had a bug that prevented installation, and has just been fixed. -- Kind regards, Loong Jin I wonder if in that case it wouldn't make more sense to allow setting an additional Build-Depends / Build-Conflicts instead of give-back once the problematic deb was fixed. So instead of: - wait for problem to be fixed or foobar to be build on that arch - gb package you would do: - extra-build-hint --conflicts foobar 1.2-3 [ - wait for foobar to get fixed or newer foobar to be build on that arch - watch the buildd retry your package automatically - throw a party ] MfG Goswin PS: replace extra-build-hint with whatever wb command does this -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130718130303.GF23957@frosties
Re: Why not to let all DDs to execute gb-command
Chow, am Tue, Jun 11, 2013 at 09:15:48AM +0800 hast du folgendes geschrieben: On Mon, Jun 10, 2013 at 07:56:24PM +0200, Anton Gladky wrote: So, I think the developer should have a set of tools (including gb and even slight removal commands), which allow him to do the most of packaging work without worrying other teams/developers. And, of course, those tools should be relatively secure not to break others work and the whole archive. gb is a harmless in this case. it is not. If you rely on random successes of your build this is worse than not providing a build at all. If there's a security issue, people will be forced to spend time on the issue. Either the Security Team or by extension the Stable Release Team, to get it built to finally include it into a point release or leave it lingering forever in p-u-new because a test case fails. It's not always the case of relying on random successes of your build. There are valid cases -- for example, if a build-dep, or a dep of a build-dep had a bug that prevented installation, and has just been fixed. I said random, not deterministic. Giving back until a certain test succeeds, for instance. Because some bad code triggers a segfault on almost every try except that it sometimes works. Kind regards Philipp Kern signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
On Tue, Jun 11, 2013 at 10:29:33AM +0200, Philipp Kern wrote: Chow, am Tue, Jun 11, 2013 at 09:15:48AM +0800 hast du folgendes geschrieben: On Mon, Jun 10, 2013 at 07:56:24PM +0200, Anton Gladky wrote: So, I think the developer should have a set of tools (including gb and even slight removal commands), which allow him to do the most of packaging work without worrying other teams/developers. And, of course, those tools should be relatively secure not to break others work and the whole archive. gb is a harmless in this case. it is not. If you rely on random successes of your build this is worse than not providing a build at all. If there's a security issue, people will be forced to spend time on the issue. Either the Security Team or by extension the Stable Release Team, to get it built to finally include it into a point release or leave it lingering forever in p-u-new because a test case fails. It's not always the case of relying on random successes of your build. There are valid cases -- for example, if a build-dep, or a dep of a build-dep had a bug that prevented installation, and has just been fixed. I said random, not deterministic. Giving back until a certain test succeeds, for instance. Because some bad code triggers a segfault on almost every try except that it sometimes works. That's a rather extreme case. I'd expect any approved DD to know better than to do something stupid like that. -- Kind regards, Loong Jin signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
Chow Loong Jin hyper...@debian.org (11/06/2013): On Tue, Jun 11, 2013 at 10:29:33AM +0200, Philipp Kern wrote: I said random, not deterministic. Giving back until a certain test succeeds, for instance. Because some bad code triggers a segfault on almost every try except that it sometimes works. That's a rather extreme case. I'd expect any approved DD to know better than to do something stupid like that. Expecting is nice, but performing a reality check[1] from time to time is even better. 1. https://lists.debian.org/debian-wb-team/ Mraw, KiBi. signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
On Tue, Jun 11, 2013 at 11:10:53AM +0200, Cyril Brulebois wrote: Chow Loong Jin hyper...@debian.org (11/06/2013): On Tue, Jun 11, 2013 at 10:29:33AM +0200, Philipp Kern wrote: I said random, not deterministic. Giving back until a certain test succeeds, for instance. Because some bad code triggers a segfault on almost every try except that it sometimes works. That's a rather extreme case. I'd expect any approved DD to know better than to do something stupid like that. Expecting is nice, but performing a reality check[1] from time to time is even better. 1. https://lists.debian.org/debian-wb-team/ Is there a specific thread I'm supposed to see here? -- Kind regards, Loong Jin signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
+++ Matthias Klumpp [2013-06-07 23:50 +0200]: 2013/6/7 Tollef Fog Heen tfh...@err.no: ]] Nicolas Dandrimont The accepted by DSA part is a bit complicated. We didn't really want to bother one of the busiest teams in Debian when the software wasn't even packaged, and, when it came up, I felt that the reception of the idea of using fedmsg was a bit lukewarm. I think we should be able to work things out when and if we have a working proof of concept. From what I remember, we did have some questions about how some unique challenges in Debian's infrastructure will be solved, yes. I don't think I've seen completely satisfactory answers about it yet either, but I'd need to go digging to be entirely sure. At the Tanglu Debian derivative (which is just in formation right now) we build packages using Jenkins CI at the moment, due to this issue and wanna-build having problems to build source-only packages and a few other issues (difficult to set-up, weird handling of buildlogs and some other related issues) Jenkins isn't perfect either, so I am currently aiming to adjust buildbot for our needs, which might be a solution (and it easier to handle than a wb infrastructure, and can be integrated with the existing Python tools). People keep trying to solve this problem (and this is good - it needs solving). Please take a look at pybit, which purports to be a distributed build-system designed to work for exactly this case, using MQ for the message-passing. http://packages.qa.debian.org/p/pybit.html I believe this to be a better fit than general-purpose build systems like jenkins and buildbot (which IME don't really fit the building of hundreds of different packages (as opposed to building a few packages often) very well). I haven't yet had a chance to use pybit for real (although I've been planning to try for a while, and might get a tuit soonish), but the people who built it are clueful, and it _ought_ to be better than rebuildd, simpler and more flexible than wanna-build, and more appropriate than jenkins and buildbot. I'd be very interested to hear from people working in this area whether it is indeed useful. Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2013061356.gg14...@stoneboat.aleph1.co.uk
Re: Why not to let all DDs to execute gb-command
Thanks all for opinions! From my point of view, this tool will not be used (hopefully) too often, but sometimes it is really necessary to try it out before uploading a new source version. On 06/09/2013 08:44 PM, Kurt Roeckx wrote: I have very mixed feelings about this. I do in general think that would be good thing. But I see many requests where I think it just doesn't make sense to me to retry it. If things fail randomly, I really want to know _why_ it fails, and want to have that fixed. I don't want to retry until it randomly works. But I do understand that it sometimes might be hard to find why it sometimes fails. Kurt, I do really appreciate your work, and know, that it takes too much time to handle such requests. But in most of cases the failing archs are mips, mipsel, sparc and never the most popular and used ones. I understand, that it would be very good to have a nice package, which always builds fine everywhere, but maintaining over 20 of them starts one to set the priorities, what to do first: whether to fix one more real bug or struggle with the random build-failures on the archs, where the package probably will never be even started. So, I think the developer should have a set of tools (including gb and even slight removal commands), which allow him to do the most of packaging work without worrying other teams/developers. And, of course, those tools should be relatively secure not to break others work and the whole archive. gb is a harmless in this case. Thank you, Anton signature.asc Description: OpenPGP digital signature
Re: Why not to let all DDs to execute gb-command
Hi, On Mon, Jun 10, 2013 at 07:56:24PM +0200, Anton Gladky wrote: So, I think the developer should have a set of tools (including gb and even slight removal commands), which allow him to do the most of packaging work without worrying other teams/developers. And, of course, those tools should be relatively secure not to break others work and the whole archive. gb is a harmless in this case. it is not. If you rely on random successes of your build this is worse than not providing a build at all. If there's a security issue, people will be forced to spend time on the issue. Either the Security Team or by extension the Stable Release Team, to get it built to finally include it into a point release or leave it lingering forever in p-u-new because a test case fails. Kind regards Philipp Kern signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
On Mon, Jun 10, 2013 at 10:43:57PM +0200, Philipp Kern wrote: Hi, On Mon, Jun 10, 2013 at 07:56:24PM +0200, Anton Gladky wrote: So, I think the developer should have a set of tools (including gb and even slight removal commands), which allow him to do the most of packaging work without worrying other teams/developers. And, of course, those tools should be relatively secure not to break others work and the whole archive. gb is a harmless in this case. it is not. If you rely on random successes of your build this is worse than not providing a build at all. If there's a security issue, people will be forced to spend time on the issue. Either the Security Team or by extension the Stable Release Team, to get it built to finally include it into a point release or leave it lingering forever in p-u-new because a test case fails. It's not always the case of relying on random successes of your build. There are valid cases -- for example, if a build-dep, or a dep of a build-dep had a bug that prevented installation, and has just been fixed. -- Kind regards, Loong Jin signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
On Wed, Jun 05, 2013 at 09:10:39PM +0200, Anton Gladky wrote: Dear all, I have a proposal to give a permission to all DDs to restart builds on failing archs e.g. execute gb-command. I think, most of developers are clever enough to define, whether the built failed accidentally and needs to be restarted, or it requires some fixing and uploading new version. It will save time for both DDs and wb-team. I have very mixed feelings about this. I do in general think that would be good thing. But I see many requests where I think it just doesn't make sense to me to retry it. If things fail randomly, I really want to know _why_ it fails, and want to have that fixed. I don't want to retry until it randomly works. But I do understand that it sometimes might be hard to find why it sometimes fails. Kurt -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130609184458.ga1...@roeckx.be
Re: Why not to let all DDs to execute gb-command
On 13234 March 1977, Philipp Kern wrote: Which begs the question if build management should be merged into dak to provide insight about which package is new. Obviously this could also be a defined interface but if we're going to start to reuse dak's interfaces for that... maybe not. Only in a way that leaves FTP* out of the general area of wb-admin. :) That is, yes, we can sure do $magic in dak whenever we process a package, and in a defined way inform/update/whatever something for you, so that you easily (and fast) know what is new in which archive/suite/... I could imagine an own table or so we update, that is then provided to you. (Hello replication :D or psql service access). Or even us running a ssh trigger command with some arguments that you then parse. Whatever. Could you come back with this topic in July? Probably on -dak list. Thanks. -- bye, Joerg -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87ip1ng8mp@lennier.ganneff.de
Re: Why not to let all DDs to execute gb-command
On 2013-06-06 21:42, Paul Tagliamonte wrote: Well, I don't think adding more kruft to dak is a great idea (I mean, if it has to happen, it has to happen), but this really shows that we need a unified way of passing machine-readable messages between services. Let's see how the GSoC project turns out; I'm willing to bet even a proof of concept would be useful to services that interdepend so closely. I mean, not to play the UNIX card, but one thing well, you know? Let's just aid in the code talking to each other better. Oh yeah, if you give us secure and reliable(!) messaging over unreliable and untrusted networks, in way that DSA accepts, I would be very happy. ;-) buildd could use it instead of throwing away calls when the main host is temporarily unreachable. Kind regards Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/0dee130e32d02efca9939545151d6...@hub.kern.lc
Re: Why not to let all DDs to execute gb-command
* Philipp Kern pk...@debian.org [2013-06-07 12:26:37 +0200]: On 2013-06-06 21:42, Paul Tagliamonte wrote: Well, I don't think adding more kruft to dak is a great idea (I mean, if it has to happen, it has to happen), but this really shows that we need a unified way of passing machine-readable messages between services. Let's see how the GSoC project turns out; I'm willing to bet even a proof of concept would be useful to services that interdepend so closely. I mean, not to play the UNIX card, but one thing well, you know? Let's just aid in the code talking to each other better. Oh yeah, if you give us secure and reliable(!) messaging over unreliable and untrusted networks, in way that DSA accepts, I would be very happy. ;-) Hi, Secure should be pretty straightforward as every message is already cryptographically signed. I think most of the work (save from packaging) will be on the reliability side, as Fedora's infrastructure is more tightly integrated than ours. The accepted by DSA part is a bit complicated. We didn't really want to bother one of the busiest teams in Debian when the software wasn't even packaged, and, when it came up, I felt that the reception of the idea of using fedmsg was a bit lukewarm. I think we should be able to work things out when and if we have a working proof of concept. [snip] Cheers, -- Nicolas Dandrimont BOFH excuse #88: Boss' kid fucked up the machine signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
]] Nicolas Dandrimont The accepted by DSA part is a bit complicated. We didn't really want to bother one of the busiest teams in Debian when the software wasn't even packaged, and, when it came up, I felt that the reception of the idea of using fedmsg was a bit lukewarm. I think we should be able to work things out when and if we have a working proof of concept. From what I remember, we did have some questions about how some unique challenges in Debian's infrastructure will be solved, yes. I don't think I've seen completely satisfactory answers about it yet either, but I'd need to go digging to be entirely sure. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k3m5sqgq@xoog.err.no
Re: Why not to let all DDs to execute gb-command
2013/6/7 Tollef Fog Heen tfh...@err.no: ]] Nicolas Dandrimont The accepted by DSA part is a bit complicated. We didn't really want to bother one of the busiest teams in Debian when the software wasn't even packaged, and, when it came up, I felt that the reception of the idea of using fedmsg was a bit lukewarm. I think we should be able to work things out when and if we have a working proof of concept. From what I remember, we did have some questions about how some unique challenges in Debian's infrastructure will be solved, yes. I don't think I've seen completely satisfactory answers about it yet either, but I'd need to go digging to be entirely sure. At the Tanglu Debian derivative (which is just in formation right now) we build packages using Jenkins CI at the moment, due to this issue and wanna-build having problems to build source-only packages and a few other issues (difficult to set-up, weird handling of buildlogs and some other related issues) Jenkins isn't perfect either, so I am currently aiming to adjust buildbot for our needs, which might be a solution (and it easier to handle than a wb infrastructure, and can be integrated with the existing Python tools). Using Debian infrastructure outside of Debian is really difficult sometimes, because it grew with Debian and many parts of it are designed with a different image of Debian in mind, when it was much smaller. But there is great work going on in maintaining and improving infrastructure :-) (after digging into it, appreciate the work all the teams do even more!) In terms of infrastructure, IMHO Ubuntu did really well with the idea of Launchpad as integrated solution. I also like the messaging idea for Debian, but it might need some plan which parts of the infrastructure it will cover and how implementation could happen without too much extra work for the DSA. Wasn't there a GSoC project about it, or did I misread that? Cheers, Matthias -- I welcome VSRE emails. See http://vsre.info/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAKNHny8d7B=voynxj75u1k_vpvw_s8is2kzyefk__oodvtc...@mail.gmail.com
Re: Why not to let all DDs to execute gb-command
On Wed, Jun 05, 2013 at 09:10:39PM +0200, Anton Gladky wrote: I think, most of developers are clever enough to define, whether the built failed accidentally and needs to be restarted, or it requires some fixing and uploading new version. It will save time for both DDs and wb-team. Generally I'd have liked that. Currently there's no proper scheme to authorize only that action (wb talks directly to the DB and does the changes). That said, many requests seem to be let's retry that, maybe it doesn't fail (for SIGSEGV/SIGBUS). The interface should at least encourage developers to think twice. Kind regards Philipp Kern signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
Hi, Am Donnerstag, den 06.06.2013, 12:24 +0200 schrieb Philipp Kern: On Wed, Jun 05, 2013 at 09:10:39PM +0200, Anton Gladky wrote: I think, most of developers are clever enough to define, whether the built failed accidentally and needs to be restarted, or it requires some fixing and uploading new version. It will save time for both DDs and wb-team. Generally I'd have liked that. Currently there's no proper scheme to authorize only that action (wb talks directly to the DB and does the changes). That said, many requests seem to be let's retry that, maybe it doesn't fail (for SIGSEGV/SIGBUS). The interface should at least encourage developers to think twice. if the DPAs get wanna-build support (which I hope) you certainly don’t want to handle nmu and gb requests for every developer’s pet repositories, so there will be a need for a general interface. Can this be done using dcut files, just as with other DPA management commands? Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
Re: Why not to let all DDs to execute gb-command
2013/6/6 Joachim Breitner nome...@debian.org: [..] Can this be done using dcut files, just as with other DPA management commands? That would be fine. Anton -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CALF6qJnyF3w-Jai2U=hgkjqy9t0j55mbj-+rfgmuai6wmm_...@mail.gmail.com
Re: Why not to let all DDs to execute gb-command
On 2013-06-06 12:40, Joachim Breitner wrote: if the DPAs get wanna-build support (which I hope) you certainly don’t want to handle nmu and gb requests for every developer’s pet repositories, so there will be a need for a general interface. Can this be done using dcut files, just as with other DPA management commands? Which begs the question if build management should be merged into dak to provide insight about which package is new. Obviously this could also be a defined interface but if we're going to start to reuse dak's interfaces for that... maybe not. Currently what wanna-build does is looking at the whole Packages and Sources files after every update by dak. This is quite time-consuming[1]. I wonder if an upload processed by dak should create a new entry in the build management and management commands could then operate on those (i.e., records that describe the target archive, the needed overlays, and the package version to build). That's kind of what sojuz does, AFAIK, but that was designed with trusted VPN'ed networks in mind and people said at times that it is odd, too. We currently have one line per package/suite tuple that's modified and the raw changes are dumped into another table. It would be cleaner if there was a build entry that's completed with a log from the buildd and a new one being created for the next build to be run. Kind regards Philipp Kern [1] Yes, I know that quinn-diff was quick for the process of getting the expected state, but we also need to sync other bits. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5ec9abdcf2a8e1ca34dbcca9c0a46...@hub.kern.lc
Re: Why not to let all DDs to execute gb-command
On Thu, Jun 06, 2013 at 09:31:12PM +0200, Philipp Kern wrote: On 2013-06-06 12:40, Joachim Breitner wrote: Which begs the question if build management should be merged into dak to provide insight about which package is new. Obviously this could also be a defined interface but if we're going to start to reuse dak's interfaces for that... maybe not. Well, I don't think adding more kruft to dak is a great idea (I mean, if it has to happen, it has to happen), but this really shows that we need a unified way of passing machine-readable messages between services. Let's see how the GSoC project turns out; I'm willing to bet even a proof of concept would be useful to services that interdepend so closely. I mean, not to play the UNIX card, but one thing well, you know? Let's just aid in the code talking to each other better. My 2¢, anyway, Paul -- .''`. Paul Tagliamonte paul...@debian.org : :' : Proud Debian Developer `. `'` 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `- http://people.debian.org/~paultag signature.asc Description: Digital signature
Re: Why not to let all DDs to execute gb-command
Am Mittwoch, den 05.06.2013, 21:10 +0200 schrieb Anton Gladky: Dear all, I have a proposal to give a permission to all DDs to restart builds on failing archs e.g. execute gb-command. I think, most of developers are clever enough to define, whether the built failed accidentally and needs to be restarted, or it requires some fixing and uploading new version. It will save time for both DDs and wb-team. +1 In Ubuntu, developers can restart builds on failing archs (via Launchpad) for package that they can upload. I used that feature several times. -- Benjamin Drung Debian Ubuntu Developer -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1370469962.12617.12.camel@deep-thought
