Re: Discarding uploaded binary packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Le 26/10/2012 02:13, Peter Miller a écrit : It may be possible to address both concerns in a different way. 1. Implement PPAs. The code is open source, get it working first, and enhance it later. 2. DDs and DMs upload source-only to their individual PPA(s). The PPA build farm builds the package on all the architectures Debian cares about. 3. When a DD or DM wants to place a package to testing or experimental, it is done as a *copy* from a PPA (no upload required). If the package in the PPA didn't build, no copy happens. This means folks who pay $$$ for uploads don't have to upload the binary package that is discarded. But it also means the package is known to build (on all architectures) before being copied into testing or experimental. The autobuilders will still see many failing builds. Perhaps even more, if you openly advertise them as a way to perform test builds. I don't see what you gain from it. Regards, Thibaut. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJQinjwAAoJEJOUU0jg3ChAUwUQAJCrR+y1wb8EebuJgW89DHAN PJkIGdB1w3M7opk+oYNybGa+vEM5/4KITDLJ1/ie/l5bauEJQHoJMEog0UoILK2H TLEO1xlbc89hK6NWwEmGnrH68yxytMJNcJszdlxAX+94S1WvDzuu8H9e3P/cVKlU xg7ZOYnI/G4CjFkRJExTdgJaQF9pz/l7MffztIAt/rDrmsowXStwlqyGAbeYlydL JBHMeM/CB0V+gIFjMmK/n34uv9oPxGU4H7hzsJaKW+A0j9anvf3n+SkW8bzvBk6f VvCmAnk8XDzdSCw7llqE0dt1M5ffnIAGUfSHbzdhplMGPYeuPpu/aB3/H4r30fpk 8jir2YWJholoG8ngk+Xr8Y+eTW26YJuutwlN3bKTETuUW9EOSsiYXnXlO0UNOEGu C4Y6xJKFbCOWStLjYj+lT843tY53xqNukTBBhfnWFd+SkajMQsJTElyiW1VvRBot lSaXtQ6pjMiPZp7oBgKHOvYZrCV7jikcqbBOojx8f8dQywpKiWmvJZyQCLY3UgSN rCS39TdSTPHDlW22KNW9F4AoQQdGxUcWC+Rw5PjQ+7G+ilKDC+7mMl1w6DXT49Af YB43+B1eZzoseQhbMQylWq2LmoYJWSgURlzzJtXalTPwyRg8qaO0qrjo2Wevu1jg +N7K43qPk3MMN8h/w0NA =N717 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/508a7938.7070...@debian.org
Re: Discarding uploaded binary packages
On 26.10.2012 01:13, Peter Miller wrote: It may be possible to address both concerns in a different way. 1. Implement PPAs. The code is open source, get it working first, and enhance it later. 2. DDs and DMs upload source-only to their individual PPA(s). The PPA build farm builds the package on all the architectures Debian cares about. 3. When a DD or DM wants to place a package to testing or experimental, it is done as a *copy* from a PPA (no upload required). If the package in the PPA didn't build, no copy happens. s/testing/unstable/, presumably? It builds everywhere is only one criterion involved in testing migration; for all of the others, migration to testing from anywhere other than unstable doesn't really make sense. Regards, Adam -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/8713c24579dac159d4024ca710485...@mail.adsl.funky-badger.org
Re: Discarding uploaded binary packages
On Sat, 2012-10-20 at 20:10 +0200, Joerg Jaspert wrote: But my point was: if we're going to be dropping the uploaded binary in the first place, why do we have to upload it? Source-only uploads would make so much more sense. Only theoretical. Practical it would mean we will have many more build failures. Hi, You cant set a ranking system counting failures. Then queue uploads according to the rank of the uploader. This way persons get responsible by constraint, even if I think majority do not need this, but those will never suffer from this ranking. Of course you can keep this information internal to the system so that only few admins should access it. You can also give the possibility for DDs to ask resetting the ranking or make it so that it looses memory after x uploads (could be implemented as an IIR filter). Cheers, signature.asc Description: This is a digitally signed message part
Re: Discarding uploaded binary packages
It may be possible to address both concerns in a different way. 1. Implement PPAs. The code is open source, get it working first, and enhance it later. 2. DDs and DMs upload source-only to their individual PPA(s). The PPA build farm builds the package on all the architectures Debian cares about. 3. When a DD or DM wants to place a package to testing or experimental, it is done as a *copy* from a PPA (no upload required). If the package in the PPA didn't build, no copy happens. This means folks who pay $$$ for uploads don't have to upload the binary package that is discarded. But it also means the package is known to build (on all architectures) before being copied into testing or experimental. -- Regards Peter Miller pmil...@opensource.org.au /\/\*http://miller.emu.id.au/pmiller/ PGP public key ID: 1024D/D0EDB64D fingerprint = AD0A C5DF C426 4F03 5D53 2BDB 18D8 A4E2 D0ED B64D See http://pgp.mit.edu/ or any PGP keyserver for public key. It's hard enough to find an error in your code when you're looking for it; it's even harder when you've assumed your code is error free. -- Steve McConnell, Code Complete -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1351210388.3139.41.ca...@hawk.miller.emu.id.au
Re: Discarding uploaded binary packages
On Sat, Oct 20, 2012 at 06:29:50PM +0200, Bernhard R. Link wrote: * Chow Loong Jin hyper...@debian.org [121020 18:10]: The only argument I have seen for binary uploads is to ensure that DDs have built the package prior to uploading it. But as someone else pointed out earlier in the thread, we seem to be trusting DDs a lot in other aspects, so why not trust that they test-build packages prior to uploading them as well? Because trusting someone in one thing is not the same as trusting someone in another. Trust works best when there is accountablity. Having the binary file around, even if it is not easily accessible on some remote archive, noone can claim I tested this, it just did work here, something must be different on the buildds and hope to get away with it. How about this then. If and when we start allowing sourceful uploads, the following rules apply: - By default, packages don't need to be accompanied by a binary package (but may be) - If a previous version of a package failed to build from source on, say, more than half our release architectures, the next upload of that package (by the same person) needs to be accompanied by a binary package. I just came up with this, and obviously this would mean some (more) code would need to be written. But I think it could satisfy both sides of the argument. -- Copyshops should do vouchers. So that next time some bureaucracy requires you to mail a form in triplicate, you can mail it just once, add a voucher, and save on postage. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121021070037.ga11...@grep.be
Re: Discarding uploaded binary packages
On Sun, Oct 21, 2012 at 10:16:24AM +0800, Chow Loong Jin wrote: There are two main arguments: why should we upload binaries if they will be discarded anyway and if we allow source-only uploads people will upload packages that weren't tested to be buildable. Please don't repeat these arguments, it's pointless. Please. Great, so let's just leave it at a stalemate and not get anything done. Repeating those two arguments over and over again is a stalemate too, and it's the current situation, apparently :( -- WBR, wRAR signature.asc Description: Digital signature
Re: Discarding uploaded binary packages
* Steve Langasek: I am aware that other such packages exist. I just don't think we should support them if they can't be bootstrapped properly. Ocaml is in this category as well, and it addresses it by bootstrapping off an upstream-provided binary blob. I'm not sure if this is the right approach. Cyclic build dependencies might require explicit overrides from time to time, but at least we start the build from source code. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/871ugs2awh@mid.deneb.enyo.de
Re: Discarding uploaded binary packages
* Joerg Jaspert: The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. Could we instruct the buildd for the upload architecture to build arch-all packages, and let the others operate as before? This should have the highest chance of success. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87zk3g0wbg@mid.deneb.enyo.de
Re: Discarding uploaded binary packages
Arno Töll arno at debian.org writes: Pretending we had a working concept to throw away binaries and how to deal with arch:all packages, why don't we introduce a control/changes file flag similar in spirit to XS-Autobuild: yes instructing dak not to throw away binaries upon explicit request - say XS-keep-binary: yes. For packages that can be built with an older version of itself, like fpc and, normally, gcc, that may not even be needed. For the harder cases… why not something that says “please keep the binary from my upload, install it on one buildd and immediately schedule a(n automatic) binNMU for that package”, so we’ll have to never keep DD-built binaries? (As for gcc… I think if a gcc-x.y upload cannot be built by the gcc-x.y already in the archive or gcc-x.$((y-1)), someone is doing something wrong… but even there, an upload specifying, using a new field in the .changes file, that the binaries should be used to immediately schedule a binNMU on the source package using them sounds like a reasonable solution to me.) While porter uploads are a useful tool, if we really want to go through with the binary package sanitising, they should *not* be allowed for the main archive, only for debian-ports. (I’ve seen enough broken uploads there to agree with the rule that only packages built on/by buildds should ever enter the archive proper.) While there… what about source packages that build arch:all packages that need to be composed from several build arches? (Thinking of the old pcc-libs case… but since M-A I’ll probably convert them to one co-installable arch:any binary, so it won’t be an issue for me any more.) bye, //mirabilos -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/loom.20121021t172735-...@post.gmane.org
Re: Discarding uploaded binary packages
Le Sun, Oct 21, 2012 at 10:20:39AM +0200, olivier sallou a écrit : But I really like the idea of sending a binary build that is dropped by the build system. It would avoid sending accidently (or not) a package that does not build at all and uses resources (servers, ...) effortless. Hi Olivier, It will happen anyway that packages will not build. For instance it happened to me in the past to upload a package built on a chroot for which I did not realise that it was using a mirror that was not up to date. There are also possible race conditions with uploads of a package's build dependancies. In all these cases, the locally built binary packages are again of limited use. Personally, for each upload I make to the Debian archive, I always send the build logs or commit them in the package's VCS. This is the first step towards debugging if the autobuilding fails. If it is necessary for DDs to certify that they built binary packages (but again, that does not prove that they tested them, etc.), how about asking them to publish the logs instead of asking them to upload the binary packages ? Or conversely, if it becomes a general practice to publish our build logs, it may become unnecessary to implement control procedures in our infrastructure. Cheers, -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121022004307.gb8...@falafel.plessy.net
Re: Discarding uploaded binary packages
On 10/17/2012 09:56 AM, Chow Loong Jin wrote: On 17/10/2012 08:36, Russell Coker wrote: On Wed, 17 Oct 2012, Barry Warsawba...@python.org wrote: I also think allowing source-only uploads makes for easier contributions, and thus hopefully more contributions. Why would it be easier? Surely we still want people to build packages first to ensure that we don't needlessly get FTBFS bugs. Because binary packages are big, and uploading them reliably from a region with crappy internet access sucks, especially when trying to upload them over SFTP. Honestly, if we're not going to be using these, why upload them? It's a pointless waste of bandwidth. Dropping the uploaded binary and rebuilding it after upload doesn't necessarily mean that we allow uploading a source-only upload. I think it would be a good thing to continue to require source + binary. What would be even better, would be to rebuild, and if there's a difference with what was uploaded (for example, calculated library dependencies), then reject the upload. The main point of dropping uploaded binary, IMO, is to make sure that the binary is built with the correct library currently in SID (not everyone uses pbuilder / cowbuilder, and mistakes can happen). Cheers, Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5082b761.2080...@debian.org
Re: Discarding uploaded binary packages
On 20/10/2012 22:38, Thomas Goirand wrote: On 10/17/2012 09:56 AM, Chow Loong Jin wrote: On 17/10/2012 08:36, Russell Coker wrote: On Wed, 17 Oct 2012, Barry Warsawba...@python.org wrote: I also think allowing source-only uploads makes for easier contributions, and thus hopefully more contributions. Why would it be easier? Surely we still want people to build packages first to ensure that we don't needlessly get FTBFS bugs. Because binary packages are big, and uploading them reliably from a region with crappy internet access sucks, especially when trying to upload them over SFTP. Honestly, if we're not going to be using these, why upload them? It's a pointless waste of bandwidth. Dropping the uploaded binary and rebuilding it after upload doesn't necessarily mean that we allow uploading a source-only upload. I think it would be a good thing to continue to require source + binary. What would be even better, would be to rebuild, and if there's a difference with what was uploaded (for example, calculated library dependencies), then reject the upload. The main point of dropping uploaded binary, IMO, is to make sure that the binary is built with the correct library currently in SID (not everyone uses pbuilder / cowbuilder, and mistakes can happen). But my point was: if we're going to be dropping the uploaded binary in the first place, why do we have to upload it? Source-only uploads would make so much more sense. The only argument I have seen for binary uploads is to ensure that DDs have built the package prior to uploading it. But as someone else pointed out earlier in the thread, we seem to be trusting DDs a lot in other aspects, so why not trust that they test-build packages prior to uploading them as well? -- Kind regards, Loong Jin signature.asc Description: OpenPGP digital signature
Re: Discarding uploaded binary packages
* Chow Loong Jin hyper...@debian.org [121020 18:10]: The only argument I have seen for binary uploads is to ensure that DDs have built the package prior to uploading it. But as someone else pointed out earlier in the thread, we seem to be trusting DDs a lot in other aspects, so why not trust that they test-build packages prior to uploading them as well? Because trusting someone in one thing is not the same as trusting someone in another. Trust works best when there is accountablity. Having the binary file around, even if it is not easily accessible on some remote archive, noone can claim I tested this, it just did work here, something must be different on the buildds and hope to get away with it. Given that source only uploads where tried in another project and the results are scary, this accountability might make the difference to make it work. And to also name another argument: having the files actually uploaded means it is easy to add additional checks. (Like starting with making sure the list of files does not differ between the two versions, or some check to see only versions of generated dependencies differ but not the packages depended and so on). Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121020162948.gb14...@client.brlink.eu
Re: Discarding uploaded binary packages
But my point was: if we're going to be dropping the uploaded binary in the first place, why do we have to upload it? Source-only uploads would make so much more sense. Only theoretical. Practical it would mean we will have many more build failures. The only argument I have seen for binary uploads is to ensure that DDs have built the package prior to uploading it. But as someone else pointed out earlier in the thread, we seem to be trusting DDs a lot in other aspects, so why not trust that they test-build packages prior to uploading them as well? And we trust people to know their way around open source and see whats clearly non-free. Or incompatible licenses (to a point). Yet, we have MANY rejects from NEW for even very simple to spot issues. Trust may work - but it has to be earned. From a rough guess, I would trust around 10 to 20% of our uploaders to do it right most of the time.[1] For some more deep insight you might want to talk to Ubuntu people. They do allow source-only uploads, and I seem to remember them having written that it lead to lots of useless uploads that just can't have been tested.[2] [1] And that doesn't mean that I think 80 or 90% of DDs are stupid/brainless idiots. (That number is smaller). But it is *WAY* to easy and convinient to go ah, just upload this, I think it works, no time for testing now, when you can do it. And this will happen. [2] I am not an ubuntu person. This is from reading stuff in mailing lists and on irc and my memory might be wrong. Go to them to get a definite say. -- bye, Joerg Flanders has cooties … Flanders has cooties … Flanders has cooties … -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87txtpj9pz@gkar.ganneff.de
Re: Discarding uploaded binary packages
* Joerg Jaspert jo...@debian.org, 2012-10-20, 20:10: For some more deep insight you might want to talk to Ubuntu people. They do allow source-only uploads, and I seem to remember them having written that it lead to lots of useless uploads that just can't have been tested. http://lists.debian.org/20091116012917.gc4...@dario.dodds.net -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121020192626.ga5...@jwilk.net
Re: Discarding uploaded binary packages
On Sun, Oct 21, 2012 at 12:10:02AM +0800, Chow Loong Jin wrote: I also think allowing source-only uploads makes for easier contributions, and thus hopefully more contributions. Why would it be easier? Surely we still want people to build packages first to ensure that we don't needlessly get FTBFS bugs. Because binary packages are big, and uploading them reliably from a region with crappy internet access sucks, especially when trying to upload them over SFTP. Honestly, if we're not going to be using these, why upload them? It's a pointless waste of bandwidth. Dropping the uploaded binary and rebuilding it after upload doesn't necessarily mean that we allow uploading a source-only upload. I think it would be a good thing to continue to require source + binary. What would be even better, would be to rebuild, and if there's a difference with what was uploaded (for example, calculated library dependencies), then reject the upload. The main point of dropping uploaded binary, IMO, is to make sure that the binary is built with the correct library currently in SID (not everyone uses pbuilder / cowbuilder, and mistakes can happen). But my point was: if we're going to be dropping the uploaded binary in the first place, why do we have to upload it? Source-only uploads would make so much more sense. There are two main arguments: why should we upload binaries if they will be discarded anyway and if we allow source-only uploads people will upload packages that weren't tested to be buildable. Please don't repeat these arguments, it's pointless. Please. -- WBR, wRAR signature.asc Description: Digital signature
Re: Discarding uploaded binary packages
Le Sat, Oct 20, 2012 at 06:29:50PM +0200, Bernhard R. Link a écrit : * Chow Loong Jin hyper...@debian.org [121020 18:10]: The only argument I have seen for binary uploads is to ensure that DDs have built the package prior to uploading it. But as someone else pointed out earlier in the thread, we seem to be trusting DDs a lot in other aspects, so why not trust that they test-build packages prior to uploading them as well? Having the binary file around, even if it is not easily accessible on some remote archive, noone can claim I tested this, it just did work here, something must be different on the buildds and hope to get away with it. In the end, it is not so relevant that a local build could work on the DD's machine. With source uploads, we will switch from a model where most of the packages that our users install were built and tested by hand, to a model where most pakcages were autobuilt. In that model, it will be important that after the build, one developer downloads the binary packages and tests them. The 10 days of staging in Unstable do not offer much protection when packages have a small number of users who are not upgrading their system every week. In that context, it would be helpful to have an email notification after the packcages are autobuilt on amd64. For command-line programs, regression tests with autopkgtest will also be instrumental in helping developers to ensure that binary packages are functional. (http://dep.debian.net/deps/dep8/) Have a nice Sunday, -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121021004724.gb23...@falafel.plessy.net
Re: Discarding uploaded binary packages
On 21/10/2012 02:10, Joerg Jaspert wrote: For some more deep insight you might want to talk to Ubuntu people. They do allow source-only uploads, and I seem to remember them having written that it lead to lots of useless uploads that just can't have been tested.[2] I am an Ubuntu developer though, and I haven't seen any evidence that source-only uploads are a problem in Ubuntu. But then again I've only been actively involved in Ubuntu development since 2008, so I wouldn't know about any issues that have appeared in the past. -- Kind regards, Loong Jin signature.asc Description: OpenPGP digital signature
Re: Discarding uploaded binary packages
On 21/10/2012 05:17, Andrey Rahmatullin wrote: On Sun, Oct 21, 2012 at 12:10:02AM +0800, Chow Loong Jin wrote: I also think allowing source-only uploads makes for easier contributions, and thus hopefully more contributions. Why would it be easier? Surely we still want people to build packages first to ensure that we don't needlessly get FTBFS bugs. Because binary packages are big, and uploading them reliably from a region with crappy internet access sucks, especially when trying to upload them over SFTP. Honestly, if we're not going to be using these, why upload them? It's a pointless waste of bandwidth. Dropping the uploaded binary and rebuilding it after upload doesn't necessarily mean that we allow uploading a source-only upload. I think it would be a good thing to continue to require source + binary. What would be even better, would be to rebuild, and if there's a difference with what was uploaded (for example, calculated library dependencies), then reject the upload. The main point of dropping uploaded binary, IMO, is to make sure that the binary is built with the correct library currently in SID (not everyone uses pbuilder / cowbuilder, and mistakes can happen). But my point was: if we're going to be dropping the uploaded binary in the first place, why do we have to upload it? Source-only uploads would make so much more sense. There are two main arguments: why should we upload binaries if they will be discarded anyway and if we allow source-only uploads people will upload packages that weren't tested to be buildable. Please don't repeat these arguments, it's pointless. Please. Great, so let's just leave it at a stalemate and not get anything done. -- Kind regards, Loong Jin signature.asc Description: OpenPGP digital signature
Re: Discarding uploaded binary packages
On 17-10-12 23:48, Matthias Klose wrote: On 17.10.2012 21:49, Steve Langasek wrote: On Wed, Oct 17, 2012 at 11:30:38AM -0700, Christoph Egger wrote: Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source Which is, frankly, an absurd requirement. Someone should fix this package to bootstrap properly, and if disallowing binary uploads forces the issue, that's a good thing. you know better. The last one I did identify is eigenbase-resgen. Others that come to mind are fpc, mlton, ... and adding new features / packages to the GNU toolchain requires manual interaction for glibc/gcc uploads, which can't be done with source only uploads. Just to be sure nobody is reading the above statement about fpc different than it should. fpc should always build with a reasonable older version of itself [1]. So yes, it needs bootstrapping, but it should only need it once (currently per architecture I believe). Paul [1] http://wiki.freepascal.org/Release_engineering#General_notes_about_release_building To create a build, you always have to start with compiler from the last previous major release (e.g. 2.0.0 for all 2.0.x and 2.2.0 releases, 2.2.0 for 2.2.x, etc.), or the last previous minor release (i.e. you can use 2.0.2 for building 2.0.4). signature.asc Description: OpenPGP digital signature
Re: Discarding uploaded binary packages
Hi, On 18.10.2012 04:29, Paul Wise wrote: On Thu, Oct 18, 2012 at 10:20 AM, Russell Coker wrote: We could have a lintian warning for any occurance of the string /home in a packaged file and have error conditions for /build and the current value of $HOME for the account running lintian. Based on a quick grep of /usr/bin on my laptop, this is going to have a fair number of false positives; commented out paths, paths in POD documentation, URLs, paths outside of /home that include /home in their name somewhere. Moreover, I bet there is a fair number of DDs building their stuff in /srv, /some/random/non-fhs/directory and whatever else one could imagine, possibly including /tmp. Scanning for a particular directory or, list of directories, does not really help. -- with kind regards, Arno Töll IRC: daemonkeeper on Freenode/OFTC GnuPG Key-ID: 0x9D80F36D signature.asc Description: OpenPGP digital signature
Re: Discarding uploaded binary packages
So yes, this is something that should be accounted for if Debian moves to a model where binary uploads are discarded and rebuilt. However, I suspect that for all the sensible cases, the proposal to add staged-build metadata (for bootstrapping circular build-dependencies on new ports) would be sufficient to make this problem go away too. But I'm a lot less concerned about the kinds of self-build-depending packages whose names are frequently taken in vain (raise your hand if you've ever tried to fix an RC bug in mlton). As one thing to keep in mind - we have an acl structure in dak. Currently it reads something like all DD keys are allowed all uploads. all DM keys are allowed their own uploads according to DM rights. all buildd keys are allowed binary only uploads for their arch. It is not impossible nor excluded to have a set of rules like all DD keys are allowed all uploads, binaries dropped all DM keys are allowed their own uploads according to DM rights, binaries dropped all buildd keys are allowed binary only uploads for their arch. one of some DDs may upload $listofpackages including binaries. all DDs may upload $listofpackages including binaries. where listofpackages is those insane needthemself ones. And could be by DD. However far we want to drive this, we can. -- bye, Joerg [...]that almost anything related to intellectual property is idiotic by it's nature, [...] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87a9vkrwor@gkar.ganneff.de
Re: Discarding uploaded binary packages
Hi, On 18.10.2012 10:50, Joerg Jaspert wrote: some DDs may upload $listofpackages including binaries. all DDs may upload $listofpackages including binaries. where listofpackages is those insane needthemself ones. And could be by DD. However far we want to drive this, we can. why don't we just let people decide themselves? We already grant every DD very liberal upload permissions to the archives because we trust them and their capabilities. Hence, I do not think we would like something like this despite of dak's support for it. Pretending we had a working concept to throw away binaries and how to deal with arch:all packages, why don't we introduce a control/changes file flag similar in spirit to XS-Autobuild: yes instructing dak not to throw away binaries upon explicit request - say XS-keep-binary: yes. -- with kind regards, Arno Töll IRC: daemonkeeper on Freenode/OFTC GnuPG Key-ID: 0x9D80F36D signature.asc Description: OpenPGP digital signature
Re: Discarding uploaded binary packages
On Thu, 18 Oct 2012, Arno Töll wrote: On 18.10.2012 04:29, Paul Wise wrote: On Thu, Oct 18, 2012 at 10:20 AM, Russell Coker wrote: We could have a lintian warning for any occurance of the string /home in a packaged file and have error conditions for /build and the current value of $HOME for the account running lintian. Based on a quick grep of /usr/bin on my laptop, this is going to have a fair number of false positives; commented out paths, paths in POD documentation, URLs, paths outside of /home that include /home in their name somewhere. Moreover, I bet there is a fair number of DDs building their stuff in /srv, /some/random/non-fhs/directory and whatever else one could imagine, possibly including /tmp. Scanning for a particular directory or, list of directories, does not really help. It does on buildds. If it is broken on the buildd, it is broken everywhere... -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121018102402.ga20...@khazad-dum.debian.net
Re: Discarding uploaded binary packages
❦ 18 octobre 2012 11:29 CEST, Arno Töll a...@debian.org : why don't we just let people decide themselves? We already grant every DD very liberal upload permissions to the archives because we trust them and their capabilities. Hence, I do not think we would like something like this despite of dak's support for it. Pretending we had a working concept to throw away binaries and how to deal with arch:all packages, why don't we introduce a control/changes file flag similar in spirit to XS-Autobuild: yes instructing dak not to throw away binaries upon explicit request - say XS-keep-binary: yes. Or we can just allow upload of *_source.changes and we trust the DD to have tried the build. -- Use free-form input when possible. - The Elements of Programming Style (Kernighan Plauger) pgpLvQiRNWiEr.pgp Description: PGP signature
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 2:56 AM, Joerg Jaspert wrote: On 13001 March 1977, Michael Gilbert wrote: So, are we ready to do this? No. Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Adjusting dak then to throw away stuff and only allow buildd keys to have .debs pass through isn't all that hard. Couldn't this be broken into two steps? Step 1: Discard arch-specific packages but keep arch all ones - dak throws away all arch-specific binary packages - also allows only buildd keys for arch-specific packages - but continues to allow DD keys for arch all Step 2: Discard all binary packages - implement some kind of arch all buildd solution - dak throws away all binary packages - only allows buildd key signatures The first would be easy (for FTP masters) and would be a good initial step forward. The second would require some hard work and may take some time. Best wishes, Mike -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANTw=mpby4_7jmg22vuwe201heszur535d8azlkyqs312ql...@mail.gmail.com
Re: Discarding uploaded binary packages
[Joerg Jaspert] As one thing to keep in mind - we have an acl structure in dak. Currently it reads something like all DD keys are allowed all uploads. all DM keys are allowed their own uploads according to DM rights. all buildd keys are allowed binary only uploads for their arch. It is not impossible nor excluded to have a set of rules like all DD keys are allowed all uploads, binaries dropped all DM keys are allowed their own uploads according to DM rights, binaries dropped all buildd keys are allowed binary only uploads for their arch. Paul Tagliamonte had a nice idea on IRC: all DD keys are allowed all uploads, binaries dropped _iff_ there is a source component This preserves the ability to upload a manual binNMU, which is not common, but is useful and sometimes necessary. (And not only for bootstrapping an arch or a compiler.) Peter -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121019023025.gd4...@p12n.org
Re: Discarding uploaded binary packages
This preserves the ability to upload a manual binNMU, which is not common, but is useful and sometimes necessary. (And not only for bootstrapping an arch or a compiler.) ...and I forgot to add that something like this is required by the GR http://www.debian.org/vote/2007/vote_002, or at least the spirit of it. (The _letter_ of the GR could be argued either way: am I technically allowed to perform combined source and binary package uploads if my binaries are automatically discarded? But in my opinion the _spirit_ of the GR is clear.) Peter -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121019030505.ge4...@p12n.org
Re: Discarding uploaded binary packages
Peter Samuelson pe...@p12n.org writes: This preserves the ability to upload a manual binNMU, which is not common, but is useful and sometimes necessary. (And not only for bootstrapping an arch or a compiler.) ...and I forgot to add that something like this is required by the GR http://www.debian.org/vote/2007/vote_002, or at least the spirit of it. (The _letter_ of the GR could be argued either way: am I technically allowed to perform combined source and binary package uploads if my binaries are automatically discarded? But in my opinion the _spirit_ of the GR is clear.) IIRC, the phrasing of that GR was very intentionally done so that it would not rule out discarding all binary uploads from DDs and requiring all packages be built on buildds. That's why it says that you're allowed to do a binary NMU if you're allowed to upload binaries with source, not if you're allowed to upload source packages in general. I'm fairly sure this exact point came up during the GR discussion, although it's been five years, so my memory isn't exact. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87a9vj402y@windlord.stanford.edu
Re: Discarding uploaded binary packages
On Thu, 2012-10-18 at 21:30:25 -0500, Peter Samuelson wrote: [Joerg Jaspert] As one thing to keep in mind - we have an acl structure in dak. Currently it reads something like all DD keys are allowed all uploads. all DM keys are allowed their own uploads according to DM rights. all buildd keys are allowed binary only uploads for their arch. It is not impossible nor excluded to have a set of rules like all DD keys are allowed all uploads, binaries dropped all DM keys are allowed their own uploads according to DM rights, binaries dropped all buildd keys are allowed binary only uploads for their arch. Paul Tagliamonte had a nice idea on IRC: all DD keys are allowed all uploads, binaries dropped _iff_ there is a source component This preserves the ability to upload a manual binNMU, which is not common, but is useful and sometimes necessary. (And not only for bootstrapping an arch or a compiler.) I had thought that was already the conclusion from the previous discussion when these problems were brought up. But it appears most of it is being rehashed again. regards, guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121019032906.ga10...@gaara.hadrons.org
Re: Discarding uploaded binary packages
On 13001 March 1977, Michael Gilbert wrote: So, are we ready to do this? No. Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Adjusting dak then to throw away stuff and only allow buildd keys to have .debs pass through isn't all that hard. -- bye, Joerg * libpng2 no libpng3 no why ? because no yes no yes no yes bullshit no yes no yes no yes stop ? no when someday beep beep beep beep (Closes: #157011) -- Christian Marillat maril...@debian.org Thu, 29 Aug 2002 16:41:58 +0200 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87obk1wpqf@gkar.ganneff.de
Re: Discarding uploaded binary packages
On Tuesday, October 16, 2012 05:04:55, martin f krafft wrote: also sprach Holger Levsen hol...@layer-acht.org [2012.10.16.0945 +0200]: We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. your point being? That our users don't seem to care, and that probably is why we haven't done anything about it. Out of curiosity, how would a user /know/ whether a package has been built via a buildd rather than on a DD's local machine? -- Chris -- Chris Knadle chris.kna...@coredump.us -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210171228.14376.chris.kna...@coredump.us
Re: Discarding uploaded binary packages
On Wed, 17 Oct 2012 12:28:14 -0400 Chris Knadle chris.kna...@coredump.us wrote: Out of curiosity, how would a user /know/ whether a package has been built via a buildd rather than on a DD's local machine? Check the wannabuild logs for the package. A listing of Installed without a build log means that the binaries for that architecture were uploaded by the DD. https://buildd.debian.org/status/package.php?p=qof No logs for amd64, so developer (me) uploads amd64 binaries. All the other Installed listings link to the build log for that version on that architecture. Any source package which only has Architecture: all binary packages is uploaded by the DD although there can be binNMU's. https://buildd.debian.org/status/package.php?p=emdebian-grip So this counts for the majority of the perl, shell, python and other interpreted language packages. Build logs are accessible via the PTS too: http://packages.qa.debian.org/e/emdebian-grip.html -- Neil Williams = http://www.linux.codehelp.co.uk/ pgpqleLrBantu.pgp Description: PGP signature
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 12:28:14PM -0400, Chris Knadle wrote: On Tuesday, October 16, 2012 05:04:55, martin f krafft wrote: also sprach Holger Levsen hol...@layer-acht.org [2012.10.16.0945 +0200]: We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. your point being? That our users don't seem to care, and that probably is why we haven't done anything about it. Out of curiosity, how would a user /know/ whether a package has been built via a buildd rather than on a DD's local machine? Everyone can check buildd.debian.org for the lack of a build log on a particular architecture for a given version. That's a fairly good indicator. -- Copyshops should do vouchers. So that next time some bureaucracy requires you to mail a form in triplicate, you can mail it just once, add a voucher, and save on postage. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121017164514.gc22...@grep.be
Re: Discarding uploaded binary packages
On Wednesday, October 17, 2012 12:45:14, Wouter Verhelst wrote: On Wed, Oct 17, 2012 at 12:28:14PM -0400, Chris Knadle wrote: On Tuesday, October 16, 2012 05:04:55, martin f krafft wrote: also sprach Holger Levsen hol...@layer-acht.org [2012.10.16.0945 +0200]: We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. your point being? That our users don't seem to care, and that probably is why we haven't done anything about it. Out of curiosity, how would a user /know/ whether a package has been built via a buildd rather than on a DD's local machine? Everyone can check buildd.debian.org for the lack of a build log on a particular architecture for a given version. That's a fairly good indicator. Okay that's good to know. Thanks. I'm glad this came up again now, because the discussion has explained some of the detial behind some of the statements I've heard at DebConfs concerning Debian will be using source-only uploads `any day now`. ;-) -- Chris -- Chris Knadle chris.kna...@coredump.us -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210171316.27314.chris.kna...@coredump.us
Re: Discarding uploaded binary packages
Hi! Joerg Jaspert jo...@debian.org writes: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source Regards Christoph -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/871ugxgddd@mitoraj.siccegge.de
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 11:30:38AM -0700, Christoph Egger wrote: Joerg Jaspert jo...@debian.org writes: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source Which is, frankly, an absurd requirement. Someone should fix this package to bootstrap properly, and if disallowing binary uploads forces the issue, that's a good thing. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Re: Discarding uploaded binary packages
On 17 October 2012 19:30, Christoph Egger christ...@debian.org wrote: Hi! Joerg Jaspert jo...@debian.org writes: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source See Debian Policy 7.8 Additional source packages used to build the binary - Built-Using [1] Is exactly what should be used here. And the bootstrap package should be uploaded into Debian, to facilitate downstream users and distributions to bootstrap that package by themselves as well provide a clear lock-step build-deps. [1] http://www.debian.org/doc/debian-policy/ch-relationships.html#s-built-using Regards, Dmitrijs. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/canbhlujckg-xsogfjevdn_bax0jvzy8jkouavvhphrkmyr7...@mail.gmail.com
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 11:30:38AM -0700, Christoph Egger wrote: Hi! Joerg Jaspert jo...@debian.org writes: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source So, you currently upload every arch with the sourceful upload? Or do you upload, wait for the buildd build-dep lock, and do a binary-only upload? I don't think anyone wants to get rid of binary-only uploads. Binary only uploads get rejected if there is no source package anyway, so it's pretty easy to still allow binary-only-uploads and strip a .deb on the sourceful upload. No reason to make this default, though. Regards Christoph -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/871ugxgddd@mitoraj.siccegge.de -- .''`. Paul Tagliamonte paul...@debian.org : :' : Proud Debian Developer `. `'` 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `- http://people.debian.org/~paultag signature.asc Description: Digital signature
Re: Discarding uploaded binary packages
* Michael Gilbert mgilb...@debian.org [121016 04:59]: Anyway, all of these build system path sanitization issues can be eliminated by using the buildds for all architectures, since paths will start with at least /build that requires root-level action to exist on users' systems. They are not eliminated by using only buildds. They are only moved towards people that want to build their privately patched software then, which is something they have a right to do. A package not building properly or differently when built in a clean system with all the build-depended packages installed and all the build-conflicted packages not installed is a critical bug. Recreating binary packages uploaded by maintainers has some merrits, but this is definitely not one of them... Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121017200750.ga5...@client.brlink.eu
Re: Discarding uploaded binary packages
Paul Tagliamonte paul...@debian.org writes: On Wed, Oct 17, 2012 at 11:30:38AM -0700, Christoph Egger wrote: Joerg Jaspert jo...@debian.org writes: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source So, you currently upload every arch with the sourceful upload? Or do you upload, wait for the buildd build-dep lock, and do a binary-only upload? Guess why this package is Arch: i386 only ;-) I'm not uploading at all just being in the same packaging team. I don't think anyone wants to get rid of binary-only uploads. I read Joerg's Mail as dropping all *.deb that aren't signed by a buildd. Regards Christoph -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87wqyog8ns@mitoraj.siccegge.de
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 4:07 PM, Bernhard R. Link brl...@debian.org wrote: * Michael Gilbert mgilb...@debian.org [121016 04:59]: Anyway, all of these build system path sanitization issues can be eliminated by using the buildds for all architectures, since paths will start with at least /build that requires root-level action to exist on users' systems. They are not eliminated by using only buildds. They are only moved towards people that want to build their privately patched software then, which is something they have a right to do. A package not building properly or differently when built in a clean system with all the build-depended packages installed and all the build-conflicted packages not installed is a critical bug. Recreating binary packages uploaded by maintainers has some merrits, but this is definitely not one of them... Really? Take a look the affected isc-dhcp package. The prefix for the amd64 architecture (the one I built and uploaded) is /home/username/ which someone with a username account can put code in. The other architectures are /build/build-isc-dhcp which does not exist without root creating it at some point. Now that's still not right, but its more of a hurdle for someone trying to take advantage of the issue. Anyway, reading again, I not sure that your reply actually considers build path sanitization problems, which is what my statement was about. Best wishes, Mike -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANTw=mnu06tmmwz9ntzrnt9vvptkuo+bdx1cp1+rs6thgor...@mail.gmail.com
Re: Discarding uploaded binary packages
* Michael Gilbert mgilb...@debian.org [121017 22:19]: Anyway, reading again, I not sure that your reply actually considers build path sanitization problems, which is what my statement was about. I'm stating that doing all the builds on buildds will not avoid the need to fix the package. (Unless you are arguing that people locally modifying their packages are supposed to get security problems). Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121017205532.gb5...@client.brlink.eu
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 4:55 PM, Bernhard R. Link wrote: * Michael Gilbert mgilb...@debian.org [121017 22:19]: Anyway, reading again, I not sure that your reply actually considers build path sanitization problems, which is what my statement was about. I'm stating that doing all the builds on buildds will not avoid the need to fix the package. Ubuntu chose to come to that conclusion on this issue. (Unless you are arguing that people locally modifying their packages are supposed to get security problems). That is true: if there is a build path sanitization issue, then if the user chooses to rebuild the package they will get their own rogue paths. So, yes, we should always fix those issues when they're found, but at least for people using buildd'd packages, it's less of a problem. Best wishes, Mike -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANTw=moucn6danyx5-hqenjzfufsaxgshahuem0_x4ox48z...@mail.gmail.com
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 5:23 PM, Michael Gilbert wrote: That is true: if there is a build path sanitization issue, then if the user chooses to rebuild the package they will get their own rogue paths. So, yes, we should always fix those issues when they're found, but at least for people using buildd'd packages, it's less of a problem. Maybe someone would be interested in writing a lintian check for these issues? Something a bit more advanced than this $ strings /sbin/dhclient | grep ^PATH PATH=/home/zero79/source/git/isc-dhcp/debian/tmp/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin would have caught the issue (advanced aspect being that reasonable paths are ok'd). Best wishes, Mike -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANTw=mmcwq_zaz_qov6poa9j5xmdcwykxgnci3kvay9pacb...@mail.gmail.com
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 05:23:22PM -0400, Michael Gilbert wrote: if there is a build path sanitization issue, then if the user chooses to rebuild the package they will get their own rogue paths. So, yes, we should always fix those issues when they're found, but at least for people using buildd'd packages, it's less of a problem. For the build path, I see no benefit in having it be always the same on buildds. What about intentionally keeping the path diverse? Heck, we could even make sure it's 4096 characters long to weed out PATH_MAX brain damage, but that's probably something that belongs in Lucas Nussbaum land. -- Copyright and patents were never about promoting culture and innovations; from the very start they were legalized bribes to give the king some income and to let businesses get rid of competition. For some history, please read https://en.wikipedia.org/wiki/Statute_of_Monopolies_1623 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121017214008.ga30...@angband.pl
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 08:56:56AM +0200, Joerg Jaspert wrote: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. It's not exactly outside of dak. I think we should sketch out what's needed post-wheezy. (I.e. it would help us if the uploads accepted would be machine readable so that we can figure out what exactly, mainly architecture-wise, entered the archive and when. Given that we won't have fully architecture independent builds we'd also need to deal with rejects because the arch:all entered the archive through another upload. I have a set of notes on that at home.) Adjusting dak then to throw away stuff and only allow buildd keys to have .debs pass through isn't all that hard. I don't think that will happen or we'll have a GR about it. (The throw-away of binary-only uploads, that is.) Kind regards Philipp Kern signature.asc Description: Digital signature
Re: Discarding uploaded binary packages
On 17.10.2012 21:49, Steve Langasek wrote: On Wed, Oct 17, 2012 at 11:30:38AM -0700, Christoph Egger wrote: Joerg Jaspert jo...@debian.org writes: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source Which is, frankly, an absurd requirement. Someone should fix this package to bootstrap properly, and if disallowing binary uploads forces the issue, that's a good thing. you know better. The last one I did identify is eigenbase-resgen. Others that come to mind are fpc, mlton, ... and adding new features / packages to the GNU toolchain requires manual interaction for glibc/gcc uploads, which can't be done with source only uploads. Matthias -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/507f27a1.2030...@debian.org
Re: Discarding uploaded binary packages
On Wed, Oct 17, 2012 at 11:48:17PM +0200, Matthias Klose wrote: On 17.10.2012 21:49, Steve Langasek wrote: On Wed, Oct 17, 2012 at 11:30:38AM -0700, Christoph Egger wrote: Joerg Jaspert jo...@debian.org writes: Its for after wheezy, definitely. Also, there are some open issues to be solved for this to happen. The most important is being able to deal with arch all packages. And worse - arch all packages able to build only on certain architectures. But thats outside dak and my area. Goto the buildd/wanna-build people to help for that. Also remeber, there are packages like cmucl that can only be built by the same upstream release of itself and can currently survive in Debian because the maintainer can upload a bootstrapped binary package along the source Which is, frankly, an absurd requirement. Someone should fix this package to bootstrap properly, and if disallowing binary uploads forces the issue, that's a good thing. you know better. The last one I did identify is eigenbase-resgen. Others that come to mind are fpc, mlton, ... I am aware that other such packages exist. I just don't think we should support them if they can't be bootstrapped properly. and adding new features / packages to the GNU toolchain requires manual interaction for glibc/gcc uploads, which can't be done with source only uploads. For the benefit of other readers on this list, the unstated context here is that there's a non-zero incidence of having to manually re-bootstrap compilers in the Ubuntu archive. The intermediate binaries used in this bootstrapping are discarded and never published to Ubuntu in order to conserve the no binary uploads rule, and this does impose some non-negligible overhead on the folks who tend these toolchain packages, due to the highly constrained build environment they have to work in. So yes, this is something that should be accounted for if Debian moves to a model where binary uploads are discarded and rebuilt. However, I suspect that for all the sensible cases, the proposal to add staged-build metadata (for bootstrapping circular build-dependencies on new ports) would be sufficient to make this problem go away too. But I'm a lot less concerned about the kinds of self-build-depending packages whose names are frequently taken in vain (raise your hand if you've ever tried to fix an RC bug in mlton). -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Re: Discarding uploaded binary packages
On Thu, 18 Oct 2012, Michael Gilbert mgilb...@debian.org wrote: Maybe someone would be interested in writing a lintian check for these issues? Something a bit more advanced than this $ strings /sbin/dhclient | grep ^PATH PATH=/home/zero79/source/git/isc-dhcp/debian/tmp/usr/sbin:/sbin:/bin:/usr/s bin:/usr/bin would have caught the issue (advanced aspect being that reasonable paths are ok'd). Having a PATH set isn't a problem if it's set to something like /sbin:/bin or something else restrictive. The PATH isn't the problem here anyway it's the use of a directory under /home which would potentially be a problem if it's used for configuration files or data files. We could have a lintian warning for any occurance of the string /home in a packaged file and have error conditions for /build and the current value of $HOME for the account running lintian. -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210181320.44098.russ...@coker.com.au
Re: Discarding uploaded binary packages
On Thu, Oct 18, 2012 at 10:20 AM, Russell Coker wrote: We could have a lintian warning for any occurance of the string /home in a packaged file and have error conditions for /build and the current value of $HOME for the account running lintian. Based on a quick grep of /usr/bin on my laptop, this is going to have a fair number of false positives; commented out paths, paths in POD documentation, URLs, paths outside of /home that include /home in their name somewhere. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caktje6glutn+ec2hmnoekkpeslf85hgbuyqmqav--mgd0yp...@mail.gmail.com
Re: Discarding uploaded binary packages
On Thu, 18 Oct 2012, Paul Wise p...@debian.org wrote: On Thu, Oct 18, 2012 at 10:20 AM, Russell Coker wrote: We could have a lintian warning for any occurance of the string /home in a packaged file and have error conditions for /build and the current value of $HOME for the account running lintian. Based on a quick grep of /usr/bin on my laptop, this is going to have a fair number of false positives; commented out paths, paths in POD documentation, URLs, paths outside of /home that include /home in their name somewhere. For a warning it doesn't matter much if we have a few false-positives. For the error conditions that I suggest for /build and $HOME I find it difficult to imagine false-positives except the case where a DD uses their own home directory as an example in help text, and that can't be a good practice anyway. -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210181337.00861.russ...@coker.com.au
Re: Discarding uploaded binary packages
On Thu, Oct 18, 2012 at 10:37 AM, Russell Coker wrote: For a warning it doesn't matter much if we have a few false-positives. For the error conditions that I suggest for /build and $HOME I find it difficult to imagine false-positives except the case where a DD uses their own home directory as an example in help text, and that can't be a good practice anyway. Since you don't seem to believe me, please try this command yourself grep -r /home /usr/bin/ /usr/games There are lots of false positives. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAKTje6FYfzcbAFwkAaQ_MKyrwSQkPp_=zib5YcMrMBC=4yb...@mail.gmail.com
Re: Discarding uploaded binary packages
also sprach olivier sallou olivier.sal...@gmail.com [2012.10.16.0752 +0200]: This is my opinion but I admit I have not followed previous discussions on the subject http://lists.debian.org/debian-security/2004/09/msg00014.html We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduckhttp://vcs-pkg.org `- Debian - when you have better things to do than fixing systems #define emacs eighty megabytes and constantly swapping. digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: Discarding uploaded binary packages
Hi, On Dienstag, 16. Oktober 2012, martin f krafft wrote: We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. your point being? (And obviously, +1 to the suggestion at hand.) cheers, Holger -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210160945.58694.hol...@layer-acht.org
Re: Discarding uploaded binary packages
On Mon, Oct 15, 2012 at 10:59:27PM -0400, Michael Gilbert wrote: I know this subject has been discussed on and off in the past, but there's new evidence that it's simply the right thing to do. Nice, although it's not new evidence we need :). The state of last discussion on the matter is that there is consensus in going ahead, here are a couple of pointers: - https://lists.debian.org/debian-devel/2011/04/msg00840.html - https://lists.debian.org/debian-devel-announce/2011/06/msg0.html So, are we ready to do this? Sure, where are your dak patches implementing it? :) Note: I don't want to smash down the discussion with a lame show me the code argument. But I do want to avoid the impression that we're unable to decide when I fact, in this case, we are and we did. But that's, unfortunately, not enough to make appear out of thin area the code implementing the decision. Cheers. -- Stefano Zacchiroli . . . . . . . z...@upsilon.cc . . . . o . . . o . o Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o Debian Project Leader . . . . . . @zack on identi.ca . . o o o . . . o . « the first rule of tautology club is the first rule of tautology club » signature.asc Description: Digital signature
Re: Discarding uploaded binary packages
also sprach Holger Levsen hol...@layer-acht.org [2012.10.16.0945 +0200]: We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. your point being? That our users don't seem to care, and that probably is why we haven't done anything about it. -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduckhttp://vcs-pkg.org `- Debian - when you have better things to do than fixing systems there's someone in my head but it's not me. -- pink floyd, the dark side of the moon, 1972 digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: Discarding uploaded binary packages
* martin f krafft madd...@debian.org, 2012-10-16, 08:21: This is my opinion but I admit I have not followed previous discussions on the subject http://lists.debian.org/debian-security/2004/09/msg00014.html We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. What makes a buildd more secure than a machine of J. Random Developer? I'm honestly curious. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121016110123.ga6...@jwilk.net
Re: Discarding uploaded binary packages
On Tue, 16 Oct 2012, Jakub Wilk jw...@debian.org wrote: * martin f krafft madd...@debian.org, 2012-10-16, 08:21: This is my opinion but I admit I have not followed previous discussions on the subject http://lists.debian.org/debian-security/2004/09/msg00014.html We have not cared enough for almost 20 years that 9 out of 10 binary packages in use (i386 until 2005, amd64 since then) are built on machines that are individually maintained according to widely varying security standards to do anything about it, AFAICT. What makes a buildd more secure than a machine of J. Random Developer? I'm honestly curious. I believe that the sysadmin skill of the people who run the build servers is greater than that of most DDs. The Debian servers are run in relatively secure environments as opposed to DD workstations being laptops that are often stored in hotel rooms and other fairly insecure environments. There are a fairly small number of Debian servers. So even if the probability of system compromise for a Debian server was the same as for a laptop owned by a random DD the fact that DD workstations outnumber Debian servers by at least 200:1 makes them more of a risk. One final think to note is that if an attacker manages to compromise a Debian server they will probably start by compromising the workstation used by a random DD. So I don't think that the case of a compromised server with thousands of secure workstations is a case to prepare for, but the case of compromised workstation(s) before a compromised server is one to prepare for. -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210162300.31295.russ...@coker.com.au
Re: Discarding uploaded binary packages
Hi, On 16.10.2012 14:00, Russell Coker wrote: There are a fairly small number of Debian servers. So even if the probability of system compromise for a Debian server was the same as for a laptop owned by a random DD the fact that DD workstations outnumber Debian servers by at least 200:1 makes them more of a risk. Not a strong argument. The impact of a compromise of a buildd [or J Random Developer's machine running the buildd] is substantially higher given the compromise would affect 30k source packages, as opposed to [1, $whatever_gregoa_maintains_today[ of packages distributed amongst 950+ individual machines. Moreover, if you go down that path, you do not win anything of the state being, as an attacker can still make a sourceful upload which enters the archive unaudited as well. Not to say, throwing away binary packages would be a bad idea though. We just need someone to care enough to implement missing bits and find a way how to deal with arch:all. -- with kind regards, Arno Töll IRC: daemonkeeper on Freenode/OFTC GnuPG Key-ID: 0x9D80F36D signature.asc Description: OpenPGP digital signature
Re: Discarding uploaded binary packages
On Tue, 16 Oct 2012, Arno Töll a...@debian.org wrote: On 16.10.2012 14:00, Russell Coker wrote: There are a fairly small number of Debian servers. So even if the probability of system compromise for a Debian server was the same as for a laptop owned by a random DD the fact that DD workstations outnumber Debian servers by at least 200:1 makes them more of a risk. Not a strong argument. The impact of a compromise of a buildd [or J Random Developer's machine running the buildd] is substantially higher given the compromise would affect 30k source packages, as opposed to [1, $whatever_gregoa_maintains_today[ of packages distributed amongst 950+ individual machines. An attacker wouldn't want to compromise all 30K packages, that would increase the risk of detection without increasing the benefit. They would want to compromise a very small number of relatively common packages. For example they could compromise Apache or a library it depends on, some library that is used by both KDE and GNOME, dash, bash, or a library used by shells. Compromising 100% of systems with a high probability of detection would be a lot less useful than compromising 50% with a low probability of detection. So compromising the workstation of a random DD who packages only some programs that aren't particularly popular wouldn't be a very effective attack in the short term. But using such an attack to target other DDs would be easier than using it to target build servers. For example an attacker who compromised a single program could file a bug report saying that it gave a SEGV when run with a new version of a particular library and have a good chance that the maintainer of the library would do a test... Moreover, if you go down that path, you do not win anything of the state being, as an attacker can still make a sourceful upload which enters the archive unaudited as well. The advantage for the good people is that source can be audited. While it is difficult to audit source it's possible without huge effort, particularly as the changes made in the process of Debian packaging are generally small when compared to the upstream source. Currently we have no certainty of the version of the libraries and compiler used for building a package. So if a package has a different binary when it's rebuilt that isn't evidence of attack. Determining what parts of the binary change might be due to actual differences in operation of the program as opposed to the same logic with different compilation is going to take some work. Comparing two different versions of a Debian package at the source level to determine if the changes appear to match the changelog shouldn't be THAT difficult. Comparing two different binaries is going to be a lot harder. -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210170023.43896.russ...@coker.com.au
Re: Discarding uploaded binary packages
]] Jakub Wilk What makes a buildd more secure than a machine of J. Random Developer? It has a smaller attack surface due to few users, firewalls, few packages installed, nobody using it for browsing the web, etc. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87txtuplni@xoog.err.no
Re: Discarding uploaded binary packages
Tollef Fog Heen tfh...@err.no writes: ]] Jakub Wilk What makes a buildd more secure than a machine of J. Random Developer? It has a smaller attack surface due to few users, firewalls, few packages installed, nobody using it for browsing the web, etc. We seem to be forgetting, that the real advantage of source-only uploads isn't necessarily security, but a controlled build environment on *all* architectures. There is sbuild, pbuilder and the rest, but there are still packages uploaded that are built in an unclean environment, thereby becoming broken in various interesting ways. Nevermind security, whether N buildds are more secure than 200N random systems scattered around the world - a controlled environment makes source-only uploads a win, without doubt. (Of course, there's the corner case of bootstrapping things, but that's a corner case, and should be handled as such, not as the norm.) -- |8] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87r4oy335m.fsf@algernon.balabit
Re: Discarding uploaded binary packages
On Oct 16, 2012, at 03:54 PM, Tollef Fog Heen wrote: ]] Jakub Wilk What makes a buildd more secure than a machine of J. Random Developer? It has a smaller attack surface due to few users, firewalls, few packages installed, nobody using it for browsing the web, etc. I also think allowing source-only uploads makes for easier contributions, and thus hopefully *more* contributions. Cheers, -Barry signature.asc Description: PGP signature
Re: Discarding uploaded binary packages
On Wed, 17 Oct 2012, Barry Warsaw ba...@python.org wrote: I also think allowing source-only uploads makes for easier contributions, and thus hopefully more contributions. Why would it be easier? Surely we still want people to build packages first to ensure that we don't needlessly get FTBFS bugs. -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201210171136.24758.russ...@coker.com.au
Re: Discarding uploaded binary packages
On 17/10/2012 08:36, Russell Coker wrote: On Wed, 17 Oct 2012, Barry Warsaw ba...@python.org wrote: I also think allowing source-only uploads makes for easier contributions, and thus hopefully more contributions. Why would it be easier? Surely we still want people to build packages first to ensure that we don't needlessly get FTBFS bugs. Because binary packages are big, and uploading them reliably from a region with crappy internet access sucks, especially when trying to upload them over SFTP. Honestly, if we're not going to be using these, why upload them? It's a pointless waste of bandwidth. -- Kind regards, Loong Jin signature.asc Description: OpenPGP digital signature
Discarding uploaded binary packages
I know this subject has been discussed on and off in the past, but there's new evidence that it's simply the right thing to do. Due to changes in upstream's build system, isc-dhcp recently started including build system paths in dhclient's search path. This got a security identifier, and we've fixed it, but really the only architecture affected was the one I built and uploaded. All of the packages built on the buildds were not since the PATH was something in /build vs. a home dir. Also, Ubuntu was not affected since all of their packages go through their buildds. Details in: http://bugs.debian.org/690532 Anyway, all of these build system path sanitization issues can be eliminated by using the buildds for all architectures, since paths will start with at least /build that requires root-level action to exist on users' systems. So, are we ready to do this? Best wishes, Mike -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANTw=MNag1=MZG3GiUCyGXsVRBjDKc62_WNLYHP5juXo=_4...@mail.gmail.com
Re: Discarding uploaded binary packages
Le 16 oct. 2012 04:59, Michael Gilbert mgilb...@debian.org a écrit : I know this subject has been discussed on and off in the past, but there's new evidence that it's simply the right thing to do. Due to changes in upstream's build system, isc-dhcp recently started including build system paths in dhclient's search path. This got a security identifier, and we've fixed it, but really the only architecture affected was the one I built and uploaded. All of the packages built on the buildds were not since the PATH was something in /build vs. a home dir. Also, Ubuntu was not affected since all of their packages go through their buildds. Details in: http://bugs.debian.org/690532 Anyway, all of these build system path sanitization issues can be eliminated by using the buildds for all architectures, since paths will start with at least /build that requires root-level action to exist on users' systems. So, are we ready to do this? +1 ;-) I agree with this. We face some cases where delivered binary have issues related to build context. Though most should be discovered by maintainer testing before upload, it would be more valid with a complete rebuild. This is my opinion but I admit I have not followed previous discussions on the subject Best wishes, Mike -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANTw=MNag1=MZG3GiUCyGXsVRBjDKc62_WNLYHP5juXo=_4...@mail.gmail.com
