Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
Following up from the conference last week - yes we're using srebuild from debian-rebuilder-setup. Thanks, Matt Bearup Software Developer – CEH, CISSP, GCUX Microsoft Azure Compute Linux -Original Message- From: Santiago Torres Arias Sent: Monday, October 7, 2019 11:49 AM To: Matt Bearup Cc: Steven Chamberlain ; kpcyrd ; Holger Levsen ; 774...@bugs.debian.org; Reproducible Builds discussion list ; Xavier Subject: Re: #774415: devscripts: please add the srebuild wrapper for reproducible builds Curious, was the srebuild the one as featured in the debian-rebuilder-setup[1] repository or the upstream one? I don't think we've faced much build issues on our side... Cheers! -Santiago [1] https://salsa.debian.org/reproducible-builds/debian-rebuilder-setup On Mon, Oct 07, 2019 at 06:03:08PM +, Matt Bearup wrote: > I have to second the issues with srebuild. We invested a lot of time to > utilize this tool in our rebuilds but faced consistent build failures. > The best explanation I could find was that the snapshots referred to in the > .buildinfo files had expired. That's not conclusive (the output wasn't clear > on the cause of failure) nor is expired repo metadata the fault of srebuild > per se. But the issue was nonetheless a blocker. > PBuilder is the most consistent build tool we've seen thus far, will have to > investigate debrebuild as well. > > Matt Bearup > Software Developer – CEH, CISSP, GCUX > Microsoft Azure Compute Linux
Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
Curious, was the srebuild the one as featured in the debian-rebuilder-setup[1] repository or the upstream one? I don't think we've faced much build issues on our side... Cheers! -Santiago [1] https://salsa.debian.org/reproducible-builds/debian-rebuilder-setup On Mon, Oct 07, 2019 at 06:03:08PM +, Matt Bearup wrote: > I have to second the issues with srebuild. We invested a lot of time to > utilize this tool in our rebuilds but faced consistent build failures. > The best explanation I could find was that the snapshots referred to in the > .buildinfo files had expired. That's not conclusive (the output wasn't clear > on the cause of failure) nor is expired repo metadata the fault of srebuild > per se. But the issue was nonetheless a blocker. > PBuilder is the most consistent build tool we've seen thus far, will have to > investigate debrebuild as well. > > Matt Bearup > Software Developer – CEH, CISSP, GCUX > Microsoft Azure Compute Linux signature.asc Description: PGP signature
Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
I have to second the issues with srebuild. We invested a lot of time to utilize this tool in our rebuilds but faced consistent build failures. The best explanation I could find was that the snapshots referred to in the .buildinfo files had expired. That's not conclusive (the output wasn't clear on the cause of failure) nor is expired repo metadata the fault of srebuild per se. But the issue was nonetheless a blocker. PBuilder is the most consistent build tool we've seen thus far, will have to investigate debrebuild as well. Matt Bearup Software Developer – CEH, CISSP, GCUX Microsoft Azure Compute Linux -Original Message- From: Reproducible-builds On Behalf Of Johannes Schauer Sent: Monday, October 7, 2019 8:10 AM To: Santiago Torres Arias Cc: Steven Chamberlain ; kpcyrd ; Holger Levsen ; 774...@bugs.debian.org; Reproducible Builds discussion list ; Xavier Subject: Re: #774415: devscripts: please add the srebuild wrapper for reproducible builds Hi, Quoting Santiago Torres Arias (2019-10-07 16:58:58) > On Mon, Oct 07, 2019 at 02:49:27PM +0200, Johannes Schauer wrote: > > The srebuild script suffers from many problems (see above). I would > > advice against using it in favour of debrebuild. If you want > > something that works "most or even half the time" then I think that > > debrebuild is what you want. > > Feel free to ask me if you have any questions about the script. > On the rebuilder side, would this work as a drop-in replacement? how > does it handle fetching dependencies from the debian archive and such? it uses snapshot.debian.org to find the snapshot with the right packages and then crafts either an sbuild command line that does the right thing or outputs manual apt-get command that will install what is necessary. Pbuilder support could be added but I do not understand enough pbuilder to know how to do that. Thanks! cheers, josch
Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
Hello Everyone, On Mon, Oct 07, 2019 at 02:49:27PM +0200, Johannes Schauer wrote: > Copious snipping performed here > The srebuild script suffers from many problems (see above). I would advice > against using it in favour of debrebuild. If you want something that works > "most or even half the time" then I think that debrebuild is what you want. > Feel free to ask me if you have any questions about the script. > On the rebuilder side, would this work as a drop-in replacement? how does it handle fetching dependencies from the debian archive and such? Cheers! -Santiago signature.asc Description: PGP signature
Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
Hi, Quoting Santiago Torres Arias (2019-10-07 16:58:58) > On Mon, Oct 07, 2019 at 02:49:27PM +0200, Johannes Schauer wrote: > > The srebuild script suffers from many problems (see above). I would advice > > against using it in favour of debrebuild. If you want something that works > > "most or even half the time" then I think that debrebuild is what you want. > > Feel free to ask me if you have any questions about the script. > On the rebuilder side, would this work as a drop-in replacement? how > does it handle fetching dependencies from the debian archive and such? it uses snapshot.debian.org to find the snapshot with the right packages and then crafts either an sbuild command line that does the right thing or outputs manual apt-get command that will install what is necessary. Pbuilder support could be added but I do not understand enough pbuilder to know how to do that. Thanks! cheers, josch signature.asc Description: signature
Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
Hi all, Quoting Holger Levsen (2019-10-06 21:02:38) > so I thought I'd be bold and add the srebuild wrapper to src:devscripts in > git this weekend... thanks for looking into this! :) > So I re-read https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774415 > rather completly and noticed, that > > - the branch devscripts-srebuild from https://salsa.debian.org/yadd/devscripts > for a long time used the 2014 srebuild script from josch and was only > 'recently' based on the 2016 debrebuild script from josch. > (The last 4 commits on this branch have all this history and thus are > easy to grasp.) Indeed. Why is checksum verification disabled in the last commit? > - the NYU rebuilders OTOH use a by now quite modified version of the > 2014 srebuild script (with support for in-toto etc), see > > https://salsa.debian.org/reproducible-builds/debian-rebuilder-setup/blob/master/builder/srebuild The original srebuild script suffered from many conceptional problems. I'd just rebase their changes onto the new codebase. For a list of flaws in the original script see: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774415#56 > - the authoritive source/git repo for josch script(s) is the #774415 bug > report? Or yadd's repo? ;) My latest script is what you already saw in #774415. > - the 2016 debrebuild script doesn't do a rebuild by itself but produces > a command which is to be run with sudo, so we need another wrapper > here. Not necessarily. See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774415#255 > - there is also > https://salsa.debian.org/reproducible-builds/attic/reprobuild/blob/master/repro-build.pl > from Steven Chamberlain... Steven, which advantages does that script have over debrebuild.pl? > - for the sake of presenting a complete picture of this discussion I want to > state that I also thought about packaging $name (srebuild, debrebuild, > repro-build, whatever) as a seperate package, not part of devscripts. I've > decided, at least for now, to first try to make it usable as part of the > devscripts packages. Maybe however we want more configurability (like the > in-toto support or other stuff which was added to NYU's srebuild fork) and > this wont work in the long term. Is it not possible to have both and still have the script in devscripts? > - I think I'd like "something working most or even half the time" installable > in Debian unstable by the end of the month. This is long overdue. (tm) (Only > halfworking would be fine (for a start) for me cause there are quite some > special cases, like binNMUs or support for unclean build envs or whatever.) Since the information about both is captured in the .buildinfo file, debrebuild.pl should be able to handle those. Do you have a counter example? > - I think I want(ed) to package the debrebuild script, as this is josch's > reimplementation of the same problem. And I thought NYU had some patches on > top of this and I was thinking to sort out this fork later (eg by making some > of their features optional), but now I've seen that they forked the old > srebuild script and I'm unsure what to do. > > Comments, suggestions or any other feedback much welcome! The srebuild script suffers from many problems (see above). I would advice against using it in favour of debrebuild. If you want something that works "most or even half the time" then I think that debrebuild is what you want. Feel free to ask me if you have any questions about the script. Thanks! cheers, josch signature.asc Description: signature
Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
hi, so I thought I'd be bold and add the srebuild wrapper to src:devscripts in git this weekend... So I re-read https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774415 rather completly and noticed, that - the branch devscripts-srebuild from https://salsa.debian.org/yadd/devscripts for a long time used the 2014 srebuild script from josch and was only 'recently' based on the 2016 debrebuild script from josch. (The last 4 commits on this branch have all this history and thus are easy to grasp.) - the NYU rebuilders OTOH use a by now quite modified version of the 2014 srebuild script (with support for in-toto etc), see https://salsa.debian.org/reproducible-builds/debian-rebuilder-setup/blob/master/builder/srebuild - the authoritive source/git repo for josch script(s) is the #774415 bug report? Or yadd's repo? ;) - the 2016 debrebuild script doesn't do a rebuild by itself but produces a command which is to be run with sudo, so we need another wrapper here. - there is also https://salsa.debian.org/reproducible-builds/attic/reprobuild/blob/master/repro-build.pl from Steven Chamberlain... - for the sake of presenting a complete picture of this discussion I want to state that I also thought about packaging $name (srebuild, debrebuild, repro-build, whatever) as a seperate package, not part of devscripts. I've decided, at least for now, to first try to make it usable as part of the devscripts packages. Maybe however we want more configurability (like the in-toto support or other stuff which was added to NYU's srebuild fork) and this wont work in the long term. - I think I'd like "something working most or even half the time" installable in Debian unstable by the end of the month. This is long overdue. (tm) (Only halfworking would be fine (for a start) for me cause there are quite some special cases, like binNMUs or support for unclean build envs or whatever.) - I think I want(ed) to package the debrebuild script, as this is josch's reimplementation of the same problem. And I thought NYU had some patches on top of this and I was thinking to sort out this fork later (eg by making some of their features optional), but now I've seen that they forked the old srebuild script and I'm unsure what to do. Comments, suggestions or any other feedback much welcome! -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Bug#774415: #774415: devscripts: please add the srebuild wrapper for reproducible builds
control: reassign -1 devscripts control: retitle -1 devscripts: please add the srebuild wrapper for reproducible builds thanks On Sat, Jun 09, 2018 at 10:33:16PM +0200, Johannes Schauer wrote: > Quoting Holger Levsen (2018-06-09 22:12:33) > > As it sounds, I now believe this script would better live in src:devscripts > > and as such I would like to reassign #774415 to devscripts - or do you see > > any issue with that? > I see no issues with that from my side. ok :) thanks! -- cheers, Holger signature.asc Description: PGP signature
