Re: Revised patch: seeking seconds

2017-08-12 Thread Johannes Schauer
Hi, Quoting Sean Whitton (2017-08-13 03:23:14) > +Reproducibility > +--- > + > +Packages should build reproducibly, which for the purposes of this > +document [#]_ means that given > + > +- a version of a source package unpacked at a given path; > +- a set of versions of installed buil

Re: Bug#844431: Reproducibility in Policy

2017-08-12 Thread Johannes Schauer
Hi, Quoting Russ Allbery (2017-08-12 09:57:44) > I think we need to add all environment variables starting with DEB_* to > the prerequisites. If you set DEB_BUILD_OPTIONS=nostrip or > DEB_BUILD_MAINT_OPTIONS=hardening=all, you'll definitely get a different > package, for instance. > > I feel lik

sbuild: please add the srebuild sbuild wrapper to reproduce builds

2017-08-01 Thread Johannes Schauer
Hi, this bug has been listed in the "NMU campaign" email on d-devel. But I wonder how it ended up there. There are still open problems that are not answered, the fix for this is still blocked by another bug (#802241) which is not on the NMU list, and the code I wrote to fix the problem (debrebuild

Re: Reproducing packages in the archive

2017-02-01 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2017-02-01 11:56:03) > On Tue, Jan 31, 2017 at 07:35:35PM +, Daniel Shahaf wrote: > > That is: if I run dpkg-buildpackage and post the .deb and .buildinfo > > somewhere, do we have a script that takes the .buildinfo as input and > > reproduces the .deb? > > AIUI sb

Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups

2017-01-30 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2017-01-30 14:58:33) > On Mon, Jan 30, 2017 at 02:47:45PM +0100, Johannes Schauer wrote: > > > b.) if thats the case, shall we scan all packages in sid for files which > > > have the same timestamp+filename but different checksums and ask for &

Re: Some Debian package upgrades are corrupting rsync "quick check" backups

2017-01-30 Thread Johannes Schauer
Hi Henrik & Holger, sbuild maintainer here. Quoting Holger Levsen (2017-01-30 14:25:51) > On Mon, Jan 30, 2017 at 01:10:12PM +0100, Mattia Rizzolo wrote: > > > Would reproducible-builds@lists.alioth.debian.org be the correct mailing > > > list to discuss this? > > the debian-buildd list or a bu

Re: Bug#774415: From srebuild sbuild-wrapper to debrebuild

2016-12-20 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2016-12-19 10:35:53) > On Mon, Dec 19, 2016 at 10:02:58AM +0100, Johannes Schauer wrote: > > Other ways to solve this problem include: > > - only accept .buildinfo files that include the .dsc filename and checksum > > - accept .changes files th

Re: Bug#774415: From srebuild sbuild-wrapper to debrebuild

2016-12-20 Thread Johannes Schauer
Hi, Quoting Johannes Schauer (2016-12-20 13:49:27) > Currently, a buildinfo file does not specify which artifacts were supposed to > be built (source,any,all). as guillem points out to me on #debian-dpkg, the Architecture field lists exactly that. It will contain "source" if th

Re: Bug#774415: From srebuild sbuild-wrapper to debrebuild

2016-12-20 Thread Johannes Schauer
Hi, Quoting Johannes Schauer (2016-12-19 10:02:58) > I also came up with another question: and as I'm implementing this, here yet another: Currently, a buildinfo file does not specify which artifacts were supposed to be built (source,any,all). What should happen if the buildinfo file wa

Re: Bug#774415: From srebuild sbuild-wrapper to debrebuild

2016-12-19 Thread Johannes Schauer
Hi, Quoting HW42 (2016-12-19 07:37:00) > So you (at least josch and ntyi) seem to prefer to have the user facing part > in sbuild/pbuilder and the common functionality in some kind of library. How > should the "library" interface look like for sbuild? pbuilder is written in bash, so a plain-text

Re: Bug#774415: From srebuild sbuild-wrapper to debrebuild

2016-12-17 Thread Johannes Schauer
Hi, Quoting Niko Tyni (2016-12-17 13:40:32) > On Thu, Dec 15, 2016 at 02:21:49PM +0100, Johannes Schauer wrote: > > Quoting Niko Tyni (2016-12-15 14:04:19) > > > In general, I like the concept of sbuild/pbuilder accepting .buildinfo > > > files > > > as

Re: Bug#774415: From srebuild sbuild-wrapper to debrebuild

2016-12-15 Thread Johannes Schauer
Hi, Quoting Niko Tyni (2016-12-15 14:04:19) > > But then on IRC, HW42 suggested to approach this problem differently. > > Instead of integrating the functionality of figuring out the right > > repositories to reproduce the contents of a buildinfo file into sbuild, > > write a tool that can drive a

Bug#847805: reprotest: document/support simple reproducibility test with sbuild

2016-12-12 Thread Johannes Schauer
Hi, Quoting Sean Whitton (2016-12-12 16:44:54) > Thank you for your replies. sbuild is definitely sufficient, it's just a bit > of a drag -- you have to rename the .changes to save the checksums, and then > run sbuild a second time, and compare. I was going to write a shell script > to do this,

Bug#847805: reprotest: document/support simple reproducibility test with sbuild

2016-12-12 Thread Johannes Schauer
Hi Sean, Quoting Sean Whitton (2016-12-12 00:44:05) > On Sun, Dec 11, 2016 at 03:12:57PM -0700, Sean Whitton wrote: > > I have sbuild properly set up on my machine, and I want to use it to > > test package reproducibility. Something like this, where PWD is an > > unpacked source package: > > > >

Re: From srebuild sbuild-wrapper to debrebuild

2016-11-16 Thread Johannes Schauer
Hi, Quoting HW42 (2016-11-17 05:10:00) > After discussing this in the irc meeting yesterday I propose that: > > - we keep it as a separate tool. > - put it in a git repo under >https://anonscm.debian.org/git/reproducible/ > - We have more than enough DDs who are willing to sponsor uploads,

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread Johannes Schauer
Hi, Quoting Daniel Kahn Gillmor (2016-11-13 21:44:22) > It is definitely not what most of us initially expected, but it is > actually what we want. > > i look at it this way: > > * Ideally, the generated binary packages are reproducible *even when >the build environment changes*. For examp

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread Johannes Schauer
Hi, Quoting Chris Lamb (2016-11-13 12:25:07) > > move the build date inside the .buildinfofile in a Build-Date field or > > similar > Hrm. Are we sure about this? My gut tells me that the external definition of > the build should not include the Build-Date. (At the very least, this is just > a dup

Re: From srebuild sbuild-wrapper to debrebuild

2016-11-10 Thread Johannes Schauer
Hi, On Thu, 10 Nov 2016 05:54:13 -0200 Johannes Schauer wrote: > On Tue, 02 Aug 2016 22:49:00 +0200 Johannes Schauer wrote: > > But then on IRC, HW42 suggested to approach this problem differently. > > Instead of integrating the functionality of figuring out the right >

Re: misleading timestamps in binnmus

2016-11-10 Thread Johannes Schauer
Hi, Quoting Niko Tyni (2016-11-10 10:01:38) > On Thu, Nov 10, 2016 at 10:34:33AM +, Holger Levsen wrote: > > can someone please point at a real life/archive example of such a file? > > (a binNMU .changes file with Binary-Only-Changes field…) > > That's in the .buildinfo file (not .changes), a

Re: misleading timestamps in binnmus

2016-11-10 Thread Johannes Schauer
Hi, Quoiting Holger Levsen (2016-11-10 07:48:33) > On Thu, Nov 10, 2016 at 10:38:45AM +0100, Emilio Pozuelo Monfort wrote: > > > I see. And this changelog.$arch is neither part of the source package, > > > the .changes file nor the .buildinfo file, it's just included in the > > > binary packages?

Re: From srebuild sbuild-wrapper to debrebuild

2016-11-09 Thread Johannes Schauer
Hi all, On Tue, 02 Aug 2016 22:49:00 +0200 Johannes Schauer wrote: > I was thinking about this issue again and thought that instead of creating a > wrapper for sbuild which then uses a chroot-setup hook to install the > dependencies, what I should instead do is to let sbuild itse

Re: sbuild should use build date as binnmu changelog date

2016-11-09 Thread Johannes Schauer
Hi Ian and reproducible-builds folks, On Wed, 9 Nov 2016 12:03:48 + Ian Jackson wrote: > Currently, when adding a changelog stanza for a binnmu (or when appending to > the version number is requested for another reason), sbuild uses the existing > source changelog timestamp when inventing th

Re: Build dependency list

2016-11-02 Thread Johannes Schauer
Hi mike, welcome to the bootstrapping problem! Quoting james michael dupont (2016-11-02 12:12:57) > Here is a question, if you were to compile an entire Debian system from > scratch with no root access, and first get all the developer packages > compiled and then use those newly built tools to bu

Bug#838188: ocaml: temporary preprocessed file paths make the ocaml compiler produce unreproducible output

2016-09-18 Thread Johannes Schauer
Source: ocaml Version: 4.02.3-7.1 Severity: wishlist Tags: patch User: reproducible-builds@lists.alioth.debian.org Usertags: toolchain randomness Hi, currently, ocaml embeds the file paths of temporary files that a preprocessor created into the debug output. This makes several source packages in

[Reproducible-builds] Bug#835165: pdf2htmlex: please make the build reproducible (buildpath)

2016-08-23 Thread Johannes Schauer
Package: pdf2htmlex Version: 0.14.6+ds-2+b1 Severity: normal User: reproducible-builds@lists.alioth.debian.org Usertags: timestamps fileordering Control: forwarded -1 https://github.com/coolwanglu/pdf2htmlEX/issues/668 Hi, the binaries produced by src:pdf2htmlex contain the build path due to call

Re: [Reproducible-builds] Moving towards buildinfo on the archive network

2016-08-03 Thread Johannes Schauer
Hi Jonathan, Quoting Jonathan McDowell (2016-07-25 22:29:39) > Having been impressed by the current status of reproducible builds and > the fact it looks like we're close to having the important pieces in > Debian proper, I have started to have a look at how I could help out > with this bug. I've

[Reproducible-builds] From srebuild sbuild-wrapper to debrebuild

2016-08-02 Thread Johannes Schauer
Hi, On Mon, 09 May 2016 21:07:40 +0200 Johannes Schauer wrote: > The main disadvantage of the current srebuild implementation is, that it will > only make use of a single snapshot.d.o timestamp. This makes it impossible to > reproduce situations where packages are not built in a clean c

Re: [Reproducible-builds] [buildd-tools-devel] Bug#825991: Bug#825991: sbuild: /etc/sbuild/sbuild.conf leaks the user home path

2016-06-01 Thread Johannes Schauer
Quoting Mattia Rizzolo (2016-06-01 11:55:31) > On Wed, Jun 01, 2016 at 11:08:27AM +0200, Johannes Schauer wrote: > > Though I am surprised that the reproducible builds machinery didn't catch > > this > > at all. It seems that sbuild is still marked as re

Re: [Reproducible-builds] [buildd-tools-devel] Bug#825991: sbuild: /etc/sbuild/sbuild.conf leaks the user home path

2016-06-01 Thread Johannes Schauer
Hi Quoting Aurelien Jarno (2016-06-01 09:53:37) > The default sbuild.conf shipped with the sbuild package is generated > using the "sbuild-dumpconfig sbuild config" command. This causes the > stats_dir entry to contain the home path of the user who has build the > package: > > | # STATS_DIR > | #

[Reproducible-builds] repro-test (was: Re: Projects and mentors wanted for our participation in upcoming outreach programmes)

2016-02-18 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2016-02-17 11:57:03) > And then I was thinking to add another project, "develop reprotest tool", > what > other ideas do you have? I have some more random thoughts about this: - unfortunately I failed to take part in that discussion in Athens but I assume you are

Re: [Reproducible-builds] Bug#138409: Bug#138409: Bug#138409: Bug#138409: dpkg-dev: please add support for .buildinfo files

2016-02-04 Thread Johannes Schauer
Hi, Quoting Jérémy Bobbio (2016-02-04 12:23:05) > We have to educate them about .buildinfo file and what the various fields > mean. We have to aim at field names that are as unambigious as possible to > avoid laying traps on users. > > For the particular case of “Installed-Transitive-Build-Depend

Re: [Reproducible-builds] Bug#138409: Bug#138409: Bug#138409: Bug#138409: dpkg-dev: please add support for .buildinfo files

2016-02-04 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2016-02-04 11:31:32) > (AFAIK transitive build-depends are all possible build depends, no, that would be the build dependency closure ;) > so if a package build depends on python2 || python3 both python versions will > be part of the transitive build depends. (Is that

Re: [Reproducible-builds] Bug#138409: Bug#138409: Bug#138409: dpkg-dev: please add support for .buildinfo files

2016-02-04 Thread Johannes Schauer
Hi, Quoting Guillem Jover (2016-02-04 09:44:13) > On Sun, 2016-01-31 at 14:43:08 +0100, Jérémy Bobbio wrote: > > and “Installed-Build-Depends” for the list of packages? > > I asked for more suggestions on #debian-dpkg, and Johannes Schauer > suggested Transitive-Build-Depends,

Re: [Reproducible-builds] Building packages in the *past* (!!)

2015-09-30 Thread Johannes Schauer
Hi, Quoting Chris Lamb (2015-09-30 11:57:20) > > There is a minimum of sanity that we should assume on the autobuilders, > > Agree in principle.. > > > namely, that packages are built on a date which is later than the one > > in the last changelog entry. > > .. but why should this matter? In fa

Re: [Reproducible-builds] .buildinfo should contain source hashes (as well as binary hashes)

2015-09-20 Thread Johannes Schauer
Hi, Quoting Ximin Luo (2015-09-20 18:49:16) > Currently, to run a DDC test, we would have to read the buildinfo file, find > the hashes of the binary build-deps, lookup the source packages that > corresponds to these hashes, find a different binary build-deps for these > hashes, and run our DDC-ch

[Reproducible-builds] binNMU detection for generating Changes field in buildinfo (was: Re: binNMU or reproducible builds (choose only one))

2015-09-20 Thread Johannes Schauer
Hi Lunar, Quoting Jérémy Bobbio (2015-09-19 16:46:36) > Simon McVittie: > > BinNMUs don't upload any source at all. They instruct the autobuilders > > to run sbuild with some non-default options ("sbuild --binNMU=2 > > --make-binNMU "Rebuild with foo 3" foo_1.2-3" will result in > > foo_1.2-3+b2_i

[Reproducible-builds] package uploaded to our repo

2015-08-12 Thread Johannes Schauer
debhelper_9.20150811.0~reproducible1.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/m

Re: [Reproducible-builds] Bug#794792: freeipmi: please make the build reproducible

2015-08-06 Thread Johannes Schauer
Hi, just some (hopefully helpful) comments about your freeipmi patch. Not CC-ing the bug so that it's completely up to you whether you want to address my comments or not. Quoting Dhole (2015-08-06 19:13:22) > The attached patch replaces the timestamp in the docs with the latest > debian/changelog

[Reproducible-builds] proposal: store information in one place instead of multiple ones

2015-07-28 Thread Johannes Schauer
Hi, here are several questions I have which, for me boil down to information being duplicated and stored in different locations, leading to possible confusion for contributors and added work when adding new bugs and issues: 1. Why is the set of bts usertags different from the set of r-b issues? T

Re: [Reproducible-builds] Bug#791815: libxslt: please support timestamps from environment

2015-07-11 Thread Johannes Schauer
Control: block -1 by 791823 On Wed, 08 Jul 2015 20:01:57 +0200 Dhole wrote: > The submitted bug for the mentioned feature in debhelper can be found > at: https://bugs.debian.org/791823 and added some bug metadata :) signature.asc Description: signature _

Re: [Reproducible-builds] Storing .deb checksums in ADMINDIR/status?

2015-06-25 Thread Johannes Schauer
Hi, Quoting Guillem Jover (2015-06-26 06:30:39) > On Tue, 2015-06-23 at 09:31:05 +0200, Jérémy Bobbio wrote: > > Some people suggested that we should record a checksum of the `.deb` > > installed as a way to unambiguously referring to a specific package. > > In principle the tuple pkgname-version

Re: [Reproducible-builds] Patches for sbuild reproducible builds

2015-02-22 Thread Johannes Schauer
Hi, Quoting Vagrant Cascadian (2015-02-14 09:24:58) > I've got a couple brief proof-of-concept patches. I committed both of your patches to the sbuild experimental git repository. Thanks! cheers, josch signature.asc Description: signature ___ Reprod

Re: [Reproducible-builds] Patches for sbuild reproducible builds

2015-02-17 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2015-02-17 09:27:22) > On Dienstag, 17. Februar 2015, Johannes Schauer wrote: > > > Also, pbuilder and sbuild would need to ensure the same exact package set > > > from build dependencies; [...] > > this has been taken care of by the srebuil

Re: [Reproducible-builds] Patches for sbuild reproducible builds

2015-02-16 Thread Johannes Schauer
Hi, Quoting Vagrant Cascadian (2015-02-16 21:01:18) > Also, pbuilder and sbuild would need to ensure the same exact package set > from build dependencies; there are various different dependency resolvers > which might result in differences that could cause unreproducibility... this has been taken

Re: [Reproducible-builds] Bug#778423: support a plain text output mode

2015-02-14 Thread Johannes Schauer
Hi, Quoting Helmut Grohne (2015-02-14 21:14:41) > A limitation of rebootstrap currently is that it can only output a build log. > Thus I embed the debbindiffs into the log. This is cumbersome to read, since > they are unrendered html. Having a plain text output mode would make reading > these logs

Re: [Reproducible-builds] [Reproducible-commits] [notes] 01/01: add new issue, randomness_in_gnu_build_id and, so far, two packages affected by it: encfs and bacula

2015-02-13 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2015-02-13 17:10:31) > On Samstag, 7. Februar 2015, Jérémy Bobbio wrote: > > I think it really is not helpful. It's like having a category > > “needs_more_work_to_understand_the_problem”. > > actually I think such a category, or even such categories, would be helpful. >

Re: [Reproducible-builds] don't panic? or: inform d-d-a

2015-02-08 Thread Johannes Schauer
Hi, Quoting Holger Levsen (2015-02-08 12:48:51) > So I think we should do a post to debian-devel-announce soonish to formalize > announce (and discuss) this mass bug filing but also and foremost announce > the > huge progresses we made. I think this intention of the mail should be made more cl

[Reproducible-builds] Bug#774554: dcmd: please support .buildinfo files

2015-01-04 Thread Johannes Schauer
Package: devscripts Version: 2.14.11 Severity: wishlist Tags: patch User: reproducible-builds@lists.alioth.debian.org Usertags: toolchain Hi, the attached patch enables dcmd to parse .buildinfo files as well. They are generated as part of the ReproducibleBuilds effort: https://wiki.debian.org/Re

[Reproducible-builds] introducing srebuild (was: Re: introducing buildinfo2snapshot)

2015-01-01 Thread Johannes Schauer
Hi, Quoting Johannes Schauer (2014-12-31 16:42:04) > today I skimmed the ReproducibleBuilds wiki page and read that "A build tool > that would reproduce a build environment using packages from > snapshot.debian.org is still missing.". > > As I understand it, this

[Reproducible-builds] introducing buildinfo2snapshot

2014-12-31 Thread Johannes Schauer
Hi, today I skimmed the ReproducibleBuilds wiki page and read that "A build tool that would reproduce a build environment using packages from snapshot.debian.org is still missing.". As I understand it, this task mainly boils down to finding a snapshot that all package versions in the .buildinfo f