Re: Debian Buster will only be 54% reproducible (while we could be at >90%)
On Sat, Mar 09, 2019 at 09:15:34PM +, Dimitri John Ledkov wrote: [packages uploaded before December 2016, which didnt see binNMU since then, dont have a .buildinfo file (and were built with a dpkg which produces unreproducible packages] > So I guess after we release buster, we should do a mass-source-nmu > no-change uploads to make the .deb reproducible as shipped in the > archive. maybe. I think I first want to have the numbers right (and then current) to see how many packages are affected. Also: a not so low percentages of packages is still uploaded as non-source-only upload. I *believe* this should be fine (as nowadays the upload contains the .buildinfo file of that developer build, so if a package is reproducible this .buildinfo file can be taken). > Is this thus then an effectively a maas bug-file request to > sourcefully rebuild packages? no, at least not yet. -- tschau, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Debian Buster will only be 54% reproducible (while we could be at >90%)
On Wed, Mar 06, 2019 at 09:40:23AM +, peter green wrote: > > Because of their design, binNMUs are unreproducible, see #894441 [3] for > > the details (in short: binNMUs are not what they are ment to be: the source > > is changed and thrown away) > To be specific, the source tree is extracted, then an entry is added to > debian/changelog and then the package is built. This modified source > tree is not retained. indeed. > It seems to me that binnmus could be made reproducible by storing the > debian/changelog modifications in the buildinfo, then re-applying it > at reproduction time. that's actually the case nowadays, not sure since when. eg https://buildinfos.debian.net/ftp-master.debian.org/buildinfo/2019/04/23/dmtx-utils_0.7.6-1.1+b1_kfreebsd-amd64.buildinfo starts like this: Format: 1.0 Source: dmtx-utils (0.7.6-1.1) Binary: dmtx-utils Architecture: kfreebsd-amd64 Version: 0.7.6-1.1+b1 Binary-Only-Changes: dmtx-utils (0.7.6-1.1+b1) sid; urgency=low, binary-only=yes . * Binary-only non-maintainer upload for kfreebsd-amd64; no source changes. * rebuild for libdmtx0b . -- kfreebsd-amd64 / kfreebsd-i386 Build Daemon (kamp) Sun, 14 Apr 2019 01:03:43 + [...] I yet have to actually test if one can bit by bit reproduce binNMUs with this information, but I'm quite very hopeful. -- tschau, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Debian Buster will only be 54% reproducible (while we could be at >90%)
On Tue, 5 Mar 2019 at 13:34, Holger Levsen wrote: > > hi, > > disclaimer: this has not yet been verified by anyone other than myself, > so I could very well be wrong. Reproducible builds are about enabling > anyone to independently verify that... ;p > > > == Reproducibility in theory == > > According to > https://tests.reproducible-builds.org/debian/buster/index_suite_amd64_stats.html > we have 26476 source packages (92.8%) which can be built reproducibly in > buster/amd64, out of 28523 source packages in total. > (These 28523 source packages build 57448 binary packages.) > > But these tests are done without looking at the actual .deb files distributed > from ftp.debian.org (and we always knew that and pointed it out: > "93% reproducible _in our current test framework_".) > So I guess after we release buster, we should do a mass-source-nmu no-change uploads to make the .deb reproducible as shipped in the archive. Is this thus then an effectively a maas bug-file request to sourcefully rebuild packages? -- Regards, Dimitri.
Re: Debian Buster will only be 54% reproducible (while we could be at >90%)
Am Mi., 6. März 2019 um 10:40 Uhr schrieb peter green : > > > Because of their design, binNMUs are unreproducible, see #894441 [3] for > > the details (in short: binNMUs are not what they are ment to be: the source > > is changed and thrown away) > To be specific, the source tree is extracted, then an entry is added to > debian/changelog and then the package is built. This modified source tree is > not retained. > [...] (Experience report incoming) I have once tried that in the Tanglu derivative, and found out that this wasn't as easy as I initially thought because a lot of packages run special tools prior to building their sources, e.g. to edit d/control or to read d/changelog and inject data in several places. So, the option there was either to create a chroot dedicated to the source package rebuild (installing all Build-Deps and Pre-Deps prior to the actual source rebuild), or to not actually rebuild the source package but just edit the d/changelog file and recreate the tarball. For Tanglu we went for the "just edit d/changelog and re-tar, re-sign & upload" which worked fine and without any noticeable issues - this was mainly due to the limited build power we had at the time. For Debian, which has a lot more resources, just full rebuilding the source with all dependencies is likely much cleaner, but this approach might be a bit slow for huge transitions. At Ubuntu, some people seem to do this process manually, that is run some scripts locally, rebuild sources locally & upload (unless that has changed recently). In general, having source d/changelog aligned with the actual binaries produced is a really great goal! Cheers, Matthias -- I welcome VSRE emails. See http://vsre.info/
re: Debian Buster will only be 54% reproducible (while we could be at >90%)
Because of their design, binNMUs are unreproducible, see #894441 [3] for the details (in short: binNMUs are not what they are ment to be: the source is changed and thrown away) To be specific, the source tree is extracted, then an entry is added to debian/changelog and then the package is built. This modified source tree is not retained. It seems to me that binnmus could be made reproducible by storing the debian/changelog modifications in the buildinfo, then re-applying it at reproduction time.
