They told me it was not totally clear what happens here, why I did this upload, what triggered the chanegs I did, and why last night.
On Tue, Sep 20, 2016 at 10:55:55PM +0000, Mattia Rizzolo wrote: > dpkg_18.104.22.168~reproducible1.dsc has just been uploaded to > https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain The changelog against 22.214.171.124~reproducible0 (i.e. what we have been running in the last months) is this: * quite a lot cleanup of the .buildinfo implementation by guillem * rewrite of the "deterministic modes in control.tar" by guillem * temporary backward compatibility for --buildinfo-id after guillem renamed the option so we can transition over our tools * "rewrote" (within quotes as it's really silly) the "disable Environment field from .buildinfo" thing, after the big modification happened to dpkg-genbuildinfo. The trigger: yesterday evening a bystander I don't know about came in the IRC channel, noticing how the diffoscope report of newly built packages in the last days was totally incomplete, only showing the diff between the .buildinfo files instead of unpacking over into the .debs. After some minutes of debugging I discovered this was caused by a change in pbuilder, which started to behave sanely¹. The buildinfo spec says² that a .dsc must be included in Checksums-Sha256 if that one is present. Regardless of whether this is a good choice, this is broken in multiple ways: * on the idea: + a .dsc could not be present for a binary build + until recently source builds were not reproducible, so an already present .dsc would be overwritten during a full build, and the .buildinfo would record the new one instead of the original * lexically: that field contains a list of built artifacts that have been built and distributed, putting a .dsc in it goes against this definition. The problem is that we do a binary-only build (-b), so the changes file does not contain the .dsc, but that one was referenced in .buildinfo nonetheless. Until pbuilder 0.226 the .dsc would have been copied over even if not referenced in .changes; starting with 0.226 it's not copied anymore. This broke diffoscope, as it considers a .buildinfo referencing a non-existing file as an invalid DotBuildinfoFile, and therefor falling back to TextFile. The fix: I went to the our dpkg sources and see why it does that, and individuated the interesting part. Then I remembered that guillem did a bunch of work on it too, and thought about looking whether some of the stuff could be merged in our tree (mostly for wider testing). Turned out that he found such thing weird too, and so guarded that part of code with an `if` that would get executed only in the case of a source build. Hence I got in touch with him, had his patches rebased, fixed a couple of glitches, and incorporated. I didn't want to squash the commits, as I would still like to keep some history of the evolution of the thing still. What I did: * rolled back the history like we do with every new dpkg release * substituted the patches for the "deterministic modes in control.tar" with one wrote by guillem * appended the patches to the .buildinfo implementation from guillem * "rewrote" (within quotes as it's really silly) the "disable Environment field from .buildinfo" thing, after the big modification happened to dpkg-genbuildinfo (the original one was from ntyni) * added back a temporary --buildinfo-identifier flag in dpkg-buildpackage after guillem renamed id. * built, tested, rebuilt, uploaded to our repo. Next steps: it would be great if somebody could figure what's the real gain of having .dsc in Checksums-Sha256. Also consider that within the context of a single sane archive (as in: files once landed don't change) like Debian's such trick is not needed, as a source package can already be identified by other information already stored in .buildinfo like Source and Version. Thanks for reading this far, Mattia (hoping to don't have to write any such long email for at least some hours…) ¹ IOW: https://bugs.debian.org/492312 https://anonscm.debian.org/git/pbuilder/pbuilder.git/commit/?id=806db12 ² https://wiki.debian.org/ReproducibleBuilds/BuildinfoSpecification#buildinfo_field_descriptions -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `-
Description: PGP signature
_______________________________________________ Reproducible-builds mailing list Reproduciblefirstname.lastname@example.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds