On Wed, Sep 23, 2015 at 03:14:10AM +0200, Holger Levsen wrote: > Hi, > > adding Antonio to the loop, keeping full quote for his benefit. > > On Samstag, 19. September 2015, Chris Lamb wrote: > > I just ran into yet another package where the contents can vary > > depending on whether the tests are run or not. > > I do think this is an interesting thing to test, but I'm not sure whether the > reproducible builds test setup is the right place for it.
For sure it's something interesting to test. IMO a source package producing different binaries based on whether the tests are run or not is a bug. > > As an example, without tests a given Python package entirely vanilla and > > is thus reproducible in our toolchain. However, executing the tests > > creates various intermediary files that are genuinely useful (eg. > > compiled versions of grammars, not .pyc files). These files are then > > installed to the binary package. > > > > I'm only really discovering these when these files themselves are > > unreproducible/non-deterministic, otherwise they are completely > > invisible. > > > > So, does this matter to us? It's strictly more of a general QA issue if > > we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an > > reproducibility PoV.. but on the other hand, our testing framework would > > make this almost trivial to detect. > > > > (Why another build? Whilst adding `nocheck` to our current `b` build > > could work, it would be a bad regression as I would dearly miss having > > the tests run in an exotic locale and timezone, etc., hence proposing a > > `c` build). > > Antonio, do you think this is something for ci.d.o? I don't think this is something that makes much sense to do in ci because: - it does not do builds, and will take binaries from the archive - the test suite is obtained from source packages also in the archive - even then, the test suite is not executed in the context of a build, but against the installed binaries, so there would be no way to check the effects of running the test suite on the produced binaries > Chris, should we forward this as a bug report for qa.d.o so someone can > implement this some day? OTOH, if you have an idea how to meaningful > integrate > this into the current reproducible setup, I'd be all ears! From the above I > can gather that this could be possible, maybe, but I'm not sure how much > interference this will cause… I think reproducibility should include requiring producing the same binaries regardless of whether tests were executed during the build. -- Antonio Terceiro <terce...@debian.org>
signature.asc
Description: Digital signature
_______________________________________________ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
