Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?
Hi, sorry for coming back to this old thread now… end of the year cleaning of old threads :) On Mittwoch, 23. September 2015, Jérémy Bobbio wrote: > My take on this: I want to wait until we can rebuild packages taken from > directly the archive. We can easily run these later rebuilds with > “nocheck”. That should enable us to spot these problems. > > What do you think? That sounds sensible to me. cheers, Holger signature.asc Description: This is a digitally signed message part. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?
Chris Lamb: > I just ran into yet another package where the contents can vary > depending on whether the tests are run or not. > > As an example, without tests a given Python package entirely vanilla and > is thus reproducible in our toolchain. However, executing the tests > creates various intermediary files that are genuinely useful (eg. > compiled versions of grammars, not .pyc files). These files are then > installed to the binary package. > > I'm only really discovering these when these files themselves are > unreproducible/non-deterministic, otherwise they are completely > invisible. > > So, does this matter to us? It's strictly more of a general QA issue if > we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an > reproducibility PoV.. but on the other hand, our testing framework would > make this almost trivial to detect. > > (Why another build? Whilst adding `nocheck` to our current `b` build > could work, it would be a bad regression as I would dearly miss having > the tests run in an exotic locale and timezone, etc., hence proposing a > `c` build). My take on this: I want to wait until we can rebuild packages taken from directly the archive. We can easily run these later rebuilds with “nocheck”. That should enable us to spot these problems. What do you think? -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?
On Wed, Sep 23, 2015 at 03:14:10AM +0200, Holger Levsen wrote: > Hi, > > adding Antonio to the loop, keeping full quote for his benefit. > > On Samstag, 19. September 2015, Chris Lamb wrote: > > I just ran into yet another package where the contents can vary > > depending on whether the tests are run or not. > > I do think this is an interesting thing to test, but I'm not sure whether the > reproducible builds test setup is the right place for it. For sure it's something interesting to test. IMO a source package producing different binaries based on whether the tests are run or not is a bug. > > As an example, without tests a given Python package entirely vanilla and > > is thus reproducible in our toolchain. However, executing the tests > > creates various intermediary files that are genuinely useful (eg. > > compiled versions of grammars, not .pyc files). These files are then > > installed to the binary package. > > > > I'm only really discovering these when these files themselves are > > unreproducible/non-deterministic, otherwise they are completely > > invisible. > > > > So, does this matter to us? It's strictly more of a general QA issue if > > we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an > > reproducibility PoV.. but on the other hand, our testing framework would > > make this almost trivial to detect. > > > > (Why another build? Whilst adding `nocheck` to our current `b` build > > could work, it would be a bad regression as I would dearly miss having > > the tests run in an exotic locale and timezone, etc., hence proposing a > > `c` build). > > Antonio, do you think this is something for ci.d.o? I don't think this is something that makes much sense to do in ci because: - it does not do builds, and will take binaries from the archive - the test suite is obtained from source packages also in the archive - even then, the test suite is not executed in the context of a build, but against the installed binaries, so there would be no way to check the effects of running the test suite on the produced binaries > Chris, should we forward this as a bug report for qa.d.o so someone can > implement this some day? OTOH, if you have an idea how to meaningful > integrate > this into the current reproducible setup, I'd be all ears! From the above I > can gather that this could be possible, maybe, but I'm not sure how much > interference this will cause… I think reproducibility should include requiring producing the same binaries regardless of whether tests were executed during the build. -- Antonio Terceiro signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?
Hi, adding Antonio to the loop, keeping full quote for his benefit. On Samstag, 19. September 2015, Chris Lamb wrote: > I just ran into yet another package where the contents can vary > depending on whether the tests are run or not. I do think this is an interesting thing to test, but I'm not sure whether the reproducible builds test setup is the right place for it. > As an example, without tests a given Python package entirely vanilla and > is thus reproducible in our toolchain. However, executing the tests > creates various intermediary files that are genuinely useful (eg. > compiled versions of grammars, not .pyc files). These files are then > installed to the binary package. > > I'm only really discovering these when these files themselves are > unreproducible/non-deterministic, otherwise they are completely > invisible. > > So, does this matter to us? It's strictly more of a general QA issue if > we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an > reproducibility PoV.. but on the other hand, our testing framework would > make this almost trivial to detect. > > (Why another build? Whilst adding `nocheck` to our current `b` build > could work, it would be a bad regression as I would dearly miss having > the tests run in an exotic locale and timezone, etc., hence proposing a > `c` build). Antonio, do you think this is something for ci.d.o? Chris, should we forward this as a bug report for qa.d.o so someone can implement this some day? OTOH, if you have an idea how to meaningful integrate this into the current reproducible setup, I'd be all ears! From the above I can gather that this could be possible, maybe, but I'm not sure how much interference this will cause… cheers, Holger signature.asc Description: This is a digitally signed message part. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?
Hi all, I just ran into yet another package where the contents can vary depending on whether the tests are run or not. As an example, without tests a given Python package entirely vanilla and is thus reproducible in our toolchain. However, executing the tests creates various intermediary files that are genuinely useful (eg. compiled versions of grammars, not .pyc files). These files are then installed to the binary package. I'm only really discovering these when these files themselves are unreproducible/non-deterministic, otherwise they are completely invisible. So, does this matter to us? It's strictly more of a general QA issue if we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an reproducibility PoV.. but on the other hand, our testing framework would make this almost trivial to detect. (Why another build? Whilst adding `nocheck` to our current `b` build could work, it would be a bad regression as I would dearly miss having the tests run in an exotic locale and timezone, etc., hence proposing a `c` build). Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org / chris-lamb.co.uk `- ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds