Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?

[Holger Levsen]

Hi,

sorry for coming back to this old thread now… end of the year cleaning of old 
threads :)

On Mittwoch, 23. September 2015, Jérémy Bobbio wrote:
> My take on this: I want to wait until we can rebuild packages taken from
> directly the archive. We can easily run these later rebuilds with
> “nocheck”. That should enable us to spot these problems.
> 
> What do you think?

That sounds sensible to me.


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?

[Jérémy Bobbio]

Chris Lamb:
> I just ran into yet another package where the contents can vary
> depending on whether the tests are run or not.
> 
> As an example, without tests a given Python package entirely vanilla and
> is thus reproducible in our toolchain. However, executing the tests
> creates various intermediary files that are genuinely useful (eg.
> compiled versions of grammars, not .pyc files). These files are then
> installed to the binary package.
> 
> I'm only really discovering these when these files themselves are
> unreproducible/non-deterministic, otherwise they are completely
> invisible.
> 
> So, does this matter to us? It's strictly more of a general QA issue if
> we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an
> reproducibility PoV.. but on the other hand, our testing framework would
> make this almost trivial to detect.
> 
> (Why another build? Whilst adding `nocheck` to our current `b` build
> could work, it would be a bad regression as I would dearly miss having
> the tests run in an exotic locale and timezone, etc., hence proposing a
> `c` build).

My take on this: I want to wait until we can rebuild packages taken from
directly the archive. We can easily run these later rebuilds with
“nocheck”. That should enable us to spot these problems.

What do you think?

-- 
Lunar.''`. 
lu...@debian.org: :Ⓐ  :  # apt-get install anarchism
`. `'` 
  `-   


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?

[Antonio Terceiro]

On Wed, Sep 23, 2015 at 03:14:10AM +0200, Holger Levsen wrote:
> Hi,
> 
> adding Antonio to the loop, keeping full quote for his benefit.
> 
> On Samstag, 19. September 2015, Chris Lamb wrote:
> > I just ran into yet another package where the contents can vary
> > depending on whether the tests are run or not.
> 
> I do think this is an interesting thing to test, but I'm not sure whether the 
> reproducible builds test setup is the right place for it.

For sure it's something interesting to test. IMO a source package
producing different binaries based on whether the tests are run or not
is a bug.

> > As an example, without tests a given Python package entirely vanilla and
> > is thus reproducible in our toolchain. However, executing the tests
> > creates various intermediary files that are genuinely useful (eg.
> > compiled versions of grammars, not .pyc files). These files are then
> > installed to the binary package.
> > 
> > I'm only really discovering these when these files themselves are
> > unreproducible/non-deterministic, otherwise they are completely
> > invisible.
> > 
> > So, does this matter to us? It's strictly more of a general QA issue if
> > we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an
> > reproducibility PoV.. but on the other hand, our testing framework would
> > make this almost trivial to detect.
> > 
> > (Why another build? Whilst adding `nocheck` to our current `b` build
> > could work, it would be a bad regression as I would dearly miss having
> > the tests run in an exotic locale and timezone, etc., hence proposing a
> > `c` build).
> 
> Antonio, do you think this is something for ci.d.o?

I don't think this is something that makes much sense to do in ci
because:

- it does not do builds, and will take binaries from the archive
- the test suite is obtained from source packages also in the archive
- even then, the test suite is not executed in the context of a build,
  but against the installed binaries, so there would be no way to check
  the effects of running the test suite on the produced binaries

> Chris, should we forward this as a bug report for qa.d.o so someone can 
> implement this some day? OTOH, if you have an idea how to meaningful 
> integrate 
> this into the current reproducible setup, I'd be all ears! From the above I 
> can gather that this could be possible, maybe, but I'm not sure how much 
> interference this will cause…

I think reproducibility should include requiring producing the same
binaries regardless of whether tests were executed during the build.

-- 
Antonio Terceiro 


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?

[Holger Levsen]

Hi,

adding Antonio to the loop, keeping full quote for his benefit.

On Samstag, 19. September 2015, Chris Lamb wrote:
> I just ran into yet another package where the contents can vary
> depending on whether the tests are run or not.

I do think this is an interesting thing to test, but I'm not sure whether the 
reproducible builds test setup is the right place for it.
 
> As an example, without tests a given Python package entirely vanilla and
> is thus reproducible in our toolchain. However, executing the tests
> creates various intermediary files that are genuinely useful (eg.
> compiled versions of grammars, not .pyc files). These files are then
> installed to the binary package.
> 
> I'm only really discovering these when these files themselves are
> unreproducible/non-deterministic, otherwise they are completely
> invisible.
> 
> So, does this matter to us? It's strictly more of a general QA issue if
> we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an
> reproducibility PoV.. but on the other hand, our testing framework would
> make this almost trivial to detect.
> 
> (Why another build? Whilst adding `nocheck` to our current `b` build
> could work, it would be a bad regression as I would dearly miss having
> the tests run in an exotic locale and timezone, etc., hence proposing a
> `c` build).

Antonio, do you think this is something for ci.d.o?

Chris, should we forward this as a bug report for qa.d.o so someone can 
implement this some day? OTOH, if you have an idea how to meaningful integrate 
this into the current reproducible setup, I'd be all ears! From the above I 
can gather that this could be possible, maybe, but I'm not sure how much 
interference this will cause…


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds