Bug#844431: Reproducibility in Policy

[Holger Levsen]

On Fri, Aug 11, 2017 at 08:35:47PM -0700, Russ Allbery wrote:
> Daniel Kahn Gillmor  writes:
> > I don't like the idea of hard-coding a fixed build path requirement into
> > debian policy. 

I don't *like* it neither but I think it's the sensible thing to do now.

> > We're over 80% with variable build paths in unstable
> > already, and i want to keep the pressure up on this.  The build location
> > should not influence the binary output.

I'd like to keep the pressure on this but and I think we can still that
while OTOH also trying to get closer to 100% first+too.

With build path variation reaching the worthwhile goal of having >98% 
reproducible
builds will be delayed by 1-2 years at least, so this is a classic "perfect is 
the
enemy of good". I don't do reproducible builds for purely academic reasons,
I foremost want them to increase the security of user systems.

> It shouldn't, but my understanding is that it currently does.  If you can
> fix that, that's great, but until that's been fixed, I don't see the harm
> in documenting this as a prerequisite for a reproducible build.  If we can
> relax that prerequisite later, great, but nothing about listing it here
> should reduce the pressure on making variable build paths work.  It just
> documents the current state of the world.

exactly.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Bug#844431: Reproducibility in Policy

[Russ Allbery]

Bill Allombert  writes:

> This require policy to define the build environment and build
> instruction much more precisely than it does now, which does not seems
> to be practical. Unless maybe if a reference implementation is provided.

I don't see anything in this proposal that would require a more precise
definition than we have in Sean's current proposal.  This is the standard
that we're already using for filing reproducible build bugs in the
archive, and it's been basically fine.

The tools aren't in place yet to make it super-easy for people to test for
themselves, but that's in the works, and that's also why it's a should
(not must) and there's infrastructure in place for Debian to check it for
you.

We can always aspire to get more formal and specific in the future, but
that's true of many other parts of Policy as well.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Reproducibility in Policy

[Bill Allombert]

On Fri, Aug 11, 2017 at 04:08:47PM -0700, Sean Whitton wrote:
> control: user debian-pol...@packages.debian.org
> control: usertag = normative proposal
> 
> Hello,
> 
>  Proposal: 
> 
> This is what Holger and I think we should add to Policy, after
> readability tweaks:
> 
> Packages should build reproducibly, which for purposes of this
> document means that given
> 
> - a version of a source package unpacked at a given path;
> - a set of versions of installed build-dependencies; and
> - a build architecture,
> 
> repeatedly building the source package on the architecture with those
> versions of the build dependencies installed will produce bit-for-bit
> identical binary packages.
> 
>  Explanation: 
> 
> The definition from the reproducible builds group[1] says:
> 
> A build is reproducible if given the same source code, build
> environment and build instructions, any party can recreate
> bit-by-bit identical copies of all specified artifacts.
> 
> The relevant attributes of the build environment, the build
> instructions and the source code as well as the expected
> reproducible artifacts are defined by ... distributors.
> 
> i.e. Debian has to define the build environment, source code and build
> instructions.  I think that my wording defines these as Debian currently
> understands them.

This require policy to define the build environment and build
instruction much more precisely than it does now, which does not
seems to be practical. Unless maybe if a reference implementation
is provided.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#844431: Reproducibility in Policy

[Johannes Schauer]

Hi,

Quoting Russ Allbery (2017-08-12 09:57:44)
> I think we need to add all environment variables starting with DEB_* to
> the prerequisites.  If you set DEB_BUILD_OPTIONS=nostrip or
> DEB_BUILD_MAINT_OPTIONS=hardening=all, you'll definitely get a different
> package, for instance.
> 
> I feel like there are a bunch of other environment variables that have to
> be consistent, although I'm not sure how to specify that since other
> environment variables shouldn't matter.  But, say, setting GNUTARGET is
> very likely to cause weirdness by changing how ld works.  There are
> probably more interesting examples.
> 
> How does the current reproducible build testing work with the environment?
> Maybe we should just document that for right now and relax it later if
> needed?

currently, dpkg-genbuildinfo records all environment variables in a .buildinfo
file which pass a whitelist check. The current whitelist is stored here:

https://anonscm.debian.org/cgit/dpkg/dpkg.git/tree/scripts/Dpkg/Build/Info.pm#n50

I'm not proposing that this whole list should be added to policy. But the list
that ends up in policy must be a subset of the list of environment variables
that dpkg-genbuildinfo stores in the .buildinfo file. Thus:

 - this list from dpkg should give a number of good suggestions of which
   environment variables should be added to policy

 - if any additional variables are added, then they must be added to
   dpkg-genbuildinfo as well.

Thanks!

cheers, josch


signature.asc
Description: signature

Bug#844431: Reproducibility in Policy

[Russ Allbery]

Daniel Kahn Gillmor  writes:
> On Fri 2017-08-11 16:08:47 -0700, Sean Whitton wrote:

>> - a version of a source package unpacked at a given path;

> I don't like the idea of hard-coding a fixed build path requirement into
> debian policy.  We're over 80% with variable build paths in unstable
> already, and i want to keep the pressure up on this.  The build location
> should not influence the binary output.

It shouldn't, but my understanding is that it currently does.  If you can
fix that, that's great, but until that's been fixed, I don't see the harm
in documenting this as a prerequisite for a reproducible build.  If we can
relax that prerequisite later, great, but nothing about listing it here
should reduce the pressure on making variable build paths work.  It just
documents the current state of the world.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Reproducibility in Policy

[Daniel Kahn Gillmor]

Thanks for the proposal.  I like it!  A few nit-picks below:

On Fri 2017-08-11 16:08:47 -0700, Sean Whitton wrote:

> - a version of a source package unpacked at a given path;

I don't like the idea of hard-coding a fixed build path requirement into
debian policy.  We're over 80% with variable build paths in unstable
already, and i want to keep the pressure up on this.  The build location
should not influence the binary output.


> repeatedly building the source package on the architecture with

maybe s/on the architecture/on any machine of the same architecture/ ?

all the best,

--dkg

Bug#844431: Reproducibility in Policy

[Russ Allbery]

Sean Whitton  writes:

>  Proposal: 

> This is what Holger and I think we should add to Policy, after
> readability tweaks:

> Packages should build reproducibly, which for purposes of this
> document means that given

> - a version of a source package unpacked at a given path;
> - a set of versions of installed build-dependencies; and
> - a build architecture,

> repeatedly building the source package on the architecture with those
> versions of the build dependencies installed will produce bit-for-bit
> identical binary packages.

I think we need to add all environment variables starting with DEB_* to
the prerequisites.  If you set DEB_BUILD_OPTIONS=nostrip or
DEB_BUILD_MAINT_OPTIONS=hardening=all, you'll definitely get a different
package, for instance.

I feel like there are a bunch of other environment variables that have to
be consistent, although I'm not sure how to specify that since other
environment variables shouldn't matter.  But, say, setting GNUTARGET is
very likely to cause weirdness by changing how ld works.  There are
probably more interesting examples.

How does the current reproducible build testing work with the environment?
Maybe we should just document that for right now and relax it later if
needed?

>  Explanation: 

> The definition from the reproducible builds group[1] says:

> A build is reproducible if given the same source code, build
> environment and build instructions, any party can recreate
> bit-by-bit identical copies of all specified artifacts.

> The relevant attributes of the build environment, the build
> instructions and the source code as well as the expected
> reproducible artifacts are defined by ... distributors.

> i.e. Debian has to define the build environment, source code and build
> instructions.  I think that my wording defines these as Debian currently
> understands them.

> Later, we could narrow the definition of build environment by adding
> more constraints, but we're not there yet.

> [1]  https://reproducible-builds.org/docs/definition/

We should add a link to that page (maybe in a footnote).

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Reproducibility in Policy

[Chris Lamb]

Dear Sean & Holger,

Thank you so much for working on this at the end of a tiring DebConf!

> […]
> Later, we could narrow the definition of build environment by adding
> more constraints, but we're not there yet.

That makes sense. Indeed, that even feels like the optimal approach
as it allows flexibility and experimentation, probably more important
the closer and closer we get to to 100% reproducibility.

Thanks again :)


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#844431: Reproducibility in Policy

[Sean Whitton]

control: user debian-pol...@packages.debian.org
control: usertag = normative proposal

Hello,

 Proposal: 

This is what Holger and I think we should add to Policy, after
readability tweaks:

Packages should build reproducibly, which for purposes of this
document means that given

- a version of a source package unpacked at a given path;
- a set of versions of installed build-dependencies; and
- a build architecture,

repeatedly building the source package on the architecture with those
versions of the build dependencies installed will produce bit-for-bit
identical binary packages.

 Explanation: 

The definition from the reproducible builds group[1] says:

A build is reproducible if given the same source code, build
environment and build instructions, any party can recreate
bit-by-bit identical copies of all specified artifacts.

The relevant attributes of the build environment, the build
instructions and the source code as well as the expected
reproducible artifacts are defined by ... distributors.

i.e. Debian has to define the build environment, source code and build
instructions.  I think that my wording defines these as Debian currently
understands them.

Later, we could narrow the definition of build environment by adding
more constraints, but we're not there yet.

[1]  https://reproducible-builds.org/docs/definition/

-- 
Sean Whitton


signature.asc
Description: PGP signature