Bug#844431: Revised patch: Oppose

[Ximin Luo]

Russ Allbery:
> Bill Allombert  writes:
> 
>> This is one of the reasons I do not attend DebConf anymore. We are an
>> online organization. Consultation should happen online. Metting are nice
>> but they should not be used to vet consensus and ignore absentees.
> 
>> So I object to Adrian being overriden on that ground.
> 
> That's only a part of what went into this, but I will strongly defend
> using the opportunity of in-person meetings to judge consensus.  It's very
> difficult to judge consensus over email because only the strong opinions
> are visible.  There are frequently situations where there's a large
> sentiment in one direction or another that isn't expressed in long threads
> full of lots of back and forth between a small handful of people who may
> or may not have representative opinions.
> 
> We can't always do it, and we obviously have to be sensitive to the fact
> that not everyone is in the room, but we're also going for consensus, not
> unanimity.  When we have the opportunity to get direction from a large
> gathering of developers, we should make use of it.
> 
> [..]

This was off-topic but now the thread is over I'd like to add some things here:

I don't think using the opportunity of in-person meetings to judge consensus is 
such a great thing. This has been a common theme recently cropping up in FOSS 
environments, pushed by certain groups and justified by the observations that 
"only strong opinions are visible [in email threads]". Much of the time, these 
groups overlap greatly with people that are used to doing things in a physical 
setting, including making decisions by judging crowd consensus.

Debian is primarily an online organisation as Bill says, these are its roots, 
this is how it became so big, and this is where the vast majority of productive 
work is done. I think discrediting all of that simply because "some people are 
loud on mailing lists" is really short-sighted and distorted. These cases are 
few, and not representative. I myself have contributed to a few of these cases, 
but again it's not representative of the vast majority of work that I do. I 
don't feel it's right for people to be judging online discussion methods, by 
focusing on a few negative cases and ignoring all the positive aspects.

Personally, and I'm sure many people are similar, I prefer to have long 
technical discussions like this in writing via email, and not face-to-face. I'm 
a very slow thinker, I don't make very good decisions in the fast-paced context 
of a normal physical conversation. If I sometimes seem like I do, it's usually 
only because I've thought about the problem beforehand and have my main points 
decided. 

Physical discussions encourage non-technical interactions - if you can pick the 
right words and presentation, you can make a crowd empathise with you for 
largely non-technical reasons. I don't think this is a good thing, we should 
recognise that this happens and not allow it to take over Debian's decision 
making processes. Online technical discussions are safer against these sorts of 
effects. People with long technical points to make, don't feel put off by the 
presence of a large non-technical crowd - and here I include "technical people" 
that have not properly thought about the topic or have no stake in the 
discussion. I truly think these latter opinions should be given less weight 
than a properly reasoned and well-thought-out opinion.

These sorts of points, which are vital to a healthy discussion, are easier to 
make in writing. You have an adequate amount of time to think, and you don't 
get interrupted by people who get bored by your thinking time and move the 
conversation on elsewhere before you have a chance to properly respond. Indeed 
in this thread there were lots of good points brought up criticising the 
wording of this policy, that nobody thought about during physical discussions 
at DebConf (which I didn't participate in for these reasons).

TL;DR: Debian is primarily an online organisation and that is its strength; 
physical meetings are overrated for making long-term technical decisions.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: Oppose

[Russ Allbery]

Bill Allombert  writes:
> On Wed, Aug 16, 2017 at 12:14:53PM -0700, Russ Allbery wrote:

>> If you have specific wording suggestions that you believe would bring
>> this Policy requirement closer in line with what we're already doing in
>> the project (and which has gotten us to 94% reproducible already),
>> please make them.

> This percentage was reached mostly by fixing software tools (compiler,
> doc generators, packaging tools) to be deterministic, rather than by
> fixing individual packages. This is a topic that is wholy absent from
> policy.

Indeed.  There are many things that go into making Debian work that are
wholly absent from Policy.  Hopefully, over time, we can slowly reduce
that, but there will always be new initiatives that aren't documented.

> For example policy could mandate that programs that set timestamps
> honour SOURCE_DATE_EPOCH.

Please propose language.  (Ideally in a separate bug, since this one is
already quite large and it's easier to address specific issues in specific
bugs.)

I'm not opposed to adding more advice and requirements that make sense,
but there are lots of things in Policy that aren't as fully described as
they possibly could be if people did more work.  I'm not willing to block
this on having the perfect language, but if you want to contribute, you're
absolutely welcome to do so.

Most packages do not have to care about SOURCE_DATE_EPOCH because it's set
by dpkg-buildpackage and consumed by the tools that are most frequently
relevant, but I'd be very happy to see that documented in Policy for the
packages that do care.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: Oppose

[Bill Allombert]

On Wed, Aug 16, 2017 at 12:19:47PM -0700, Don Armstrong wrote:
> On Wed, 16 Aug 2017, Bill Allombert wrote:
> > But as a technical document, it is lacking practical recommendation
> > for maintainers how to make sure their package build reproducibly
> 
> The practical recommendations for maintainers are encoded separately, as
> they're evolving. See https://wiki.debian.org/ReproducibleBuilds/Howto
> for example.

> > and create confusion on the goal by using the term 'reproducible
> > build' when 'robust build' is meant (which is a worthwile goal by
> > itself, but a different project nevertheless).
> 
> I'm not convinced that this change will reduce confusion, as the
> reproducible build project has been using this language in Debian for
> many years now.

It is apparent that the reproducible build project has broadened their
focus following the progress they made, which is logical.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#844431: Revised patch: Oppose

[Bill Allombert]

On Wed, Aug 16, 2017 at 12:14:53PM -0700, Russ Allbery wrote:
> If you have specific wording suggestions that you believe would bring this
> Policy requirement closer in line with what we're already doing in the
> project (and which has gotten us to 94% reproducible already), please make
> them.

This percentage was reached mostly by fixing software tools (compiler, 
doc generators, packaging tools) to be deterministic, rather than by
fixing individual packages. This is a topic that is wholy absent from
policy.

For example policy could mandate that programs that set timestamps honour
SOURCE_DATE_EPOCH.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#844431: Revised patch: Oppose

[Don Armstrong]

On Wed, 16 Aug 2017, Bill Allombert wrote:
> But as a technical document, it is lacking practical recommendation
> for maintainers how to make sure their package build reproducibly

The practical recommendations for maintainers are encoded separately, as
they're evolving. See https://wiki.debian.org/ReproducibleBuilds/Howto
for example.

> and create confusion on the goal by using the term 'reproducible
> build' when 'robust build' is meant (which is a worthwile goal by
> itself, but a different project nevertheless).

I'm not convinced that this change will reduce confusion, as the
reproducible build project has been using this language in Debian for
many years now.

But I could be wrong. Please propose an alternative patch to policy
which addresses your concerns if you feel strongly about it.

-- 
Don Armstrong  https://www.donarmstrong.com

Maybe I did steal your heart
and I am such a perfect criminal
that you never noticed
 -- a softer world #481
http://www.asofterworld.com/index.php?id=481

Bug#844431: Revised patch: Oppose

[Russ Allbery]

Bill Allombert  writes:

> This is one of the reasons I do not attend DebConf anymore. We are an
> online organization. Consultation should happen online. Metting are nice
> but they should not be used to vet consensus and ignore absentees.

> So I object to Adrian being overriden on that ground.

That's only a part of what went into this, but I will strongly defend
using the opportunity of in-person meetings to judge consensus.  It's very
difficult to judge consensus over email because only the strong opinions
are visible.  There are frequently situations where there's a large
sentiment in one direction or another that isn't expressed in long threads
full of lots of back and forth between a small handful of people who may
or may not have representative opinions.

We can't always do it, and we obviously have to be sensitive to the fact
that not everyone is in the room, but we're also going for consensus, not
unanimity.  When we have the opportunity to get direction from a large
gathering of developers, we should make use of it.

But I'm taking this position for a large number of reasons, of which
consensus at DebConf is only one.

> But as a technical document, it is lacking practical recommendation for
> maintainers how to make sure their package build reproducibly and create
> confusion on the goal by using the term 'reproducible build' when
> 'robust build' is meant (which is a worthwile goal by itself, but a
> different project nevertheless).

If you have specific wording suggestions that you believe would bring this
Policy requirement closer in line with what we're already doing in the
project (and which has gotten us to 94% reproducible already), please make
them.  I am not at all trying to rule out constructive suggestions to make
the language better, either now or in subsequent revisions of Policy.  I
think what we've got is pretty good, and I am comfortable with putting it
into Policy now, but concrete wording proposals with justification would
be welcome.  Like everything else in Policy, we can always improve how we
describe our project-wide baseline.

It's not normally Policy's role to offer detailed advice on how to meet a
particular requirement.  For example, Policy mentions debhelper in a few
footnotes but doesn't document how to use it to create compliant packages
in general.  That's not its purpose; usually that sort of documentation
can be better-maintained by other teams, such as the reproducible builds
team.

The definition in the current proposal is intentionally weaker (in the
sense that fewer packages would fail it) than what current reproducible
build testing is doing in a few places (such as with environment
variables) because it takes a conservative stance to set a baseline and it
errs on the side of a clear and simple problem statement.  It's possible
that we'll want to make it more complex in the future, but I think with
environment variables we should first have a discussion (Ximin and I
started that; I probably should move it off this thread) because I'm not
sure that's the best approach.  In the meantime, this achieves the goal of
declaring a baseline that Debian packages should be reproducible under
controlled and simple-to-state circumstances, which is something for which
I'm quite sure we have general project consensus.

If you believe it is premature to specify this in Policy entirely, I
strongly disagree, and am not persuadable on that point.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: Oppose

[Bill Allombert]

On Wed, Aug 16, 2017 at 09:30:23AM -0700, Russ Allbery wrote:
> As Policy Editor (a delegated position), based on my read of project
> consensus including in-person verification of that consensus at DebConf
> 17, I am formally declaring that I believe this change has consensus
> despite your opposition.  We will therefore include this change in the
> next release of Policy.

This is one of the reasons I do not attend DebConf anymore. We are an
online organization. Consultation should happen online. Metting are nice
but they should not be used to vet consensus and ignore absentees.

So I object to Adrian being overriden on that ground.

Now it seems to me this policy proposal is a way to acknowledge the
great work of the reproducible build project. As such it is probably
fine and they amply deserve praise and acknowledgement.

But as a technical document, it is lacking practical recommendation for
maintainers how to make sure their package build reproducibly and create
confusion on the goal by using the term 'reproducible build' when 
'robust build' is meant (which is a worthwile goal by itself, but a
different project nevertheless). 

So I am concerned we are putting the cart before the horse.

Cheers,
Another Policy Editor (a delegated position).
-- 
Bill. 

Imagine a large red swirl here. 

Bug#844431: Revised patch: Oppose

[Russ Allbery]

Just to be completely, 100% clear: I will not be responding further to
this line of argument in this bug.  If you disagree with my decision as a
project delegate, I've spelled out your possible next steps under Debian's
governance process.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: Oppose

[Adrian Bunk]

On Wed, Aug 16, 2017 at 09:30:23AM -0700, Russ Allbery wrote:
>...
> This text is a formalization and simplification of existing practice that
> we worked out in conjuction with the reproducible builds team and that
> strikes a balance between attempting to enumerate all the causes of
> nonreproducibility (which would be quite difficult to do) and providing
> some clear guidance to maintainers about what types of output variance
> they *don't* have to worry about (since obviously packages can't be
> reproducible under all circumstances and in all environments).  The
> intention is to set a minimum bar that packages should be trying to meet,
>...

The definition of reproducibility in policy does not match any past, 
present or future practice in Debian.

And no current or currently planned reproducible testing does test
or is intended to test whether packages meet this minimum bar.

> This is directly in the center of Policy's normal role of standardizing
> and documenting best practice that has been developed elsewhere in the
> project.
>...

If it would actually standardize what is considered reproducible
in Debian everything would be fine.

> The definition is not decoupled from current practice.  It is roughly
> equivalent to the information currently captured in *.buildinfo files
> while being easily comprehensible to people who haven't studied
> *.buildinfo files.
>...

2 of the 5 items in policy require changes to .buildinfo, and for a 
third I cannot easily comprehend whether it would require changes to 
.buildinfo since it is unclear what it is supposed to mean:

- a set of environment variable values;

.buildinfo currently records only some environment variables.
If all or different ones are allowed to vary that is a change.
I am actually surprised that the latest set of suggested permitted
variations does not seem to be based on the existing list currently
used for .buildinfo

- a version of a source package unpacked at a given path;

The path is currently not in .buildinfo

- a build architecture;

What is the intended purpose of this, especially what is this supposed 
to output for i386 builds on amd64 kernel?
.buildinfo currently follows dpkg-architecture, and outputs i386.
i386/amd64 kernels is a build variation in the reproducible builds
infrastructure that does result in packages being built differently,
which makes it unclear whether this difference was supposed to be
addressed here.

> Finally, Policy in no way constrains people from filing bugs or reporting
> issues (via whatever means, such as tracker.debian.org) in packages about
> things that are not spelled out in Policy.
>...

https://tracker.debian.org/pkg/hsqldb1.8.0
"Does not build reproducibly during testing"

This statement in tracker is automatically generated based on results 
from the reproducible builds infrastructure.

Is it acceptable to claim in tracker that a package is not reproducible, 
when that package might actually be reproducible based on the definition 
of reproducibility spelled out in Policy? [1]

cu
Adrian

[1] as explained earlier, it is not obvious whether or not this
specific package is reproducible according to Policy

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: Oppose

[Russ Allbery]

Adrian Bunk  writes:

> I hereby oppose the addition of this to policy.

> It is not true that this would be "Debian's precisification" of
> reproducible builds.

> The definition does not match any past, present or future practice in
> Debian.

> Including the people who want this change to policy, there seems to be 
> noone intending to use this definition of reproducibility.

> Adding this to policy would do more harm than good.

Let me get the formal part of this out of the way first:

As Policy Editor (a delegated position), based on my read of project
consensus including in-person verification of that consensus at DebConf
17, I am formally declaring that I believe this change has consensus
despite your opposition.  We will therefore include this change in the
next release of Policy.

If you disagree, your choices of action are appealing to the Technical
Committee under section 6.1 of the Constitution (I'm fine with using that
section and letting the TC take a majority vote), or propose a GR under
section 4.1.3.

Okay, now, why I'm taking that stance:

This text is a formalization and simplification of existing practice that
we worked out in conjuction with the reproducible builds team and that
strikes a balance between attempting to enumerate all the causes of
nonreproducibility (which would be quite difficult to do) and providing
some clear guidance to maintainers about what types of output variance
they *don't* have to worry about (since obviously packages can't be
reproducible under all circumstances and in all environments).  The
intention is to set a minimum bar that packages should be trying to meet,
and to lay the groundwork for future work.

This is directly in the center of Policy's normal role of standardizing
and documenting best practice that has been developed elsewhere in the
project.  The project is already comfortable with filing bugs against
packages for being non-reproducible under this criteria of
non-reproducibility, and we already have put significant work into
establishing a baseline and have a firm understanding of how close we're
currently coming to meeting that baseline.  This meets the bar for
maturity of work that we look for in major Policy changes.  As with many
other things in Debian, we hope to improve further later on, and to raise
the Policy bar over time as we develop better tools and better
understanding, but this bar is one that we can start recognizing with
normal-priority bugs right now.

The bug severity was specifically chosen to not kick any packages out of
Debian and to not make reproducible builds mandatory, simply to recognize
them as bugs that we as a project want to see fixed.  This feels like the
right balance to strike at this point.  More work would be required to
make them RC.

The definition is not decoupled from current practice.  It is roughly
equivalent to the information currently captured in *.buildinfo files
while being easily comprehensible to people who haven't studied
*.buildinfo files.  More precision will be possible in the future, but we
don't have to wait for that to set the simpler bar.

On the consensus side, there was a rare opportunity here to get a measure
of consensus from a large section of the project.  Holger specifically
asked for a show of hands and a show of objections (there were none) for
including this standard in Policy, and we had various discussions with
other people over the course of DebConf.  We have a much better read on
project consensus for this than we have for many other things in Debian.

Finally, Policy in no way constrains people from filing bugs or reporting
issues (via whatever means, such as tracker.debian.org) in packages about
things that are not spelled out in Policy.  This is a core principle of
Policy maintenance that we have held to for the more than a decade I've
been involed in Policy maintenance.  Policy is not an exhaustive list of
the possible bugs in packages, and never will be, and Policy will never
prevent people from filing bugs against packages at the severity that they
think is appropriate.  The definition of reproducibility is no exception
to this general rule.

Rather, the general project stance has been that Policy spells out the
things that are less open to debate, and bugs filed on the basis of things
that aren't in Policy are more at the maintainer's discretion (assuming
obvious common sense is applied).  And that's what I would expect for any
bugs filed about reproducible builds failing criteria more strict than
those stated in Policy (such as differing build paths) until such time as
project consensus builds that we want to hold all packages to that
stricter standard regardless of maintainer preference.

To be clear, the above discussion is intended as an explanation for this
decision, not a continuation of debate.  If you disagree with the above,
you should probably address those objections to the Technical Committee; I
feel like I have a pretty complete 

Bug#844431: Revised patch: Oppose

[Adrian Bunk]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Sat, Aug 12, 2017 at 03:34:35PM -0700, Sean Whitton wrote:
>...
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or 
> build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
> See the file ``upgrading-checklist`` for information about policy
> which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
> often creates either static linking or shared library conflicts, and,
> most importantly, increases the difficulty of handling security
> vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition `_.

I hereby oppose the addition of this to policy.

It is not true that this would be "Debian's precisification"
of reproducible builds. 

The definition does not match any past, present or future practice in Debian.

Including the people who want this change to policy, there seems to be 
noone intending to use this definition of reproducibility.

Adding this to policy would do more harm than good.

E.g. tracker.d.o saying "Does not build reproducibly during testing" 
based on a definition of reproducibility that is quite different from 
the official "Debian precisification" would only create confusion.

> Sean Whitton

cu
Adrian

- -- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEOvp1f6xuoR0v9F3wiNJCh6LYmLEFAlmUZAsACgkQiNJCh6LY
mLG9RhAAjr0dgpxSv9lnqM3+4AR3JeWwTaj9J118Efsr4qmSbgK9gE3HE3bL7zXG
OJHE5AqGZidx/Oyw/+TVLq3cHEi+6WfgJcwNzFeRAa7fAv+BKSJJ4T9dhOBYvmfs
YN/BfIhU8j4bQppVFtsduprdxooBx9bHWO/lFzCLl/cZOZ7RPOCya7iXcgEgWuA2
SAo96bcDeL3h5I/qM7fBLcm4Yvca219u8RoD7HqQNcmEI53CKS5qIW1cy0wkNbUy
Pqgovee2GpW7WkgqdG92E770/m2tcxdQQywVf5IeLHiSfJ0VP9dGFOoQCsnXZgvg
4GGstXzTJ2OEKMQ2QK1938Tne0S1WIG5o2zLEzOpHqw11Z9TsRg94CRm0/f/tfNt
ym35/N3qNdjERzozTQckbz4ZKCyLKJU3AIxGOH1U1caIjSNBbWY+nGAu62SzY9fb
IVdmKBkqL+c0MT4AW4yRUjFQ/EZYQNkWrh9USPAlgtWdIfjP4ERJ+60RJcRSgYvz
cJJw8DfDKYTNI6sgu0W++rhv89J4eAFdBKDmBazO5gLnFYBacgrFXW9HvwkxCcSZ
WJUlcuEalDpZrtPKGYO5arQp/vWWqXsVBzZeUphi6UbUjmCw+1M4emJh9Zk41jU3
BeTKcjh/hr0tihUvXhZKAJ85HmSkVLjPqZfY/DNiDecr9q+ZdvQ=
=i6V/
-END PGP SIGNATURE-