Bug#844431: Revised patch: seeking seconds

[Chris Lamb]

Bill,

> > > Now compare with reproducible build. You get some error report you
> > > cannot reproduce, do some change following the help provided and
> > > hope for the best. Then some day later you get the same error
> > > report.
> > 
> > I'd dearly love to know when/where this occurred if you can provide a
> > reference.
> 
> This happens for errors listed on the reproducible-build.org website.
> I do not speak about bug report here.

Oh, I see. I was interpreting your "same error report" and similar
phrasings to mean some active, human-driven notification and interaction
such as a bug report or private mail, rather than something cold and
automated from a not — yet! — perfect CI framework.

Thank you for clarifying, but please do also bear with us whilst we
improve the reliability of the jenkins.debian.org results.


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#844431: Revised patch: seeking seconds

[Bill Allombert]

On Wed, Aug 16, 2017 at 05:40:23PM -0700, Chris Lamb wrote:
> Hi Bill,
> 
> > Now compare with reproducible build. You get some error report you
> > cannot reproduce, do some change following the help provided and
> > hope for the best. Then some day later you get the same error
> > report.
> 
> I'd dearly love to know when/where this occurred if you can provide a
> reference.

This happens for errors listed on the reproducible-build.org website.
I do not speak about bug report here.

> This is not our, and certainly not my own, intention when filing
> reproducibility-related bugs, which always include a well-intentioned
> patch.

I know from experience you and the reproducible-build team report
excellent bug report with good patches. That is not the issue but you do
not need policy to continue doing that.

However, are not maintainers expected to make their packages
policy-compliant without waiting for bug report ?
Are not maintainers supposed to be proactive and try to fix
issues that they became aware without waiting for someone to fill a bug ?
Are not users allowed to fill bugs when packages does not seem to comply
with documented expectation ?

As I said, if this policy is only meant to be a vehicule for the
reproducible-build team, then it is fine by me. However if it means
for general audience, then it is premature.
It would be best to focus first on requiring all generators to be
deterministic. One this step is reached (we are already close thanks to
the reproducible-build team work), it will much easier for maintainers
to deal wih reproducibility issues because they will not need
work-around.

Cheers,
-- 
Bill. 

Imagine a large red swirl here.

Bug#844431: Revised patch: seeking seconds

[Chris Lamb]

Hi Bill,

> Now compare with reproducible build. You get some error report you
> cannot reproduce, do some change following the help provided and
> hope for the best. Then some day later you get the same error
> report.

I'd dearly love to know when/where this occurred if you can provide a
reference.

This is not our, and certainly not my own, intention when filing
reproducibility-related bugs, which always include a well-intentioned
patch.


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Russ Allbery:
> Ximin Luo  writes:
> 
>> Fair enough. I actually spotted that but thought it was better to get
>> "something" into Policy rather than nitpick. I guess other people were
>> thinking similar things. Well, lesson learnt, I will be more forceful
>> next time.
> 
>> The sentence I amended said "most environment variables" so our intent
>> is clear. If we want to fix this now, I would suggest amending:
> 
>> - a set of environment variable values; and
>> + a set of reserved environment variable values; and
> 
>> then later:
> 
>> + A "reserved" environment variable is defined as DEB_*, DPKG_, 
>> SOURCE_DATE_EPOCH, BUILD_PATH_PREFIX_MAP, variables listed by 
>> dpkg-buildflags and other variables explicitly used by buildsystems to 
>> affect build output, excluding any variables used by non-build programs to 
>> affect their behaviour. Explicitly, this excludes TERM, HOME, LOGNAME, USER, 
>> PATH and likely any variables ending with *PATH.
> 
> We intentionally didn't spell this out in this much detail because it felt
> better to defer this (stricter) bar until we have documentation of the
> *.buildinfo file, and also because we were worried about the list changing
> (once it goes into Policy, it's more irritating to change).  The current
> standard in Policy is intentionally weaker than this in order to be
> simpler.
> 
> I still lean towards taking this approach, because I'm pretty worried
> about the scope of:
> 
> other variables explicitly used by buildsystems to affect build output
> 
> That's not really an enumerable list.  My recommendation, if you want to
> allow some environment variables to vary without affecting
> reproducibility, is to explicitly list the set of environment variables
> that can vary, rather than trying to list the ones that have to remain
> fixed.
> 

Intuitively it feels weird to say "if you vary USER, the output must remain 
fixed", but also "if you vary RANDOMUNIQUESPECIALSNOWFLAKEVARIABLE then the 
output is allowed to change".

Certain environment variables have become convention to affect a build, like 
CFLAGS, and even debuild(1) doesn't clear them - but clears the other envvars. 
That is what I was going on.

> But, more fundamentally, I'm dubious that weakening the environment
> variable set is a good use of anyone's time.  Why not define reproducible
> builds as setting a specific set of environment variables and no others?
> We're long past the point where building packages in an isolated
> environment with a fixed set of environment variables is a great hardship
> or even particularly unusual.  I think the effort would be better spent on
> fixing (with enumerated exceptions) the set of environment variables set
> by buildds, sbuild, pbuilder, and other infrastructure that builds
> packages than in making packages tolerate random environment variables
> being set during the build.  It's really hard to track down all the
> environment variable settings that might affect Autoconf, the build tools,
> document formatters, and so forth.
> 

My proposal was the opposite, to *strengthen* the definition that was already 
accepted - I *don't* think we should track down all those variables and make 
packages immune to them, that is why I added "other variables explicitly used 
by buildsystems to affect build output" etc. OTOH, some other variables are 
used by non-build tools, such as LC_*, USER, etc. Since they affect non-build 
programs, they possibly may be set in a developer's normal environment, so just 
running "debian/rules build" will pick these up. Then, the build should stay 
the same despite these other variables.

If a build tool needs to be run in a specific locale, it should either use a 
locale-independent sorting program, or set LC_ALL explicitly itself regardless 
of what the parent environment says.

This doesn't contradict us from using a fixed or mostly-clean environment in 
sbuild, pbuilder, debuild, etc.

Now that I think about it however, it's probably not reasonable to expect that 
the output remains the same when PATH is changed. On tests.r-b.org we vary it 
by appending a dummy value [1] but if the user adds their own stuff to the 
beginning then the output may well change. There is probably no point in trying 
to prevent that in all packages. In a sense, it does very much affect what 
build tools are run, even though non-build programs also use it. However, my 
gut feeling still says that it's not right for the locale (LC_*) to affect a 
build process. I will try to think of a more precise way to express this 
difference.

X

[1] https://tests.reproducible-builds.org/debian/index_variations.html

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Bill Allombert  writes:
> On Wed, Aug 16, 2017 at 09:36:04AM -0700, Russ Allbery wrote:

>> Note that, for most developers, this is pretty much equivalent to the
>> current situation with FTBFS on, say, s390 architectures.  Or even
>> issues with running under whichever init system is not the one the
>> maintainer personally uses.

> Debian provides porter box for that purpose. This means if your package
> FTBFS on s390 you can login to a s390 porter box, use sbuild to set up a
> build environment, fix the problem and then check the package now build
> correctly.

> Now compare with reproducible build. You get some error report you
> cannot reproduce, do some change following the help provided and hope
> for the best. Then some day later you get the same error report.

This hasn't been my experience with reproducible build bug reports.  Once
there's a bug report of unreproducibility under some specific situation,
I've always been able to reproduce it by doing multiple builds with that
specific variation and seeing how the output changes.

I agree that this may not always be the case, but it's also not always the
case that one can reproduce an s390 buildd failure on a porter box.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Bill Allombert]

On Wed, Aug 16, 2017 at 09:36:04AM -0700, Russ Allbery wrote:
> Note that, for most developers, this is pretty much equivalent to the
> current situation with FTBFS on, say, s390 architectures.  Or even issues
> with running under whichever init system is not the one the maintainer
> personally uses.

Debian provides porter box for that purpose. This means if your package
FTBFS on s390 you can login to a s390 porter box, use sbuild to set up a
build environment, fix the problem and then check the package now build
correctly.

Now compare with reproducible build. You get some error report you
cannot reproduce, do some change following the help provided and
hope for the best. Then some day later you get the same error
report.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Ximin Luo  writes:

> Fair enough. I actually spotted that but thought it was better to get
> "something" into Policy rather than nitpick. I guess other people were
> thinking similar things. Well, lesson learnt, I will be more forceful
> next time.

> The sentence I amended said "most environment variables" so our intent
> is clear. If we want to fix this now, I would suggest amending:

> - a set of environment variable values; and
> + a set of reserved environment variable values; and

> then later:

> + A "reserved" environment variable is defined as DEB_*, DPKG_, 
> SOURCE_DATE_EPOCH, BUILD_PATH_PREFIX_MAP, variables listed by dpkg-buildflags 
> and other variables explicitly used by buildsystems to affect build output, 
> excluding any variables used by non-build programs to affect their behaviour. 
> Explicitly, this excludes TERM, HOME, LOGNAME, USER, PATH and likely any 
> variables ending with *PATH.

We intentionally didn't spell this out in this much detail because it felt
better to defer this (stricter) bar until we have documentation of the
*.buildinfo file, and also because we were worried about the list changing
(once it goes into Policy, it's more irritating to change).  The current
standard in Policy is intentionally weaker than this in order to be
simpler.

I still lean towards taking this approach, because I'm pretty worried
about the scope of:

other variables explicitly used by buildsystems to affect build output

That's not really an enumerable list.  My recommendation, if you want to
allow some environment variables to vary without affecting
reproducibility, is to explicitly list the set of environment variables
that can vary, rather than trying to list the ones that have to remain
fixed.

But, more fundamentally, I'm dubious that weakening the environment
variable set is a good use of anyone's time.  Why not define reproducible
builds as setting a specific set of environment variables and no others?
We're long past the point where building packages in an isolated
environment with a fixed set of environment variables is a great hardship
or even particularly unusual.  I think the effort would be better spent on
fixing (with enumerated exceptions) the set of environment variables set
by buildds, sbuild, pbuilder, and other infrastructure that builds
packages than in making packages tolerate random environment variables
being set during the build.  It's really hard to track down all the
environment variable settings that might affect Autoconf, the build tools,
document formatters, and so forth.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Bill Allombert:
> On Tue, Aug 15, 2017 at 07:49:55PM +, Holger Levsen wrote:
>> Also what you are saying ("a package that is reproducible according to the
>> policy definition must not show up as non-reproducible in tracker/DDPO based
>> on results from the reproducible infrastructure") doesnt really makes sense:
>> if a package shows up as unreproducible somewhere, it's not reproducible
>> according to our definition!
> 
> Again, reproducible means that it _can_ be reproduced. As long as a well-know
> process allows to reproduce the package, it is reproducible.
> 
> What you define is a different concept that deserve a different name.
> 
> Cheers,
> 

I think it is OK to call this "reproducible", it's a natural language word and 
these are always dependent on some context. Technically, everything is 
reproducible if you know the state of the machine when the original build was 
started. Some other projects give you a VM and tell you to build in the VM. 
That would be a "well-known process". But nobody really knows what's in the VM 
so it's not helpful for security. Having a strict definition of 
reproducibility, helps us be more convinced that the build process is really 
only dependent on the source code and build tools, and a very restricted set of 
other inputs.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Bill Allombert  writes:

> I am still concerned that there will be no reliable way for maintainers
> to check whether a package is reproducible according to policy before
> uploading it to the archive.

Ximin answered this, but I also wanted to note that while having such a
tool would be ideal, we don't have such tools for every aspect of Policy,
and I'm generally comfortable with that.  There are a lot of elements in
building a distribution where we can't proactively test exhaustively and
maintainers have to be reactive.  Obviously, full testing in advance is
best, but we can live with some reactive bug fixing.

There is an infrastructure that will test your package for reproducibility
after upload and let you know if that fails, which is better than some
other aspects of Policy.

Note that, for most developers, this is pretty much equivalent to the
current situation with FTBFS on, say, s390 architectures.  Or even issues
with running under whichever init system is not the one the maintainer
personally uses.  Maintainers generally do not proactively test such
things; they follow best practices, use standard tools, and then
reactively respond to bugs filed or by failures detected by other parts of
the infrastructure when those tools fail for some reason.  And generally
that's fine; lots of proactive testing for the maintainer would often be a
waste of their time.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Adrian Bunk:
> On Wed, Aug 16, 2017 at 03:43:00PM +, Ximin Luo wrote:
>> Adrian Bunk:
>>> On Wed, Aug 16, 2017 at 11:37:00AM +, Ximin Luo wrote:
 [..]

 Fair enough. I actually spotted that but thought it was better to get 
 "something" into Policy rather than nitpick. I guess other people were 
 thinking similar things. Well, lesson learnt, I will be more forceful next 
 time.
 ...
>>>
>>> What is the point of getting "something" into policy, when it is known 
>>> to not match existing practice and that what is being added to policy 
>>> will be ignored by everyone?
>>>
 The sentence I amended said "most environment variables" so our intent is 
 clear.
 ...
>>>
>>> This is not about "intent", this is about giving an exact definition
>>> of reproducibility for Debian.
>>>
>>> The definition should then match what is recorded in .buildinfo
>>> and what the reproducible builds infrastructure is testing.
>>>
>>
>> The exact wording that was added, was a too-loose requirement. I'm now 
>> proposing to make the requirement more strict, in accordance with the tests 
>> that we're running. Do you have any comments on my proposal?
>> ...
> 
> Will both dpkg-genbuildinfo and the reproducible builds infrastructure 
> implement this definition?
> 
> Any definition is fine for me as long as it will match what is actually 
> being done.[1]
> 
> [1] not excluding future changes, but at the time the policy will be 
> published it should match reality
> 

dpkg-genbuildinfo isn't about testing reproducibility, it's about recording the 
build environment: 
https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles#Semantics

"A buildinfo file is a record of what a particular builder did to build some 
binary output.

It contains as much information about the build environment as is reasonable to 
distribute, and attempts to include all information needed to reproduce that 
build.

It does NOT imply that everything contained within "should" be necessary to 
reproduce the build. [..]"

AIUI, the tests.reproducible-builds.org infrastructure does already include 
what I suggested to be added. That is, it doesn't report "unreproducible" if 
varying those reserved variables, would change the binary hash. It could be a 
bit clearer in the UI on which suites vary the build-path however, that is an 
improvement to be made. For buster (where the build-path is not varied) then an 
"unreproducible" status would mean it's not meeting the definition (that 
includes my addition of these "reserved variables").

It is true that we might report "reproducible" even if the output varies based 
on the envvar UNIQUERANDOMENVVAR, because we don't explicitly vary that. But a 
lot of other QA-related tests in Debian would give "good" results even for 
"bad" packages that are bad in a very esoteric way. It's the intent that 
matters, and I think that's captured well in the policy, especially with my 
proposed newer wording, and also in the practical tests that we're already 
running.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Wed, Aug 16, 2017 at 03:43:00PM +, Ximin Luo wrote:
> Adrian Bunk:
> > On Wed, Aug 16, 2017 at 11:37:00AM +, Ximin Luo wrote:
> >> [..]
> >>
> >> Fair enough. I actually spotted that but thought it was better to get 
> >> "something" into Policy rather than nitpick. I guess other people were 
> >> thinking similar things. Well, lesson learnt, I will be more forceful next 
> >> time.
> >> ...
> > 
> > What is the point of getting "something" into policy, when it is known 
> > to not match existing practice and that what is being added to policy 
> > will be ignored by everyone?
> > 
> >> The sentence I amended said "most environment variables" so our intent is 
> >> clear.
> >> ...
> > 
> > This is not about "intent", this is about giving an exact definition
> > of reproducibility for Debian.
> > 
> > The definition should then match what is recorded in .buildinfo
> > and what the reproducible builds infrastructure is testing.
> > 
> 
> The exact wording that was added, was a too-loose requirement. I'm now 
> proposing to make the requirement more strict, in accordance with the tests 
> that we're running. Do you have any comments on my proposal?
>...

Will both dpkg-genbuildinfo and the reproducible builds infrastructure 
implement this definition?

Any definition is fine for me as long as it will match what is actually 
being done.[1]

> X

cu
Adrian

[1] not excluding future changes, but at the time the policy will be 
published it should match reality

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Adrian Bunk:
> On Wed, Aug 16, 2017 at 11:37:00AM +, Ximin Luo wrote:
>> [..]
>>
>> Fair enough. I actually spotted that but thought it was better to get 
>> "something" into Policy rather than nitpick. I guess other people were 
>> thinking similar things. Well, lesson learnt, I will be more forceful next 
>> time.
>> ...
> 
> What is the point of getting "something" into policy, when it is known 
> to not match existing practice and that what is being added to policy 
> will be ignored by everyone?
> 
>> The sentence I amended said "most environment variables" so our intent is 
>> clear.
>> ...
> 
> This is not about "intent", this is about giving an exact definition
> of reproducibility for Debian.
> 
> The definition should then match what is recorded in .buildinfo
> and what the reproducible builds infrastructure is testing.
> 

The exact wording that was added, was a too-loose requirement. I'm now 
proposing to make the requirement more strict, in accordance with the tests 
that we're running. Do you have any comments on my proposal?

- a set of environment variable values; and
+ a set of reserved environment variable values; and

then later:

+ A "reserved" environment variable is defined as DEB_*, DPKG_, 
SOURCE_DATE_EPOCH, BUILD_PATH_PREFIX_MAP, variables listed by dpkg-buildflags 
and other variables explicitly used by buildsystems to affect build output, 
excluding any variables used by non-build programs to affect their behaviour. 
Explicitly, this excludes TERM, HOME, LOGNAME, USER, PATH and likely any 
variables ending with *PATH.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Wed, Aug 16, 2017 at 11:37:00AM +, Ximin Luo wrote:
> Adrian Bunk:
> > On Wed, Aug 16, 2017 at 10:24:07AM +, Mattia Rizzolo wrote:
> >> On Tue, 15 Aug 2017, 11:02 p.m. Adrian Bunk  wrote:
> >>
> >>> Tracker:
> >>> https://tracker.debian.org/pkg/hsqldb1.8.0
> >>> "Does not build reproducibly during testing"
> >>
> >> And indeed it's not reproducible according to policy: it's storing the
> >> build user at the very least.
> >> ...
> > 
> > What makes you so confident that this package is not reproducible 
> > according to policy?
> > 
> > According to policy, storing the value of $USER in the binary
> > is clearly permitted for a reproducible package. [1]
> > 
> > As long as the reproducible builds infrastructure varies $USER instead 
> > of following the policy definition, it is not suitable for determining 
> > whether or not a package is reproducible according to policy.
> > 
> > And what the reproducible builds infrastructure pushes as
> >Does not build reproducibly during testing
> > to tracker and DDPO is therefore not usable for determining
> > reproducibility according to policy.
> > 
> > cu
> > Adrian
> > 
> > [1] I haven't checked what exactly this package does
> > 
> 
> Fair enough. I actually spotted that but thought it was better to get 
> "something" into Policy rather than nitpick. I guess other people were 
> thinking similar things. Well, lesson learnt, I will be more forceful next 
> time.
>...

What is the point of getting "something" into policy, when it is known 
to not match existing practice and that what is being added to policy 
will be ignored by everyone?

> The sentence I amended said "most environment variables" so our intent is 
> clear.
>...

This is not about "intent", this is about giving an exact definition
of reproducibility for Debian.

The definition should then match what is recorded in .buildinfo
and what the reproducible builds infrastructure is testing.

> X

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Adrian Bunk:
> On Wed, Aug 16, 2017 at 10:24:07AM +, Mattia Rizzolo wrote:
>> On Tue, 15 Aug 2017, 11:02 p.m. Adrian Bunk  wrote:
>>
>>> Tracker:
>>> https://tracker.debian.org/pkg/hsqldb1.8.0
>>> "Does not build reproducibly during testing"
>>
>> And indeed it's not reproducible according to policy: it's storing the
>> build user at the very least.
>> ...
> 
> What makes you so confident that this package is not reproducible 
> according to policy?
> 
> According to policy, storing the value of $USER in the binary
> is clearly permitted for a reproducible package. [1]
> 
> As long as the reproducible builds infrastructure varies $USER instead 
> of following the policy definition, it is not suitable for determining 
> whether or not a package is reproducible according to policy.
> 
> And what the reproducible builds infrastructure pushes as
>Does not build reproducibly during testing
> to tracker and DDPO is therefore not usable for determining
> reproducibility according to policy.
> 
> cu
> Adrian
> 
> [1] I haven't checked what exactly this package does
> 

Fair enough. I actually spotted that but thought it was better to get 
"something" into Policy rather than nitpick. I guess other people were thinking 
similar things. Well, lesson learnt, I will be more forceful next time.

The sentence I amended said "most environment variables" so our intent is 
clear. If we want to fix this now, I would suggest amending:

- a set of environment variable values; and
+ a set of reserved environment variable values; and

then later:

+ A "reserved" environment variable is defined as DEB_*, DPKG_, 
SOURCE_DATE_EPOCH, BUILD_PATH_PREFIX_MAP, variables listed by dpkg-buildflags 
and other variables explicitly used by buildsystems to affect build output, 
excluding any variables used by non-build programs to affect their behaviour. 
Explicitly, this excludes TERM, HOME, LOGNAME, USER, PATH and likely any 
variables ending with *PATH.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Wed, Aug 16, 2017 at 10:24:07AM +, Mattia Rizzolo wrote:
> On Tue, 15 Aug 2017, 11:02 p.m. Adrian Bunk  wrote:
> 
> > Tracker:
> > https://tracker.debian.org/pkg/hsqldb1.8.0
> > "Does not build reproducibly during testing"
> 
> And indeed it's not reproducible according to policy: it's storing the
> build user at the very least.
>...

What makes you so confident that this package is not reproducible 
according to policy?

According to policy, storing the value of $USER in the binary
is clearly permitted for a reproducible package. [1]

As long as the reproducible builds infrastructure varies $USER instead 
of following the policy definition, it is not suitable for determining 
whether or not a package is reproducible according to policy.

And what the reproducible builds infrastructure pushes as
   Does not build reproducibly during testing
to tracker and DDPO is therefore not usable for determining
reproducibility according to policy.

cu
Adrian

[1] I haven't checked what exactly this package does

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Bill Allombert:
> On Tue, Aug 15, 2017 at 01:00:00PM -0700, Russ Allbery wrote:
>> Adrian Bunk  writes:
>>
>>> Future policy versions might change this definition, but whatever latest
>>> policy states has to be the definition used by both packages and the
>>> reproducible builds team.
>>
>>> Another example is that a package that is reproducible according to the 
>>> policy definition must not show up as non-reproducible in tracker/DDPO 
>>> based on results from the reproducible infrastructure.
>>
>> This in absolutely no way constrains the reproducible build team from
>> working on raising the bar in the future.
> 
> Adrian is speaking of DDPO, not of reproducible-builds.org.
> reproducible-builds.org website woud still be free to list other
> requirements, and DDPO could even display both results.
> 
> I am still concerned that there will be no reliable way for maintainers
> to check whether a package is reproducible according to policy before
> uploading it to the archive.
> 

Did nobody mention

$ reprotest --dont-vary build_path auto xxx.dsc -- schroot unstable-amd64-sbuild

yet?

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Mattia Rizzolo]

On Tue, 15 Aug 2017, 11:02 p.m. Adrian Bunk  wrote:

> Tracker:
> https://tracker.debian.org/pkg/hsqldb1.8.0
> "Does not build reproducibly during testing"
>

And indeed it's not reproducible according to policy: it's storing the
build user at the very least.

>
> Let's look at the mdds package, that has red unreproducible entries in
> the maintainer dashboard:
>
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/mdds.html
>
> mdds is unreproducible only in sid since more things (including the
> build path) are varied there. The information behind "differences"
> confirms that the build path is the only issue.
>
> According to policy, mdds is reproducible.
>

And indeed its unreproducibility is not reported in tracker and ddpo (DMD
does because it's using a source data that includes everything, not just
the state we want to push.  But then, DMD has a tendency to show *lots* of
things, if you disagree with it, please take it to the DMD maintainer, not
us).

Unless policy is supposed to be completely detached from reality,
> the criteria for claiming in various places that a package is
> unreproducible have to match the policy definition of reproducibility.
>

IMHO, you are arguing about a non existent issue.  I believe we are always
being reasonable, otherwise I'd like to ask you to point us to actual
situation where we could have acted better.  Yes, I'm aware of the
src:libreoffice case.

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Tue, Aug 15, 2017 at 04:01:00PM -0700, Sean Whitton wrote:
> On Wed, Aug 16 2017, Adrian Bunk wrote:
> 
> > This is about the reproducible builds team not using policy as a stick 
> > for claiming a bar higher than what policy actually defines.
> >
> > Is it really allowed to claim that a package is not reproducible,
> > when it actually is reproducible according to policy?
> 
> Yes, if your bug is of 'wishlist' severity.

You are not answering to any of the examples I gave in the email you
answered to.

"Does not build reproducibly during testing" statements in Tracker,
and red warnings in DDPO and Maintainer Dashboard.

Is it for you as policy editor perfectly fine or a bug in the 
reproducible infrastructure when Tracker states (based on the
latest results from the reproducible builds infrastructure)
"Does not build reproducibly during testing" for packages
that are reproducible according to policy?

> Sean Whitton

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[Sean Whitton]

On Wed, Aug 16 2017, Adrian Bunk wrote:

> This is about the reproducible builds team not using policy as a stick 
> for claiming a bar higher than what policy actually defines.
>
> Is it really allowed to claim that a package is not reproducible,
> when it actually is reproducible according to policy?

Yes, if your bug is of 'wishlist' severity.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Adrian Bunk  writes:

> This is not about experimenting for raising the bar in the future.

> This is about the reproducible builds team not using policy as a stick 
> for claiming a bar higher than what policy actually defines.

> Is it really allowed to claim that a package is not reproducible,
> when it actually is reproducible according to policy?

Yes.  Ideally one would distinguish between those various definitions of
reproducible, though, and present all of them.

> Unless policy is supposed to be completely detached from reality, the
> criteria for claiming in various places that a package is unreproducible
> have to match the policy definition of reproducibility.

No, I don't agree.  This is not how we do things in Debian.  There is
quite a bit of information that we give developers about possible flaws in
their package, from Lintian and build log analysis and many other things,
that is not required by Policy.  This is no different.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Bill Allombert]

On Tue, Aug 15, 2017 at 01:00:00PM -0700, Russ Allbery wrote:
> Adrian Bunk  writes:
> 
> > Future policy versions might change this definition, but whatever latest
> > policy states has to be the definition used by both packages and the
> > reproducible builds team.
> 
> > Another example is that a package that is reproducible according to the 
> > policy definition must not show up as non-reproducible in tracker/DDPO 
> > based on results from the reproducible infrastructure.
> 
> This in absolutely no way constrains the reproducible build team from
> working on raising the bar in the future.

Adrian is speaking of DDPO, not of reproducible-builds.org.
reproducible-builds.org website woud still be free to list other
requirements, and DDPO could even display both results.

I am still concerned that there will be no reliable way for maintainers
to check whether a package is reproducible according to policy before
uploading it to the archive.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#844431: Revised patch: seeking seconds

[Bill Allombert]

On Tue, Aug 15, 2017 at 07:49:55PM +, Holger Levsen wrote:
> Also what you are saying ("a package that is reproducible according to the
> policy definition must not show up as non-reproducible in tracker/DDPO based
> on results from the reproducible infrastructure") doesnt really makes sense:
> if a package shows up as unreproducible somewhere, it's not reproducible
> according to our definition!

Again, reproducible means that it _can_ be reproduced. As long as a well-know
process allows to reproduce the package, it is reproducible.

What you define is a different concept that deserve a different name.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Tue, Aug 15, 2017 at 01:00:00PM -0700, Russ Allbery wrote:
>...
> This in absolutely no way constrains the reproducible build team from
> working on raising the bar in the future, just as the absence of this
> language from Policy did not prevent them from starting to work on this
> problem four years ago.  They should continue to work on making package
> builds more reproducible and raising the bar for reproducibility as makes
> sense for their goals and judging the impact of that.  Once any new
> requirements reach maturity and look feasible and have some project
> committment, we'll change Policy to set a new baseline for the whole
> project.  But the reproducible builds work should not *wait* for that, and
> should definitely push forward and experiment just as they have up until
> now.
>...

This is not about experimenting for raising the bar in the future.

This is about the reproducible builds team not using policy as a stick 
for claiming a bar higher than what policy actually defines.

Is it really allowed to claim that a package is not reproducible,
when it actually is reproducible according to policy?

Let me explain with examples how this information is presented 
to maintainers:

Tracker:
https://tracker.debian.org/pkg/hsqldb1.8.0
"Does not build reproducibly during testing"

DDPO:
https://qa.debian.org/developer.php?email=debian-openoff...@lists.debian.org
red "unrep" entries

Maintainer dashboard:
https://udd.debian.org/dmd/?email1=debian-openoffice%40lists.debian.org
red "(un)reproducible" entries [1]

Let's look at the mdds package, that has red unreproducible entries in
the maintainer dashboard:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/mdds.html

mdds is unreproducible only in sid since more things (including the 
build path) are varied there. The information behind "differences"
confirms that the build path is the only issue.

According to policy, mdds is reproducible.

Unless policy is supposed to be completely detached from reality,
the criteria for claiming in various places that a package is 
unreproducible have to match the policy definition of reproducibility.

cu
Adrian

[1] the FTBFS entries are actually problems in the reproducible infrastructure

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[Sean Whitton]

Hello,

On Tue, Aug 15 2017, Russ Allbery wrote:

> Adrian Bunk  writes:
>
>> Future policy versions might change this definition, but whatever
>> latest policy states has to be the definition used by both packages
>> and the reproducible builds team.
>
>> Another example is that a package that is reproducible according to
>> the policy definition must not show up as non-reproducible in
>> tracker/DDPO based on results from the reproducible infrastructure.
>
> This seems really inflexible and unnecessarily absolutist.  I don't
> agree with taking this approach.
>
> The point of adding this definition to Policy is that we're setting a
> new minimum bar for packages in Debian to meet.

Right.  More generally, the only thing that this new section of policy
constrains is the severities that may be used when filing bugs.  This is
how policy changes are meant to work -- they control bug severities
against packages, not what appears on tracking web pages.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Adrian Bunk  writes:

> Future policy versions might change this definition, but whatever latest
> policy states has to be the definition used by both packages and the
> reproducible builds team.

> Another example is that a package that is reproducible according to the 
> policy definition must not show up as non-reproducible in tracker/DDPO 
> based on results from the reproducible infrastructure.

This seems really inflexible and unnecessarily absolutist.  I don't agree
with taking this approach.

The point of adding this definition to Policy is that we're setting a new
minimum bar for packages in Debian to meet.  We're giving official
blessing to this requirement for Debian packages (at the normal bug level,
not RC bug, for now), meaning this is a goal that the project is working
towards and something every packager should think about at this level.

This in absolutely no way constrains the reproducible build team from
working on raising the bar in the future, just as the absence of this
language from Policy did not prevent them from starting to work on this
problem four years ago.  They should continue to work on making package
builds more reproducible and raising the bar for reproducibility as makes
sense for their goals and judging the impact of that.  Once any new
requirements reach maturity and look feasible and have some project
committment, we'll change Policy to set a new baseline for the whole
project.  But the reproducible builds work should not *wait* for that, and
should definitely push forward and experiment just as they have up until
now.

I do think it might be worth considering distinguishing between packages
that are minimally reproducible and packages that meet higher
reproducibility bars (such as not caring about the location of the build
tree) in reporting infrastructure like tracker.  But I'm totally fine with
surfacing failures on new, higher bars in places like tracker before we
change Policy, just like we've been surfacing reproducibility failures
before Policy said anything about it at all.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Holger Levsen]

On Tue, Aug 15, 2017 at 09:05:29PM +0300, Adrian Bunk wrote:
> Is identical building on any kernel required (and tested)?

no and no.

it's only required that the results is reproducible, that is bit by bit 
identical…
 
> Will every reproducible package in buster build identical on the
> bullseye+1 kernel 2022.11.321 ? [1]

my crystal ball is broken, sorry…
 
> [1] the wheezy LTS updates are now built on buildds running stretch
> kernels, and in buster we will have the similar situation that
> nearly everyting in the initial release will be built on stretch
> kernels while post-release updates will be built on buster,
> bullseye and bullseye+1 kernels
 
there surely are situations where different kernels (much like different
libraries) will cause variations in the build artifacts. many times it
will not matter and sometimes it will and we don't really have much experience
with that *yet*.

I'm inclined to file a bug report against dpkg to document the kernel 
used in the .buildinfo files and (hereby) am asking the reproducible builds team
for comments / advice on this.

And probably we should amend debian-policy for this too, but I also 
think we'd rather do this via a new bug report at some later point in time.

Thanks, Adrian, for making sure we don't forget (this pretty old aspect).


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Bug#844431: Revised patch: seeking seconds

[Holger Levsen]

On Tue, Aug 15, 2017 at 10:09:30PM +0300, Adrian Bunk wrote:
> > > I would expect the reproducible builds team to not submit any bugs
> > > regarding varied environment variables as long as as the official
> > > definition of reproducibility in policy states that this is not required
> > > for a package to be reproducible.

I believe we'll continue to file sensible bug reports like we have done in
the last four years.

> Another example is that a package that is reproducible according to the 
> policy definition must not show up as non-reproducible in tracker/DDPO 
> based on results from the reproducible infrastructure.
 
I disagree that we should modfiy the results of actual tests based on wishful
thinking or some definition somewhere, even if it's our beloved debian-policy.

It's certainly possible that a package becomes unreproducible, for known or
unknown reasons (hopefully by now we understand most of them (or have the means
to find out)), at any point in time. And then tracker/DDPO should certainly
show that…

Also what you are saying ("a package that is reproducible according to the
policy definition must not show up as non-reproducible in tracker/DDPO based
on results from the reproducible infrastructure") doesnt really makes sense:
if a package shows up as unreproducible somewhere, it's not reproducible
according to our definition!


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Tue, Aug 15, 2017 at 11:49:22AM -0700, Russ Allbery wrote:
> Adrian Bunk  writes:
> 
> > I would expect the reproducible builds team to not submit any bugs
> > regarding varied environment variables as long as as the official
> > definition of reproducibility in policy states that this is not required
> > for a package to be reproducible.
> 
> I believe the planned next step here is to publish the *.buildinfo files,
> which contain a specification of the environment variables the build cares
> about, and then Policy can be modified to include a description of
> *.buildinfo files and how to use them.  As part of those changes, we'd
> certainly update the definition of reproducible to reference matching the
> environment specified in the corresponding *.buildinfo file.

I do understand that.

My point is that we now have an official definition what is required
for a package to be reproducible, and what is not required.

Future policy versions might change this definition,
but whatever latest policy states has to be the definition
used by both packages and the reproducible builds team.

Another example is that a package that is reproducible according to the 
policy definition must not show up as non-reproducible in tracker/DDPO 
based on results from the reproducible infrastructure.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Adrian Bunk  writes:

> I would expect the reproducible builds team to not submit any bugs
> regarding varied environment variables as long as as the official
> definition of reproducibility in policy states that this is not required
> for a package to be reproducible.

I believe the planned next step here is to publish the *.buildinfo files,
which contain a specification of the environment variables the build cares
about, and then Policy can be modified to include a description of
*.buildinfo files and how to use them.  As part of those changes, we'd
certainly update the definition of reproducible to reference matching the
environment specified in the corresponding *.buildinfo file.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Sat, Aug 12, 2017 at 03:34:35PM -0700, Sean Whitton wrote:
>...
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
>...

Is identical building on any kernel required (and tested)?

Examples:

A self-compiled kernel with CONFIG_IPV6=n

Imagine the next time Linus changes the kernel versioning,
he chooses ..
Will every reproducible package in buster build identical on the
bullseye+1 kernel 2022.11.321 ? [1]

> Sean Whitton

cu
Adrian

[1] the wheezy LTS updates are now built on buildds running stretch
kernels, and in buster we will have the similar situation that
nearly everyting in the initial release will be built on stretch
kernels while post-release updates will be built on buster,
bullseye and bullseye+1 kernels

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[Adrian Bunk]

On Sat, Aug 12, 2017 at 11:23:14AM -0700, Sean Whitton wrote:
>...
> - for now, we only require reproducibility when the set of environment
>   variable values set is exactly the same
> 
>   This is because
> 
>   - the reproducible builds team aren't yet totally clear on the
> variables that they think may be allowed to vary
> 
>   - we should wait until .buildinfo is properly documented in policy,
> and then we can refer to that file
>...

I would expect the reproducible builds team to not submit any bugs 
regarding varied environment variables as long as as the official
definition of reproducibility in policy states that this is not
required for a package to be reproducible.

> Sean Whitton

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#844431: Revised patch: seeking seconds

[gregor herrmann]

On Sat, 12 Aug 2017 15:34:35 -0700, Sean Whitton wrote:

> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or 
> build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
> See the file ``upgrading-checklist`` for information about policy
> which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
> often creates either static linking or shared library conflicts, and,
> most importantly, increases the difficulty of handling security
> vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition `_.


Seconded.

Thanks to everyone for their work on this.


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at/ - Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   


signature.asc
Description: Digital Signature

Bug#844431: Revised patch: seeking seconds

[Holger Levsen]

On Sat, Aug 12, 2017 at 03:34:35PM -0700, Sean Whitton wrote:
> Here is an updated patch addressing these.  I reworded it to use
> 'recommended' and changed the tone to better suit policy.
> 
> Thank you Ximin, Russ and Johannes!
> 
> > "precisification" -> "more precise version"
> 
> Our definition is not actually a /version/ of the
> reproducible-builds.org definition -- that would imply that our
> definition could replace the reproducible-builds.org definition, like
> upgrading a package.
> 
> 'precisification' means roughly "filling out the missing specification
> when it is appropriate to fill it out", which is what the r-p.org
> definition instructs distributors to do.
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or 
> build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
> See the file ``upgrading-checklist`` for information about policy
> which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
> often creates either static linking or shared library conflicts, and,
> most importantly, increases the difficulty of handling security
> vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition `_.

seconded & thanks for these improvements!


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Bug#844431: Revised patch: seeking seconds

[Sean Whitton]

On Sat, Aug 12 2017, Ximin Luo wrote:

> Thanks! Seconded.

Just to be clear, we are waiting on one more second for the version
that refers to build and target architecture.

-- 
Sean Whitton

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Sean Whitton:
> [..]
> 
> Here is an updated patch addressing these.  I reworded it to use
> 'recommended' and changed the tone to better suit policy.
> 
> Thank you Ximin, Russ and Johannes!
> 
>> "precisification" -> "more precise version"
> 
> Our definition is not actually a /version/ of the
> reproducible-builds.org definition -- that would imply that our
> definition could replace the reproducible-builds.org definition, like
> upgrading a package.
> 
> 'precisification' means roughly "filling out the missing specification
> when it is appropriate to fill it out", which is what the r-p.org
> definition instructs distributors to do.
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or 
> build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
> See the file ``upgrading-checklist`` for information about policy
> which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
> often creates either static linking or shared library conflicts, and,
> most importantly, increases the difficulty of handling security
> vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition `_.
> 
> 

Thanks! Seconded.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Sean Whitton]

Hello,

On Sat, Aug 12 2017, Russ Allbery wrote:

> I suspect we want to say build and host architecture for right now.
> (Maybe we can later aspire to making the build architecture not
> matter.)

On Sat, Aug 12 2017, Ximin Luo wrote:

> To echo dkg and others' comments, it would be nice if we could add
> here:
>
> +Packages are encouraged to produce bit-for-bit identical binary
> packages even +if most environment variables and build paths are
> varied. This is technically +more difficult at the time of writing,
> but it is intended that this stricter +definition would replace the
> above one, when appropriate in the future.

Here is an updated patch addressing these.  I reworded it to use
'recommended' and changed the tone to better suit policy.

Thank you Ximin, Russ and Johannes!

> "precisification" -> "more precise version"

Our definition is not actually a /version/ of the
reproducible-builds.org definition -- that would imply that our
definition could replace the reproducible-builds.org definition, like
upgrading a package.

'precisification' means roughly "filling out the missing specification
when it is appropriate to fill it out", which is what the r-p.org
definition instructs distributors to do.

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index 127b125..6e32870 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or build 
system (for
 example, a package that builds the same source multiple times to
 generate different binary packages).
 
+Reproducibility
+---
+
+Packages should build reproducibly, which for the purposes of this
+document [#]_ means that given
+
+- a version of a source package unpacked at a given path;
+- a set of versions of installed build dependencies;
+- a set of environment variable values;
+- a build architecture; and
+- a host architecture,
+
+repeatedly building the source package for the build architecture on
+any machine of the host architecture with those versions of the build
+dependencies installed and exactly those environment variable values
+set will produce bit-for-bit identical binary packages.
+
+It is recommended that packages produce bit-for-bit identical binaries
+even if most environment variables and build paths are varied.  It is
+intended for this stricter standard to replace the above when it is
+easier for packages to meet it.
+
 .. [#]
See the file ``upgrading-checklist`` for information about policy
which has changed between different versions of this document.
@@ -790,3 +812,7 @@ generate different binary packages).
often creates either static linking or shared library conflicts, and,
most importantly, increases the difficulty of handling security
vulnerabilities in the duplicated code.
+
+.. [#]
+   This is Debian's precisification of the `reproducible-builds.org
+   definition `_.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Johannes Schauer  writes:

> Policy §4.9 defines "build architecture" in the context of
> dpkg-architecture already and I think what you mean here is either "host
> architecture" or at least "build and host architecture" or you need to
> mention that you are only talking about native builds where build and
> host architecture are equal.

I suspect we want to say build and host architecture for right now.
(Maybe we can later aspire to making the build architecture not matter.)

Thanks, good catch!

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Johannes Schauer]

Hi,

Quoting Sean Whitton (2017-08-13 03:23:14)
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,

Policy §4.9 defines "build architecture" in the context of dpkg-architecture
already and I think what you mean here is either "host architecture" or at
least "build and host architecture" or you need to mention that you are only
talking about native builds where build and host architecture are equal.

Thanks!

cheers, josch


signature.asc
Description: signature

Bug#844431: Revised patch: seeking seconds

[Holger Levsen]

On Sat, Aug 12, 2017 at 01:18:23PM -0700, Russ Allbery wrote:
> > +Packages are encouraged to produce bit-for-bit identical binary packages 
> > even
> > +if most environment variables and build paths are varied. This is 
> > technically
> > +more difficult at the time of writing, but it is intended that this 
> > stricter
> > +definition would replace the above one, when appropriate in the future.
> 
> > If this type of "intent" wording is not appropriate for Policy then
> > disregard what I'm saying, I don't wish to block this patch for this
> > reason.
> 
> Oh, that's a good way to capture that.  This seems fine to me, and I have
> no objections to adding this advice.  Seconded the original with or
> without this addition.
 
I'm also seconding the original with or without this addition.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Ximin Luo  writes:

> To echo dkg and others' comments, it would be nice if we could add here:

> +Packages are encouraged to produce bit-for-bit identical binary packages even
> +if most environment variables and build paths are varied. This is technically
> +more difficult at the time of writing, but it is intended that this stricter
> +definition would replace the above one, when appropriate in the future.

> If this type of "intent" wording is not appropriate for Policy then
> disregard what I'm saying, I don't wish to block this patch for this
> reason.

Oh, that's a good way to capture that.  This seems fine to me, and I have
no objections to adding this advice.  Seconded the original with or
without this addition.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Ximin Luo]

Sean Whitton:
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..cc4b020 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or 
> build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,
> +
> +repeatedly building the source package on any machine of the same
> +architecture with those versions of the build dependencies installed
> +and exactly those environment variable values set will produce
> +bit-for-bit identical binary packages.
> +

To echo dkg and others' comments, it would be nice if we could add here:

+Packages are encouraged to produce bit-for-bit identical binary packages even
+if most environment variables and build paths are varied. This is technically
+more difficult at the time of writing, but it is intended that this stricter
+definition would replace the above one, when appropriate in the future.

If this type of "intent" wording is not appropriate for Policy then disregard 
what I'm saying, I don't wish to block this patch for this reason.

>  .. [#]
> See the file ``upgrading-checklist`` for information about policy
> which has changed between different versions of this document.
> @@ -790,3 +806,7 @@ generate different binary packages).
> often creates either static linking or shared library conflicts, and,
> most importantly, increases the difficulty of handling security
> vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition `_.
> 

"precisification" -> "more precise version"

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Bug#844431: Revised patch: seeking seconds

[Russ Allbery]

Sean Whitton  writes:

> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..cc4b020 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or 
> build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,
> +
> +repeatedly building the source package on any machine of the same
> +architecture with those versions of the build dependencies installed
> +and exactly those environment variable values set will produce
> +bit-for-bit identical binary packages.
> +
>  .. [#]
> See the file ``upgrading-checklist`` for information about policy
> which has changed between different versions of this document.
> @@ -790,3 +806,7 @@ generate different binary packages).
> often creates either static linking or shared library conflicts, and,
> most importantly, increases the difficulty of handling security
> vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition `_.

Seconded.

-- 
Russ Allbery (r...@debian.org)   

Bug#844431: Revised patch: seeking seconds

[Ondrej Novy]

Hi,

2017-08-12 14:23 GMT-04:00 Sean Whitton :

> control: tag -1 +patch
>
> This patch incorporates the feedback given on the proposal I sent
> yesterday, both in this bug and in person from Russ and Holger (thank
> you to all).
>

seconded, thanks for working on this.

-- 
Best regards
 Ondřej Nový

Email: n...@ondrej.org
PGP: 3D98 3C52 EB85 980C 46A5  6090 3573 1255 9D1E 064B

Bug#844431: Revised patch: seeking seconds

[Holger Levsen]

On Sat, Aug 12, 2017 at 11:23:14AM -0700, Sean Whitton wrote:
> I am seeking formal seconds for this patch, from any DD.
> 
> In particular:
> 
> - for now, we only require reproducibility when the set of environment
>   variable values set is exactly the same
> 
>   This is because
> 
>   - the reproducible builds team aren't yet totally clear on the
> variables that they think may be allowed to vary
> 
>   - we should wait until .buildinfo is properly documented in policy,
> and then we can refer to that file
> 
> - we don't require reproducibility when build paths vary
> 
>   This is because
> 
>   - since there is not a consensus on whether we should require this,
> and there is strong consensus on the requirement of reproducibility
> if the path does /not/ vary, this issue should not block this change.
> We should open a separate bug against debian-policy
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..cc4b020 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or 
> build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,
> +
> +repeatedly building the source package on any machine of the same
> +architecture with those versions of the build dependencies installed
> +and exactly those environment variable values set will produce
> +bit-for-bit identical binary packages.
> +
>  .. [#]
> See the file ``upgrading-checklist`` for information about policy
> which has changed between different versions of this document.
> @@ -790,3 +806,7 @@ generate different binary packages).
> often creates either static linking or shared library conflicts, and,
> most importantly, increases the difficulty of handling security
> vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition `_.

very happily seconded, many thanks to everyone who has contributed to this bug
directly or "indirectly" (I'm thinking specifically about Lunar here).


-- 
cheers,
Holger (who watched 
http://meetings-archive.debian.net/pub/debian-meetings/2017/debconf17/reproducible-builds-status-update.vp8.webm
 today and was equally happy when seeing the whole audience agreeing this 
should be in policy - and the applause after Russ's closing statement was also 
very very nice…!)


signature.asc
Description: Digital signature

Bug#844431: Revised patch: seeking seconds

[Sean Whitton]

control: tag -1 +patch

This patch incorporates the feedback given on the proposal I sent
yesterday, both in this bug and in person from Russ and Holger (thank
you to all).

I am seeking formal seconds for this patch, from any DD.

In particular:

- for now, we only require reproducibility when the set of environment
  variable values set is exactly the same

  This is because

  - the reproducible builds team aren't yet totally clear on the
variables that they think may be allowed to vary

  - we should wait until .buildinfo is properly documented in policy,
and then we can refer to that file

- we don't require reproducibility when build paths vary

  This is because

  - since there is not a consensus on whether we should require this,
and there is strong consensus on the requirement of reproducibility
if the path does /not/ vary, this issue should not block this change.
We should open a separate bug against debian-policy

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index 127b125..cc4b020 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or build 
system (for
 example, a package that builds the same source multiple times to
 generate different binary packages).
 
+Reproducibility
+---
+
+Packages should build reproducibly, which for the purposes of this
+document [#]_ means that given
+
+- a version of a source package unpacked at a given path;
+- a set of versions of installed build dependencies;
+- a set of environment variable values; and
+- a build architecture,
+
+repeatedly building the source package on any machine of the same
+architecture with those versions of the build dependencies installed
+and exactly those environment variable values set will produce
+bit-for-bit identical binary packages.
+
 .. [#]
See the file ``upgrading-checklist`` for information about policy
which has changed between different versions of this document.
@@ -790,3 +806,7 @@ generate different binary packages).
often creates either static linking or shared library conflicts, and,
most importantly, increases the difficulty of handling security
vulnerabilities in the duplicated code.
+
+.. [#]
+   This is Debian's precisification of the `reproducible-builds.org
+   definition `_.

-- 
Sean Whitton


signature.asc
Description: PGP signature