Re: Moving towards a deb-buildinfo(5) Format 1.0

2017-03-11 Thread Mattia Rizzolo
On Sat, Mar 11, 2017 at 03:33:05AM +0100, Guillem Jover wrote:
> I forgot to mention two curretly pending issues, not listed previously:
> 
> * An error in dpkg-genbuildinfo caused by arch-qualified dependencies
>   on a virtual package.
> * Broken dependency recursor in dpkg-genbuildinfo

I suppose there are tracking bugs around for these.

> > So given the above, I've queued a minimal change declaring the format
> > 1.0 for dpkg 1.18.23 or .24, please shout if you see any additional
> > problem or blocker.
> 
> This has happened now, given that the above are implementation details,
> and do not really affect the format.

Thank you for all your work!

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2017-03-10 Thread Guillem Jover
Hi again!

[ I just had this drafted around, and though I just update and send it
  out as a status update. ]

On Sun, 2017-02-19 at 18:52:16 +0100, Guillem Jover wrote:
> On Sat, 2016-11-12 at 19:04:53 +0100, Guillem Jover wrote:
> > As I've mentioned elsewhere, I've noticed several things with the
> > current .buildinfo format, even after the cleanup pre-merge, that
> > I'd like to fix or change so that we can hopefully reach Format 1.0.
> 
> Ok, let's see what's the current status:

[…]

I forgot to mention two curretly pending issues, not listed previously:

* An error in dpkg-genbuildinfo caused by arch-qualified dependencies
  on a virtual package.

  This was in dpkg 1.18.23. An implementation detail though, even if
  unfortunate.

* Broken dependency recursor in dpkg-genbuildinfo

  The dependency traversal in dpkg-genbuildinfo is pretty much broken
  for many/most Multi-Arch cases. But the same applies to
  dpkg-checkbuilddeps and the Dpkg::Deps perl module. This needs major
  rework and is in a way a pre-existing problem, and in any case an
  implementation detail.

> > I'll probably do some of the fixes already and bump Format to 0.2
> > and after the discussion settles we can perhaps do a 0.3, and see how
> > it goes, and iterate until it looks good, at which point we'd declare
> > it 1.0, ideally before the freeze. :)
> 
> So given the above, I've queued a minimal change declaring the format
> 1.0 for dpkg 1.18.23 or .24, please shout if you see any additional
> problem or blocker.

This has happened now, given that the above are implementation details,
and do not really affect the format.

Thanks,
Guillem

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2017-02-26 Thread James McCoy
On Sun, Feb 19, 2017 at 06:52:16PM +0100, Guillem Jover wrote:
> Hi!
> 
> On Sat, 2016-11-12 at 19:04:53 +0100, Guillem Jover wrote:
> > As I've mentioned elsewhere, I've noticed several things with the
> > current .buildinfo format, even after the cleanup pre-merge, that
> > I'd like to fix or change so that we can hopefully reach Format 1.0.
> 
> Ok, let's see what's the current status:
> 
> > Some of the issues, that bother me:
> > 
> > * .buildinfo files are not currently signed
> 
> Fixed. Pending debsign (from devscipts) doing the same.

Merged this tonight, but still need to get release team approval to make
it into Stretch.  Additional changes would be necessary if the declared
Format changes, since debsign takes precautions about acting on a newer
Format than it is known to grok.

Cheers,
-- 
James
GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7  2D23 DFE6 91AE 331B A3DB

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds


Re: Moving towards a deb-buildinfo(5) Format 1.0

2017-02-20 Thread Daniel Kahn Gillmor
On Sun 2017-02-19 16:34:45 -0500, Guillem Jover wrote:
> On Sun, 2017-02-19 at 11:39:28 -0800, Vagrant Cascadian wrote:
>> On 2017-02-19, Guillem Jover wrote:
>> >> * .buildinfo files are not generated when creating source-only uploads
>> >
>> > Fixed. Now always generated.
>> 
>> On a related note, is it currently possible to create a .buildinfo with
>> both the source and binary, but a corresponding .changes with only the
>> source?
>
> Yes, in the same way it's been possible to do so for source-only
> uploads up-to-now when using just dpkg-buildpackage:
>
>   $ dpkg-buildpackage --changes-option=-S

Thanks for this note!  this is very useful, and i wish i'd known about
it long ago.  It would have been part of my usual invocation already.

Many thanks for pushing buildinfo this far into dpkg, i'm excited to see
what we can do with it in the future.

 --dkg


signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2017-02-19 Thread Guillem Jover
On Sun, 2017-02-19 at 11:39:28 -0800, Vagrant Cascadian wrote:
> On 2017-02-19, Guillem Jover wrote:
> >> * .buildinfo files are not generated when creating source-only uploads
> >
> > Fixed. Now always generated.
> 
> On a related note, is it currently possible to create a .buildinfo with
> both the source and binary, but a corresponding .changes with only the
> source?

Yes, in the same way it's been possible to do so for source-only
uploads up-to-now when using just dpkg-buildpackage:

  $ dpkg-buildpackage --changes-option=-S

Thanks,
Guillem

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds


Re: Moving towards a deb-buildinfo(5) Format 1.0

2017-02-19 Thread Vagrant Cascadian
On 2017-02-19, Guillem Jover wrote:
>> * .buildinfo files are not generated when creating source-only uploads
>
> Fixed. Now always generated.

On a related note, is it currently possible to create a .buildinfo with
both the source and binary, but a corresponding .changes with only the
source?

This would really facilitate source-only uploads, but with binaries
listed in the .buildinfo produced by the uploader.

Something similar is possible with sbuild's --source-only-changes
option, where it creates both a *_source.changes and appropriate
*_ARCH.changes. It would be nice to have a similar feature for
.buildinfo.


Thanks everyone for getting dpkg support for .buiildinfo this far!


live well,
  vagrant


signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2017-02-19 Thread Guillem Jover
Hi!

On Sat, 2016-11-12 at 19:04:53 +0100, Guillem Jover wrote:
> As I've mentioned elsewhere, I've noticed several things with the
> current .buildinfo format, even after the cleanup pre-merge, that
> I'd like to fix or change so that we can hopefully reach Format 1.0.

Ok, let's see what's the current status:

> Some of the issues, that bother me:
> 
> * .buildinfo files are not currently signed

Fixed. Pending debsign (from devscipts) doing the same.

> * .buildinfo filename

Fixed. Now they use the same format as .changes files.

> * dpkg-genbuildinfo injects itself into debian/files

I think this is fine for now, we can always revisit later on. In any
case this is an implementation detail not affecting the .buildinfo
format.

> * .buildinfo files are not generated when creating source-only uploads

Fixed. Now always generated.

> * .buildinfo has some issues when including .dsc information
> 
>   Only the .dsc file is referenced not all of its contents, it might
>   be better to match .changes logic here. Also the “source”
>   pseudo-architecture does not get added to the Architecture field.
>   I'll just do at least the latter, I'm open for discussion on the
>   former.

Partially fixed. The source is now included in the Architecture
field. The inclusion is not recursive, but I think this is fine,
and we should be able to add them if we deem it important in the
future w/o breaking the format.

> * Some of the environment variables seem superfluous or leaks

Ignored. This is valuable information, so it seems fine. Also new
variables are alos tracked now.

> I'll probably do some of the fixes already and bump Format to 0.2
> and after the discussion settles we can perhaps do a 0.3, and see how
> it goes, and iterate until it looks good, at which point we'd declare
> it 1.0, ideally before the freeze. :)

So given the above, I've queued a minimal change declaring the format
1.0 for dpkg 1.18.23 or .24, please shout if you see any additional
problem or blocker.

Thanks,
Guillem

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-14 Thread Guillem Jover
Hi!

On Sun, 2016-11-13 at 14:21:45 +0100, Johannes Schauer wrote:
> Also see:
> 
> https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles#Semantics
> 
> I've heard many upstream developers who were initially very much against
> purging the timestamp when the build was done from their build artifacts
> because they valued the information of when a build was done (whatever their
> reasons are). So this information could simply be retained in that field in 
> the
> .buildinfo file.

I've always claimed that myself, and that was one of the reasons I was
reluctant to eliminate the date from the ar containers, I guess at the
time I could not fully express concretely my gut feeling, but now I
can. :)

The build date is important, because there are actions and events that
are time-based, but are still external to the confinement of the build
environment.

Say, a disk failure corrupting data on the chroot; a broken
debootstrap creating disfunctional chroots, etc, etc. Some of those
might not be immediately visible inside the affected system. But once
known it is useful to be able to say which packages might be suspect
by matching the event date ranges. Of course if the builds end up not
matching other reproducible artifacts then those will be suspect, but
if all reproducers have built using the same external event generator
then that might be harder to see. :)

Thanks,
Guillem

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds


Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-14 Thread Holger Levsen
On Mon, Nov 14, 2016 at 05:44:22AM +0900, Daniel Kahn Gillmor wrote:
> >> Multiple builds of the same source package will set SOURCE_DATE_EPOCH to
> >> the same value but will result in a different Build-Date.
> It is definitely not what most of us initially expected, but it is
> actually what we want.
[...] 
> In short, we *want* buildinfos to vary, while we want the generated
> binary artifacts to be reproducible.

well. our reasoning a year ago for identical buildinfo files (for
different builds of the same package) was the idea, that multiple people
could sign these buildinfo files to confirm they could reproduce these
builds.

having different buildinfo files to confirm identical builds makes
confirming a bit harder.

OTOH this will safe us from dealing with detached signatures as all
buildinfo files can just be signed inline.


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread Johannes Schauer
Hi,

Quoting Daniel Kahn Gillmor (2016-11-13 21:44:22)
> It is definitely not what most of us initially expected, but it is
> actually what we want.
> 
> i look at it this way:
> 
>  * Ideally, the generated binary packages are reproducible *even when
>the build environment changes*.  For example, I build a package as
>the user "dkg" on machine "alice" in path /home/dkg/src/foo, and you
>build it as "lamby" on machine "bob" in path
>/home/lamby/work/foo/foo, and we should get the same outcome.
> 
>  * The buildinfo file documents things that *might* influence the build,
>but it also documents things that *should not* influence the build.
>Two differing buildinfo files that produced the same output
>effectively say "even when the build environment varies in the way
>that these two do, the package is still reproducible"

another thing to think about: Thinking "lets remove everything that should not
influence the build from the buildinfo" is futile in the first place. The most
obvious example is the "Installed-Build-Depends" field. The build output should
probably not depend on many of the Essential:yes packages like bash, sed or
gzip. Still we list these packages in the .buildinfo. In fact, we cannot know
whether or not the build is independent of the bash, sed or gzip package
versions (or architectures) if we would *not* list them. So look at it this
way:

By listing lots of extra information like package versions, build time and
build path we store information that we can later use to identify whether or
not the artifacts produced by a bit are independent from them. If we have two
buildinfo files with different build time but the same hashes, then we can
easily conclude that apparently the build is invariant of the time. If we have
two buildinfo files with different dpkg versions but the same hashes then we
can conclude that apparently, these two dpkg versions have the same effect on
the build. Without listing this info, we cannot make this inferences.

Of course this does not mean that we include *all* the happenstance information
in a buildinfo file. We don't include the username or the specs of the machine
the build was run on. But we still include the interesting information. What
constitutes "interesting" is of course up to debate but by the arguments that
were brought up in earlier mails, I'd argue that datapoints like build date and
build path are still "interesting" pieces of information to keep around for
now.

Thanks!

cheers, josch


signature.asc
Description: signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread Daniel Kahn Gillmor
On Sun 2016-11-13 22:33:29 +0900, Chris Lamb wrote:

>> Multiple builds of the same source package will set SOURCE_DATE_EPOCH to
>> the same value but will result in a different Build-Date.
>
> … but that would mean that a reproducible build will result in .buildinfo
> files with different contents (varying on Build-Date).
>
> That seems, at the very least, somewhat non-intuitive to me.

It is definitely not what most of us initially expected, but it is
actually what we want.

i look at it this way:

 * Ideally, the generated binary packages are reproducible *even when
   the build environment changes*.  For example, I build a package as
   the user "dkg" on machine "alice" in path /home/dkg/src/foo, and you
   build it as "lamby" on machine "bob" in path
   /home/lamby/work/foo/foo, and we should get the same outcome.

 * The buildinfo file documents things that *might* influence the build,
   but it also documents things that *should not* influence the build.
   Two differing buildinfo files that produced the same output
   effectively say "even when the build environment varies in the way
   that these two do, the package is still reproducible"

 * We actually don't want people to have to replicate the exact build
   environment to get a binary match.  I think it was Ximin who pointed
   out: "all software is reproducible if you create an exact
   atom-by-atom copy of the original build computer before building".
   But that's not what we really mean by reproducible builds.

In short, we *want* buildinfos to vary, while we want the generated
binary artifacts to be reproducible.

   --dkg


signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread HW42
Chris Lamb:
> Hey Johannes,
> 
>> Multiple builds of the same source package will set SOURCE_DATE_EPOCH to
>> the same value but will result in a different Build-Date.
> 
> … but that would mean that a reproducible build will result in .buildinfo
> files with different contents (varying on Build-Date).

A .buildinfo file documents the build and is not expected to be
identical between different builds (see also Josch's link). For example
when using sbuild you will always get a different Build-Path if you use
the default settings (and this should be fine).

> That seems, at the very least, somewhat non-intuitive to me.

Yes ;]

> A case might even be made that varying on Build-Date makes our distribution
> problem more difficult; as the files aren't identical we can't easily
> "de-duplicate" them with detached signatures. Perhaps I'm missing something
> obvious.

As described above that's by design and when getting different
.buildinfos from different builders there will be more differences
(Build-Path, Environment(, Build-Architecture)). So a trivial
de-duplication won't work anyway.



signature.asc
Description: OpenPGP digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread Chris Lamb
Hey Johannes,

> Multiple builds of the same source package will set SOURCE_DATE_EPOCH to
> the same value but will result in a different Build-Date.

… but that would mean that a reproducible build will result in .buildinfo
files with different contents (varying on Build-Date).

That seems, at the very least, somewhat non-intuitive to me.

A case might even be made that varying on Build-Date makes our distribution
problem more difficult; as the files aren't identical we can't easily
"de-duplicate" them with detached signatures. Perhaps I'm missing something
obvious.


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread Johannes Schauer
Hi,

Quoting Chris Lamb (2016-11-13 12:25:07)
> > move the build date inside the .buildinfofile in a Build-Date field or
> > similar
> Hrm. Are we sure about this? My gut tells me that the external definition of
> the build should not include the Build-Date. (At the very least, this is just
> a duplication of SOURCE_DATE_EPOCH?)

the Build-Date is not SOURCE_DATE_EPOCH because the latter is deterministic
(obtained from the last Debian changelog entry). Multiple builds of the same
source package will set SOURCE_DATE_EPOCH to the same value but will result in
a different Build-Date.

The .buildinfo files are not supposed to document the minimum amount of
information that should be required to reproduce the build. Whether or not you
use the information included in the .buildinfo to reproduce the build is up to
you. For example the Build-Path is also still part of the .buildinfo even
though we currently demand builds to be independent of it. In the same vane,
you might later decide that the build should also be independent of the
Build-Architecture. As such, the .buildinfo file contains lots of happenstance
information.

Also see:

https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles#Semantics

I've heard many upstream developers who were initially very much against
purging the timestamp when the build was done from their build artifacts
because they valued the information of when a build was done (whatever their
reasons are). So this information could simply be retained in that field in the
.buildinfo file.

Thanks!

cheers, josch


signature.asc
Description: signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-13 Thread Chris Lamb
Dear Guillem,

First, thanks for all your great work on this.

> * .buildinfo filename

I personally find the current filenames pretty ugly and would prefer them
to match the .changes file. :) Note that they will also differ in raw
content once signed too (!)

> move the build date inside the .buildinfo
>   file in a Build-Date field or similar

Hrm. Are we sure about this? My gut tells me that the external definition
of the build should not include the Build-Date. (At the very least, this is
just a duplication of SOURCE_DATE_EPOCH?)

> * dpkg-genbuildinfo injects itself into debian/files

(Can you clarify what the issue is here? I've not heard of this before…)

> * Some of the environment variables seem superfluous or leaks

Which were we thinking of including? Perhaps attaching/linking to a
concrete example would be helpful here.


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Moving towards a deb-buildinfo(5) Format 1.0

2016-11-12 Thread HW42
Guillem Jover:
> Hi!
> 
> As I've mentioned elsewhere, I've noticed several things with the
> current .buildinfo format, even after the cleanup pre-merge, that
> I'd like to fix or change so that we can hopefully reach Format 1.0.
> 
> Some of the issues, that bother me:
> 
> * .buildinfo files are not currently signed
> 
>   I just need to do that, but given that this is not final I didn't
>   see it as a big problem when I noticed just after the first release
>   introducing this. Ximin also filed #843925 about this later on.

I think, unless you propose something else as the currently planed
OpenPGP "clear text signature", there is consensus about this.

> * .buildinfo filename
> 
>   Having the buildinfo filename be derived from its contents and the
>   current date is annoying and makes finding it more difficult, as you
>   have to scan the .changes file if it's around, but cannot link it
>   directly otherwise. It also leaves dangling files around when building
>   the same package multiple times, just because the date increases. If
>   we have already established that the .buildinfo contents will vary
>   across different builders anyway, which means the .changes files
>   contents will too, I don't see why we need to distinguish the
>   .buildinfo filename generation from the .changes filename.
> 
>   So I'd probably prefer to use the same algo as used for .changes
>   filename generation.

I think (but I'm not sure) the motivation for the current version is to
reflect that there can be different .buildinfos for the same build
product and in contrast to .changes files .buildinfos should not be
thrown away. So giving them probably unique filenames seem reasonable.

>   And move the build date inside the .buildinfo file in a Build-Date
>   field or similar (as I had for some time on my cleanup branch).

I'm not sure what I think about this. One point worth considering when
discussing this is that we already include a timestamp in the signature.

[...]
> * .buildinfo files are not generated when creating source-only uploads
> 
>   I should probably check the exact conditions, but it might be worth
>   including a .buildinfo file if it exists and is in debian/files, even
>   when doing something like:
> 
> $ dpkg-buildpackage -us -uc --change-option=-S
> 
>   because then we could "be sure" the maintainer built something. :)

Not sure if I understand all details here, but sounds sensible in
general.

> * .buildinfo has some issues when including .dsc information
> 
>   Only the .dsc file is referenced not all of its contents, it might
>   be better to match .changes logic here. Also the “source”
>   pseudo-architecture does not get added to the Architecture field.
>   I'll just do at least the latter, I'm open for discussion on the
>   former.

The .dsc already describes the source package completely through the
checksum references for the rest of the package. OTOH I don't see why
adding the rest of the source package would harm.

> * Some of the environment variables seem superfluous or leaks
> 
>   Several of the envvars are either always set by dpkg-buildpackage
>   anyway (like SOURCE_DATE_EPOCH, although the user might have set it
>   explicitly and that will be honored),

I would say that's exactly the reason for including it.

>   or should be irrelevant for
>   the reproducible build, as we should be fixing packages to make them
>   impervious to change in their values (LANG, LC_* etc).

.buildinfos document the build and are designed [0] to include
information about the build which should not be required to reproduce the
build artifacts.

>   Some of which kind of leak information about the build system.
> 
>   I'm not extremely fussed about this point though, because this might
>   still be interesting information to record, but only as long as it
>   does not leak possibly sensitive build system information. Locales
>   used might be too revealing perhaps. But then reportbug and similar
>   leak this and much more so, dunno.

Normally I'm all for privacy by default but OTOH given how leaky the
build process is (see the work of the last years ;P) anybody concerned
about leaks through their builds should/must build in a sanitized
environment anyway.

> I'll probably do some of the fixes already and bump Format to 0.2
> and after the discussion settles we can perhaps do a 0.3, and see how
> it goes, and iterate until it looks good, at which point we'd declare
> it 1.0, ideally before the freeze. :)

Sounds good, and thanks for your work :]


[0]: https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles



signature.asc
Description: OpenPGP digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Moving towards a deb-buildinfo(5) Format 1.0

2016-11-12 Thread Guillem Jover
Hi!

As I've mentioned elsewhere, I've noticed several things with the
current .buildinfo format, even after the cleanup pre-merge, that
I'd like to fix or change so that we can hopefully reach Format 1.0.

Some of the issues, that bother me:

* .buildinfo files are not currently signed

  I just need to do that, but given that this is not final I didn't
  see it as a big problem when I noticed just after the first release
  introducing this. Ximin also filed #843925 about this later on.

* .buildinfo filename

  Having the buildinfo filename be derived from its contents and the
  current date is annoying and makes finding it more difficult, as you
  have to scan the .changes file if it's around, but cannot link it
  directly otherwise. It also leaves dangling files around when building
  the same package multiple times, just because the date increases. If
  we have already established that the .buildinfo contents will vary
  across different builders anyway, which means the .changes files
  contents will too, I don't see why we need to distinguish the
  .buildinfo filename generation from the .changes filename.

  So I'd probably prefer to use the same algo as used for .changes
  filename generation. And move the build date inside the .buildinfo
  file in a Build-Date field or similar (as I had for some time on my
  cleanup branch).

* dpkg-genbuildinfo injects itself into debian/files

  I still tend to think this is the right way to do it (instead of
  letting dpkg-buildpackage add the entry). But this can cause issues
  when running it multiple times, but it does not cause them (or
  should not), with a stable filename, so fixing the above would cover
  this one too.

* .buildinfo files are not generated when creating source-only uploads

  I should probably check the exact conditions, but it might be worth
  including a .buildinfo file if it exists and is in debian/files, even
  when doing something like:

$ dpkg-buildpackage -us -uc --change-option=-S

  because then we could "be sure" the maintainer built something. :)

* .buildinfo has some issues when including .dsc information

  Only the .dsc file is referenced not all of its contents, it might
  be better to match .changes logic here. Also the “source”
  pseudo-architecture does not get added to the Architecture field.
  I'll just do at least the latter, I'm open for discussion on the
  former.

* Some of the environment variables seem superfluous or leaks

  Several of the envvars are either always set by dpkg-buildpackage
  anyway (like SOURCE_DATE_EPOCH, although the user might have set it
  explicitly and that will be honored), or should be irrelevant for
  the reproducible build, as we should be fixing packages to make them
  impervious to change in their values (LANG, LC_* etc). Some of which
  kind of leak information about the build system.

  I'm not extremely fussed about this point though, because this might
  still be interesting information to record, but only as long as it
  does not leak possibly sensitive build system information. Locales
  used might be too revealing perhaps. But then reportbug and similar
  leak this and much more so, dunno.


I'll probably do some of the fixes already and bump Format to 0.2
and after the discussion settles we can perhaps do a 0.3, and see how
it goes, and iterate until it looks good, at which point we'd declare
it 1.0, ideally before the freeze. :)

Thanks,
Guillem

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds