Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-03 Thread Thomas Schmitt
Hi, > +++ libisoburn-1.3.2/libisoburn/isoburn.c Oh nostalgy. :)) - General questions: Is it really desirable/correct to override the timestamps of files when they get put into the ISO image ? I.e. isn't reproducible build

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-04 Thread Thomas Schmitt
Hi, (this time i hopefully managed to exempt you from Pkg-libburnia-devel list moderation) Requests for agreement: - Do you agree that a new -as mkisofs option --set_all_file_dates "$timestring" which sets atime, mtime, and ctime of all files, covers your patches 0001-source_date_

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-05 Thread Thomas Schmitt
Hi, meanwhile i remember that xorriso ISO production already went through a Debian Reproducible Builds discussion: http://lists.gnu.org/archive/html/bug-xorriso/2015-06/msg6.html This discussion yielded that your xorriso-1.3.2 with SOURCE_DATE_EPOCH will not work reproducibly if you present

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-11 Thread Thomas Schmitt
Hi, i wrote: > > Release 1.4.6 might come soon. [after enough testing] > > Then it depends on whether my Debian sponsor is on holiday. Chris Lamb wrote: > For this, or any other packages that you wish to see in Debian, I would > be very happy to sponsor. Please let me know when appropriate with a

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-12 Thread Thomas Schmitt
Hi, i wrote: > > It would be quite cumbersome to produce upstream 1.4.5 releases of the > > libraries for Sid. Chris Lamb wrote: > It was just a general and friendly offer, I didn't mean for it to come > across as a request or require a time-consuming justification if you did > not want to proce

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-13 Thread Thomas Schmitt
Hi, Daniel Kahn Gillmor wrote: > Would it possible to generate the GPT GUID based on a digest of the > contents of the ISO itself? > [...] > That would give you identical GUIDs for identical ISOs, and distinct > GUIDs for ISOs that vary in any way The general use case of mkisofs is to produce the

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-13 Thread Thomas Schmitt
Hi i wrote: > we could only use the first 528 bytes of the ISO before we have to > determine the GUID. At 528 begins the CRC32 This is not correct, obviously, as libisofs must look ahead for computing the CRC32. I will have to investigate how much head buffering is done and whether there is enou

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-15 Thread Thomas Schmitt
Hi, i wrote: > > This becomes lengthy. Wiki article size. Chris Lamb wrote: > I can't try and convince you -- one last time -- that a single global > argument, perhaps set via the environment, would be one way to simplify > this? I am open to the idea of a bundle-it-all option when all issues an

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-16 Thread Thomas Schmitt
Hi, uploaded is http://www.gnu.org/software/xorriso/xorriso-1.4.5.tar.gz MD5 2e64a46b50c5192a0797e63dd3182b6a Version timestamp : 2016.08.16.131907 which obeys SOURCE_DATE_EPOCH by stating in its man pages ENVIRONMENT ... SOURCE_DATE_EPOCH belongs to the specs of reproducible-

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Thomas Schmitt
Hi, i am pondering when to release xorriso-1.4.6 et.al. 1.4.5 passed my rebuild-ISO regression tests by reporting no differences of file content or boot equipment as far as xorriso can replay the latter. Normally i would wait a few weeks whether my daily backups or some courageous user find any o

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Thomas Schmitt
Hi, Chris Lamb wrote: > May I propose the following behaviour *iff* SOURCE_DATE_EPOCH > is set: > > - mtimes are taken from stat(2). [The FS creator is responsible for > ensuring they are reproducible, possibly by some "clamping" call > to `find -newermt | xargs touch …`.] > > - atime is cop

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Thomas Schmitt
Hi, i wrote: > > As it is now, it is the clearest no-brainer: > > Export SOURCE_DATE_EPOCH and forget about any other time issues which > > you don't create yourself by own xorriso arguments. Chris Lamb wrote: > I'm sorry but I don't quite follow and I don't want to read what I want > to read. C

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-20 Thread Thomas Schmitt
Hi, uploaded is http://www.gnu.org/software/xorriso/xorriso-1.4.5.tar.gz MD5 245708a0530741dcb2cc30ee4fadaa29 Version timestamp : 2016.08.20.102601 Its -as mkisofs option --set_all_file_dates understands pseudo-timestamp "set_to_mtime", which changes atime and ctime immediately before wri

[Reproducible-builds] libburn-1.4.4-1 suddenly became unreproducible

2016-08-26 Thread Thomas Schmitt
Hi, (please Cc me with replies) I need a hint how to fix "captures_build_path" with libburn: https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/libburn.html "If using autoconf, make sure you call ./configure via a relative and not absolute path." The tests 2016-07-26 amd

Re: libburn-1.4.4-1 suddenly became unreproducible

2016-08-27 Thread Thomas Schmitt
Hi, thanks for looking into the problem. Reiner Herrmann wrote: > Since recently we are variying the build path. A good move, then. (Is it sure that inode numbers differ, too ? :)) I assume that the libburn failure of 2016-08-22 23:30 means that the successes of libisofs 2016-08-22 23:43 and of

Do the file timestamps in ISOs matter ?

2016-09-01 Thread Thomas Schmitt
Hi, do the file timestamps matter in ISOs of debian-cd, debian-live, or in other Debian ISOs ? I ask this because xorriso-1.4.6 will obey environment variable SOURCE_DATE_EPOCH of reproducible-builds. I then plan to propose to debian ISO producers to use xorriso-1.4.6 with new original ISOs and

Re: Do the file timestamps in ISOs matter ?

2016-09-02 Thread Thomas Schmitt
Hi, i wrote: > > new program option: --set_all_file_dates [...] Chris Lamb wrote: > I believe this option is entirely unnecessary. No need to > complicate things. It is architecturally needed, because if SOURCE_DATE_EPOCH sets the default, there must be some option to set non-default values, es

Re: Do the file timestamps in ISOs matter ?

2016-09-02 Thread Thomas Schmitt
Hi, Chris Lamb wrote: > I just don't see this usecase of being "partly" reproducible being remotely > useful to anyone, ever. I'm probably misunderstanding something, however. All three possible behaviors lead to reproducibility if the input trees of the ISO production runs are sufficiently simil

Re: Do the file timestamps in ISOs matter ?

2016-09-02 Thread Thomas Schmitt
Hi, i wrote: > > All three possible behaviors lead to reproducibility if the > > input trees of the ISO production runs are sufficiently > > similar. Sam Hartman wrote: > So, are you using a definition of reproducible different than the > resulting iso will have the same SHA-1 hash? I mean ident

Re: Do the file timestamps in ISOs matter ?

2016-09-12 Thread Thomas Schmitt
Hi, by the narrow margin of one vote, the proposal "set_to_mtime" wins. Committed as http://libburnia-project.org/changeset/5757 man 1 xorrisofs of the next release will state: SOURCE_DATE_EPOCH belongs to the specs of reproducible-builds.org. It is supposed to be either undefined or to co

[Reproducible-builds] How to react on "the second build failed, even though the first build was successful" ?

2015-12-11 Thread Thomas Schmitt
Hi, (please Cc me with replies) On https://tracker.debian.org/pkg/libisoburn i see under "action needed" a link to https://reproducible.debian.net/rb-pkg/unstable/amd64/libisoburn.html with the statement that it would FTBS on second try. Now what action is needed here ? Both builds look fla

Re: [Reproducible-builds] How to react on "the second build failed, even though the first build was successful" ?

2015-12-11 Thread Thomas Schmitt
Hi, > so we don't use up too much of your time: Probably yours is more prescious than mine. :)) > there is nothing to react to I will patiently wait until this is sorted out and the ugly warning vanishes from the package tracker. > Regarding Perl: I just wanted to emphasize my i-didn't-do-i

Re: [Reproducible-builds] How to react on "the second build failed, even though the first build was successful" ?

2015-12-14 Thread Thomas Schmitt
Hi, Holger Levsen wrote: > See https://reproducible.debian.net/reproducible.html#variation to learn how > 1st+2nd build differ. The second setup should be no problem. (The use of two different message languages quite devalues the diff, though.) The reason for my post is/was that i cannot spot a

Re: [Reproducible-builds] How to react on "the second build failed, even though the first build was successful" ?

2015-12-14 Thread Thomas Schmitt
Hi, > I've triggered a test of libisoburn in unstable/amd64 Sunshine ! One stain less on the package. Have a nice day :) Thomas ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/

Re: [Reproducible-builds] generating reproducible ISOs with xorriso

2015-06-04 Thread Thomas Schmitt
Hi, > I participate in the Debian Reproducible Builds project I saw some sin-list page about packages shortly after xorriso-1.4.0 was released. It complained about libburn's doc/doxygen.conf.in which i hope to have fixed by http://libburnia-project.org/changeset/5446 > I wanted to see if xorr

Re: [Reproducible-builds] generating reproducible ISOs with xorriso

2015-06-04 Thread Thomas Schmitt
Hi, > If we can identify the specific commands (beyond what you point out > below), would there a general interest upstream in something like a > --reproducible=TARGETDATE The syntax would have to be different and probably a more comprehensive name will come to us when we know what xorriso featur

Re: [Reproducible-builds] generating reproducible ISOs with xorriso

2015-06-04 Thread Thomas Schmitt
Hi, the current situation with data extents looks not good for the purpose of reproducability. The files are grafted into a red-black tree according to their inode and device numbers on hard disk. This is done to merge hardlinks. The tree is then serialized into an array which gets sorted accordi

Re: [Reproducible-builds] generating reproducible ISOs with xorriso

2015-06-05 Thread Thomas Schmitt
Hi, > > red-black tree > it makes sense why it is this way, and it also > makes sense why this does not look good for reproducibility. :/ It is slightly overdesigned. :)) A sorted and deduplicated array would do the same. > > Brute force would be a giant weight list > well, -sort-weight-list is

Re: [Reproducible-builds] generating reproducible ISOs with xorriso

2015-06-05 Thread Thomas Schmitt
Hi, About the --sort-weight-list approach which is possible with already released xorriso versions: > (find . -type f -print0 | xargs -0 md5sum | sort | cut -f2- -d/ ; find . > -mindepth 1 \! -type f | sort | cut -f2- -d/ ) | awk '{ N=N+1; print N " " > $0 }' I misunderstood the role of md5sum h

Re: [Reproducible-builds] generating reproducible ISOs with xorriso

2015-06-05 Thread Thomas Schmitt
Hi, i wrote: > > md5sum [...] seems surplus. Daniel Kahn Gillmor wrote: > Right, but it would seem to fail for hardlinked files or deduped files, > because it would weight one of the files in different places than the > other. Oh. I misunderstood the md5sum part again. It's for the content indee