Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-20 Thread Thomas Schmitt
Hi, uploaded is http://www.gnu.org/software/xorriso/xorriso-1.4.5.tar.gz MD5 245708a0530741dcb2cc30ee4fadaa29 Version timestamp : 2016.08.20.102601 Its -as mkisofs option --set_all_file_dates understands pseudo-timestamp "set_to_mtime", which changes atime and ctime immediately before wri

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Thomas Schmitt
Hi, i wrote: > > As it is now, it is the clearest no-brainer: > > Export SOURCE_DATE_EPOCH and forget about any other time issues which > > you don't create yourself by own xorriso arguments. Chris Lamb wrote: > I'm sorry but I don't quite follow and I don't want to read what I want > to read. C

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Chris Lamb
> > *iff* SOURCE_DATE_EPOCH is set: > > > > - mtimes are taken from stat(2). [The FS creator is responsible for > > ensuring they are reproducible, possibly by some "clamping" call > > to `find -newermt | xargs touch …`.] > > > > - atime is copied from mtime [as FS creator "can't" set that].

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Thomas Schmitt
Hi, Chris Lamb wrote: > May I propose the following behaviour *iff* SOURCE_DATE_EPOCH > is set: > > - mtimes are taken from stat(2). [The FS creator is responsible for > ensuring they are reproducible, possibly by some "clamping" call > to `find -newermt | xargs touch …`.] > > - atime is cop

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Chris Lamb
Hi Thomas, > - Verify that SOURCE_DATE_EPOCH works as promised and expected. > (Promised is to set the defaults of three xorriso settings. >Expected is that these defaults suffice for reliable reproducibility.) My current schedule is to test this and report back in the next 4 or 5 days. As

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-19 Thread Thomas Schmitt
Hi, i am pondering when to release xorriso-1.4.6 et.al. 1.4.5 passed my rebuild-ISO regression tests by reporting no differences of file content or boot equipment as far as xorriso can replay the latter. Normally i would wait a few weeks whether my daily backups or some courageous user find any o

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-16 Thread Chris Lamb
Hi Thomas, > uploaded is > http://www.gnu.org/software/xorriso/xorriso-1.4.5.tar.gz [..] > which obeys SOURCE_DATE_EPOCH by stating in its man pages Awesome! :) Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org / chris-lamb.co.uk `- __

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-16 Thread Thomas Schmitt
Hi, uploaded is http://www.gnu.org/software/xorriso/xorriso-1.4.5.tar.gz MD5 2e64a46b50c5192a0797e63dd3182b6a Version timestamp : 2016.08.16.131907 which obeys SOURCE_DATE_EPOCH by stating in its man pages ENVIRONMENT ... SOURCE_DATE_EPOCH belongs to the specs of reproducible-

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-15 Thread Chris Lamb
> If file "$HOME/project_42_iso_ids" exists, then the lengthy parameter > is taken from there. It defines a timestamp, a disk GUID, and what else > might turn out to be in need of being reproduced. I also understand and appreciate that some use-cases need more entropy than a 32-bit integer can pro

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-15 Thread Holger Levsen
On Mon, Aug 15, 2016 at 02:43:49PM +0200, Thomas Schmitt wrote: > Control via environment is unpopular on my side because it would be > a novelty in the user interface of xorriso. are you sure? I mean, usually stuff like locales and language settings are already taken from the environment… SOURCE

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-15 Thread HW42
Thomas Schmitt: > A new source of irreproducibility appeared: Future xorriso versions. > [...] > > I will of course try to keep such changes as rare as possible. But it > cannot be totally guaranteed that the same input data and options will > yield the same ISO with future versions of xorriso. >

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-15 Thread Thomas Schmitt
Hi, i wrote: > > This becomes lengthy. Wiki article size. Chris Lamb wrote: > I can't try and convince you -- one last time -- that a single global > argument, perhaps set via the environment, would be one way to simplify > this? I am open to the idea of a bundle-it-all option when all issues an

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-13 Thread Thomas Schmitt
Hi i wrote: > we could only use the first 528 bytes of the ISO before we have to > determine the GUID. At 528 begins the CRC32 This is not correct, obviously, as libisofs must look ahead for computing the CRC32. I will have to investigate how much head buffering is done and whether there is enou

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-13 Thread Thomas Schmitt
Hi, Daniel Kahn Gillmor wrote: > Would it possible to generate the GPT GUID based on a digest of the > contents of the ISO itself? > [...] > That would give you identical GUIDs for identical ISOs, and distinct > GUIDs for ISOs that vary in any way The general use case of mkisofs is to produce the

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-12 Thread Daniel Kahn Gillmor
On Fri 2016-08-12 15:27:40 -0400, Thomas Schmitt wrote: > Although grub-mkrescue probably can live with poor GPT GUIDs, i meanwhile > found a use case in xorriso where user defined modification-date does not > express the desire for reproducibile GUIDs: xorriso command > -boot_image "any" "replay".

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-12 Thread Thomas Schmitt
Hi, i wrote: > > It would be quite cumbersome to produce upstream 1.4.5 releases of the > > libraries for Sid. Chris Lamb wrote: > It was just a general and friendly offer, I didn't mean for it to come > across as a request or require a time-consuming justification if you did > not want to proce

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-11 Thread Chris Lamb
> It would be quite cumbersome to produce upstream 1.4.5 releases of the > libraries for Sid. It was just a general and friendly offer, I didn't mean for it to come across as a request or require a time-consuming justification if you did not want to proceed. Regards, -- ,''`. : :'

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-11 Thread Thomas Schmitt
Hi, i wrote: > > Release 1.4.6 might come soon. [after enough testing] > > Then it depends on whether my Debian sponsor is on holiday. Chris Lamb wrote: > For this, or any other packages that you wish to see in Debian, I would > be very happy to sponsor. Please let me know when appropriate with a

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-11 Thread Chris Lamb
Dear Thomas, > i now implemented the new -as mkisofs option: > > --set_all_file_dates timestring >Set mtime, atime, and ctime of all files and directories to the >given time. Thank you so much for your efforts on this and the other changes. :) > - Use xorriso-1.4.5 snapshot 2

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-05 Thread Chris Lamb
> http://lists.gnu.org/archive/html/bug-xorriso/2015-06/msg9.html > "we have /BOOT/GRUB/GRUB.CFG;1 with extent 2316 in the first build, >and extent 47 in the second." Looks like I can reproduce that here: https://gist.githubusercontent.com/lamby/66fd5ef5a76874e47b3d75d6ff472208/raw/

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-05 Thread Chris Lamb
Thomas Schmitt wrote: > This discussion yielded that your xorriso-1.3.2 with SOURCE_DATE_EPOCH > will not work reproducibly if you present the same input file tree on > different hard disks or after copying it to another location in the > global tree. Just to confirm, this is due to non-determins

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-05 Thread Thomas Schmitt
Hi, meanwhile i remember that xorriso ISO production already went through a Debian Reproducible Builds discussion: http://lists.gnu.org/archive/html/bug-xorriso/2015-06/msg6.html This discussion yielded that your xorriso-1.3.2 with SOURCE_DATE_EPOCH will not work reproducibly if you present

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-04 Thread Chris Lamb
Hi Thomas, Thanks again for the view and battling the list moderation! :) > --set_all_file_dates "$timestring" > which sets atime, mtime, and ctime of all files, covers your patches > 0001-source_date_epoch.patch > 0002-set-default-timestamp.patch Assuming it also sets iso_write

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-04 Thread Thomas Schmitt
Hi, (this time i hopefully managed to exempt you from Pkg-libburnia-devel list moderation) Requests for agreement: - Do you agree that a new -as mkisofs option --set_all_file_dates "$timestring" which sets atime, mtime, and ctime of all files, covers your patches 0001-source_date_

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-03 Thread Chris Lamb
> Is it really desirable/correct to override the timestamps of files when > they get put into the ISO image ? I.e. isn't reproducible build supposed to > be identical only if the input is identical ? Good question. At least for modification times, I think we *could* go with asking users to run som

Re: [Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-03 Thread Thomas Schmitt
Hi, > +++ libisoburn-1.3.2/libisoburn/isoburn.c Oh nostalgy. :)) - General questions: Is it really desirable/correct to override the timestamps of files when they get put into the ISO image ? I.e. isn't reproducible build

[Reproducible-builds] Reproducible patches for libisoburn and libisofs

2016-08-02 Thread Chris Lamb
[Please keep myself and reproducible-builds@lists.alioth.debian.org in CC] Hey, Thomas Schmitt impressed upon me to send my work in progress patches to make reproducible ISO images. I'm not 100% convinced by them all but I guess it would be good to throw them over-the-wall and see what you guys t