Re: Do the file timestamps in ISOs matter ?
Hi, by the narrow margin of one vote, the proposal "set_to_mtime" wins. Committed as http://libburnia-project.org/changeset/5757 man 1 xorrisofs of the next release will state: SOURCE_DATE_EPOCH belongs to the specs of reproducible-builds.org. It is supposed to be either undefined or to contain a decimal number which tells the seconds since january 1st 1970. If it contains a number, then it is used as time value to set the default of --modification-date= and --gpt_disk_guid.The default of --set_all_file_dates is then "set_to_mtime". Startup files and program options can override the effect of SOURCE_DATE_EPOCH. This refers to the description of two new options: --gpt_disk_guid value Control whether an emerging GPT shall get a randomly generated disk GUID or whether the GUID is supplied by the user. Value "random" is default. Value "modification-date" produces a low quality GUID from the value set by option --modification-date=. [...] --set_all_file_dates timestring [...] If the timestring is "set_to_mtime", then the atime and ctime of each file and directory get set to the value found in their mtime. [...] I am now beginning final regression tests for the release of xorriso-1.4.6. Have a nice day :) Thomas ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Do the file timestamps in ISOs matter ?
Hi, i wrote: > > All three possible behaviors lead to reproducibility if the > > input trees of the ISO production runs are sufficiently > > similar. Sam Hartman wrote: > So, are you using a definition of reproducible different than the > resulting iso will have the same SHA-1 hash? I mean identical results. Bit by bit. The question to decide is: What is considered a sufficiently similar input tree in respect to timestamps ? Or in the context of ISO producers: How much flattening (non-injectivity) is tolerable. How much fidelity (non-aliasing) is needed ? The answer depends on the expectations of the programs inside the ISO or expectations of programs and informations which get imported from elsewhere by the programs in the ISO. (The expectations of already installed software on the target system are restricted to those of boot firmware, i assume.) For example, assume that "make" is supposed to act on ISO content. In that case, high timestamp fidelity would be needed. Have a nice day :) Thomas ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Do the file timestamps in ISOs matter ?
> "Thomas" == Thomas Schmitt writes: Thomas> Hi, Thomas> Chris Lamb wrote: >> I just don't see this usecase of being "partly" reproducible >> being remotely useful to anyone, ever. I'm probably >> misunderstanding something, however. Thomas> All three possible behaviors lead to reproducibility if the Thomas> input trees of the ISO production runs are sufficiently Thomas> similar. So, are you using a definition of reproducible different than the resulting iso will have the same SHA-1 hash? ;If so, I think several of us would be helped if you would explain your definition and propose an algorithm to determine whether two isos are identical under your definition. --Sam ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Do the file timestamps in ISOs matter ?
Hi,
Chris Lamb wrote:
> I just don't see this usecase of being "partly" reproducible being remotely
> useful to anyone, ever. I'm probably misunderstanding something, however.
All three possible behaviors lead to reproducibility if the input trees
of the ISO production runs are sufficiently similar.
The undecided question is, as the subject says, how much similarity is
most likely needed. The more - the harder to reproduce.
- If ctime and/or atime of files in the input tree matter, then they
should not be overridden by SOURCE_DATE_EPOCH.
I.e.: --set_all_file_dates "default"
Reproducibility would break quite easily unless the input filesystem
is read-only.
- If mtime stamps of input tree matter, then they should not be overridden
by SOURCE_DATE_EPOCH.
I.e.: --set_all_file_dates "set_to_mtime"
Reproducibility would survive input tree operations like cp -a.
- File timestamps in the ISO do not matter and rather are just a potential
obstacle for reproducibility, then SOURCE_DATE_EPOCH should override them.
I.e.: --set_all_file_dates ="$SOURCE_DATE_EPOCH"
Reproducibility would survive any operation that preserves name tree,
ownership, permissions, and file content.
(Ownership and permissions could be flattened by xorrisofs option -r.)
The latter two are what i would consider reasonable for general purposes.
I myself with my purpose of repacking existing ISOs and checking for
regressions could even live with the first one.
> Therefore folding it all into inheriting {a,c}time from mtime iff S_D_E
> is the logical conclusion,
If "iff" means "if and only if", then i have to contradict:
SOURCE_DATE_EPOCH is not the only way to achieve mtime to ctime,atime copying.
That's a consequence of how i implement it.
So it's only "if", not "iff".
> there is need to for people who "just"
> want a reproducible ISO to tediously scour the documentation to discover
> they need to set --set_all_file_dates.
No. SOURCE_DATE_EPOCH makes the ISO reproducible if it does not get
overriden by non-constant parameters of the program options which one
would have to look for (after man page section ENVIRONMENT listed them
as explanation of SOURCE_DATE_EPOCH).
Another precondition is that the input trees are sufficiently similar,
of course.
The proposal to set all file timestamps to SOURCE_DATE_EPOCH would actually
"just" ease the production of reproducible ISOs.
> > grub-mkrescue
> They won't be run with SOURCE_DATE_EPOCH exported.
When xorriso-1.4.6 is out i plan to ask grub-devel to introduce an option
in grub-mkrescue which overrides the "now" time when the argument for
xorrisofs option --modification-date= is determined.
Maybe i can talk them into deducing the --modification-date= string from
SOURCE_DATE_EPOCH.
(If "set_to_mtime" wins the election, then the actual time value of
SOURCE_DATE_EPOCH will get overridden by --modification-date= completely.)
Have a nice day :)
Thomas
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Do the file timestamps in ISOs matter ?
> > I believe [--set_all_file_dates] is entirely unnecessary
>
> if SOURCE_DATE_EPOCH sets the default, there must be some option to set
> non-default values, especially the value which is default without
> SOURCE_DATE_EPOCH.
Consider the state space:
If someone does not set SOURCE_DATE_EPOCH, they are going to get an
unreproducible regardless. Therefore, whichever set_all_file_dates option
they choose is entirely meaningless and thus xorriso can simply inherit
{a,c,m}time just as before.
If someone does set SOURCE_DATE_EPOCH, they want a reproducible image by
definition. Thus, inheriting {a,c}time from the filesystem makes no sense
as their image will then not be reproducible and setting {a,c,m}time to a
specific date is not necessary as that can be trivially done pre-build.
Therefore folding it all into inheriting {a,c}time from mtime iff S_D_E
is the logical conclusion, from the angles of complicating xorriso itself
and complicating the end-user UI for someone who wants a reproducible ISO.
Putting it another way, the "semi-reproducible" scenario seems so unlikely
we can entirely discount it and thus there is need to for people who "just"
want a reproducible ISO to tediously scour the documentation to discover they
need to set --set_all_file_dates.
> I decided to let SOURCE_DATE_EPOCH only set the defaults of existing
> and new options, in order to protect ISO producers like grub-mkrescue from
> negative effects by overriding their program options.
They won't be run with SOURCE_DATE_EPOCH exported.
> Not to forget that it offers those users a workaround, who do not agree
> with the final decision about the question posed here, but want to use
> SOURCE_DATE_EPOCH nevertheless.
I just don't see this usecase of being "partly" reproducible being remotely
useful to anyone, ever. I'm probably misunderstanding something, however.
Regards,
--
,''`.
: :' : Chris Lamb
`. `'` la...@debian.org / chris-lamb.co.uk
`-
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Do the file timestamps in ISOs matter ?
Hi, i wrote: > > new program option: --set_all_file_dates [...] Chris Lamb wrote: > I believe this option is entirely unnecessary. No need to > complicate things. It is architecturally needed, because if SOURCE_DATE_EPOCH sets the default, there must be some option to set non-default values, especially the value which is default without SOURCE_DATE_EPOCH. I decided to let SOURCE_DATE_EPOCH only set the defaults of existing and new options, in order to protect ISO producers like grub-mkrescue from negative effects by overriding their program options. (grub-mkrescue uses --modification-date= to set the text in ISO 9660 field Volume Modification Date, so that GRUB command search --fs-uuid --set -MM-DD-hh-mm-ss-cc can look it up.) Not to forget that it offers those users a workaround, who do not agree with the final decision about the question posed here, but want to use SOURCE_DATE_EPOCH nevertheless. Have a nice day :) Thomas ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Do the file timestamps in ISOs matter ?
Hi Thomas, > Still undecided is whether [SOURCE_DATE_EPOCH] shall by default override > all timestamps inside the ISO I believe this steps outside the scope of what exporting SOURCE_DATE_EPOCH should do, both philosophically, æsthetically, and practically as it prevents ISO producers from having differing mtimes in files. It's easy enough to clamp (or simply override) the mtimes prior to calling xorriso anyway… > [SOURCE_DATE_EPOCH] shall only copy each file's mtime to the same file's > ctime and atime (to ease reproduction of input file trees). +1 for this. > xorriso will support all three models by allowing to override > SOURCE_DATE_EPOCH > by one of three parameters to a new program option: > --set_all_file_dates default > --set_all_file_dates set_to_mtime > --set_all_file_dates ="$SOURCE_DATE_EPOCH" Given that there is no believable use-case where one would be exporting SOURCE_DATE_EPOCH but somehow desire the atime and ctime to vary but have a fixed mtime, I believe this option is entirely unnecessary. No need to complicate things. Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org / chris-lamb.co.uk `- ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
