On Mon, Jan 16, 2023, 12:41 AM Simon Josefsson via Gnulib discussion list < [email protected]> wrote:
> Bruno Haible <[email protected]> writes: > > > Paul Eggert wrote: > >> some users want to "trust but verify" and a reproducible > >> tarball is easier to audit than a non-reproducible one, so for these > >> users it can be a win to omit the irrelevant data from the tarball. > > > > Reproducibility can be implemented in different ways: > > - by omitting irrelevant data from the tarball, > > - by having a customized comparison program 'diff', such that > > "diff --ignore-irrelevant-metadata contents1 contents2" > > would ignore the irrelevant parts. > > The problem with a --ignore-irrelevant-metadata approach is that it will > be a judgement call what is irrelevant, and two projects may have > different philosophies that are mutually incompatible. > > A devils advocate case: consider a build-system that embeds the > source-code timestamp information in the binary, and the binary sends of > a hash of its executable binary to a remote server for verification > purposes. In some projects this may be what you want to achieve. Then > ignoring this particular metadata will be a critical failure for that > project. > > I think it is a worthy goal to reach a tarball that is deterministically > and one-way reproducable from git source code [for the same set of tool > versions]. > > >> when I do an 'ls > >> -l' of a source directory that I got from a distribution tarball, it's > >> useful to see the last time the contents of each source file was > changed > >> upstream. > > > > OK, now we're discussing different ways to make a tarball reproducible. > > That's nice, because Simon's proposal was to make all timestamps equal, > > and that puts me off. > > In binutils-2.40.tar.bz2 all files are from 2023-01-14. > > In android-studio-2021.3.1.17-linux.tar.gz all files are from 2010-01-01. > > It gives me as a user no idea whether this tarball is 13 years old, > > 2 years old, or from yesterday. > > > > I much prefer Paul's approach, since it still conveys meaningful > > timestamps: > > I agree! > > I even wonder if the binutils tarball build properly on say HP-UX then? > > >> For TZDB, where users have long wanted reproducibility, I use something > >> like this in a Makefile recipe for each source file $$file: > >> > >> time=`git log -1 --format='tformat:%ct' $$file` && > >> touch -cmd @$$time $$file > > > > That's good for the files that are under version control. > > > >> 2. What about platform-independent files that are automatically created > >> from source files from the repository, and that are shipped in the > >> release tarball? > > > > For these, you could unpack the tarball, see in which order the > timestamps > > are, and then assign artificial timestamps, in the same order but exactly > > 2 seconds apart. For example, if the tarball contains > > under version control: > > hello.c 2023-01-14 13:28:14 > > configure.ac 2023-01-01 14:03:07 > > and not under version control: > > configure 2023-01-15 04:09:10 > > config.h.in 2023-01-15 04:05:19 > > then you would determine the > > max_timestamp_under_vc = max { 2023-01-14 13:28:14, 2023-01-01 > 14:03:07 } > > = 2023-01-14 13:28:14 > > and then, since config.h.in is older than configure: > > touch -m (max_timestamp_under_vc + 2 seconds) config.h.in > > touch -m (max_timestamp_under_vc + 4 seconds) configure > > > > You can do this without knowing the Makefile rules or scripts which > created > > config.h.in and configure. > > > > The increment of 2 seconds is, of course, for VFAT file systems, which > have > > only 2 seconds of resolution for file modification times. > > Clever! > > To implement this we would need a dist-hook to do the 'touch -m ...' > dance on all files. > > I somewhat fear that the solution here will be more of a problem than > the original problem due to the complexity. > > Does anyone see a problem with this approach? Do you think it is a good > idea? I like it and don't see any further problems, except for the > complexity but I don't see a way to reduce it. > I like it, too. >
