Package: debhelper Version: 9.20151005 Severity: wishlist Tags: patch User: reproducible-builds@lists.alioth.debian.org Usertags: toolchain timestamps X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
As noticed by Santiago Vila [1], timestamps of directories created by dh_installchangelogs (and probably dh_install and dh_installdocs) can vary depending on the filesystem readdir() order. This is because the 'cp --parents -p' calls create subdirectories with different timestamps depending on the order of files processed. If the directory gets created as a parent of an installed file, its timestamp is first preserved, but if other files are later copied under it, the stamp gets updated. There's a recipe that shows this happening in [2]. The simplest fix I can come up with is to add a 'sort -z' in the find + xargs pipeline. See the attached proposed patches. Please note that I've done only light testing of the patches, so review would be very welcome. The line wrapping changings are because t/size enforces a maximum of 160 characters per line. FWIW, it seems to me that using 'cpio -pdm' instead of 'cp --parents -p' would be cleaner and more efficient, but that's a more intrusive change with more risk for breakage. And perhaps there's a reason for using 'cp' that I'm missing? [1] thread at http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20151005/003556.html http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20151012/003574.html [2] http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20151005/003567.html -- Niko Tyni nt...@debian.org
>From 7ad0f54e56e86773e18d7614da53dd76ecf0adfa Mon Sep 17 00:00:00 2001 From: Niko Tyni <nt...@debian.org> Date: Mon, 12 Oct 2015 14:47:35 +0300 Subject: [PATCH 1/2] dh_installexamples: use find + xargs instead of find -exec Commit 71d06996481c6187b016c3b14b7e9faaf8adc432 changed most of the find -exec invocations in dh_install* programs, but this one remains. Use xargs instead to prepare for an additional sort command in the pipeline. --- dh_installexamples | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/dh_installexamples b/dh_installexamples index c39e899..9ec6914 100755 --- a/dh_installexamples +++ b/dh_installexamples @@ -97,7 +97,8 @@ foreach my $package (@{$dh{DOPACKAGES}}) { my $pwd=`pwd`; chomp $pwd; my $exclude2 = '-type f'.$exclude; - complex_doit("cd '$dir' && find '$basename' -type f$exclude -exec cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples \\;"); + complex_doit("cd '$dir' && " . + "find '$basename' -type f$exclude -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples"); } else { doit("cp", '--reflink=auto', "-a", $example, -- 2.5.1
>From 7e962d5d847c7f6d365ad1a9fc256a9b56438a1a Mon Sep 17 00:00:00 2001 From: Niko Tyni <nt...@debian.org> Date: Mon, 12 Oct 2015 15:05:06 +0300 Subject: [PATCH 2/2] dh_install*: sort file lists passed to 'cp --parents -p' for reproducibility The order in which the files are copied can affect directory timestamps when copying subdirectory hierarchies. --- dh_install | 6 ++++-- dh_installdocs | 4 +++- dh_installexamples | 3 ++- 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/dh_install b/dh_install index 9c8d23c..769d9d4 100755 --- a/dh_install +++ b/dh_install @@ -214,11 +214,13 @@ foreach my $package (getpackages()) { my $pwd=`pwd`; chomp $pwd; complex_doit("cd '$dir' && " . - "find '$basename' $exclude \\( -type f -or -type l \\) -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/$dest/"); + "find '$basename' $exclude \\( -type f -or -type l \\) -print0 | LC_ALL=C sort -z | " . + "xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/$dest/"); # cp is annoying so I need a separate pass # just for empty directories complex_doit("cd '$dir' && " . - "find '$basename' $exclude \\( -type d -and -empty \\) -print0 | xargs -0 -I {} cp --reflink=auto --parents -a {} $pwd/$tmp/$dest/"); + "find '$basename' $exclude \\( -type d -and -empty \\) -print0 | LC_ALL=C sort -z | " . + "xargs -0 -I {} cp --reflink=auto --parents -a {} $pwd/$tmp/$dest/"); } else { doit("cp", '--reflink=auto', "-a", $src, "$tmp/$dest/"); diff --git a/dh_installdocs b/dh_installdocs index 45b5338..7eccab8 100755 --- a/dh_installdocs +++ b/dh_installdocs @@ -232,7 +232,9 @@ foreach my $package (@{$dh{DOPACKAGES}}) { my $pwd=`pwd`; chomp $pwd; my $docdir = "$pwd/$tmp/usr/share/doc/$package"; - complex_doit("cd '$dir' && find '$basename' \\( -type f -or -type l \\)$exclude -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $docdir"); + complex_doit("cd '$dir' && " . + "find '$basename' \\( -type f -or -type l \\)$exclude -print0 | LC_ALL=C sort -z | " . + "xargs -0 -I {} cp --reflink=auto --parents -dp {} $docdir"); } else { doit("cp", '--reflink=auto', "-a", $doc, diff --git a/dh_installexamples b/dh_installexamples index 9ec6914..7b28776 100755 --- a/dh_installexamples +++ b/dh_installexamples @@ -98,7 +98,8 @@ foreach my $package (@{$dh{DOPACKAGES}}) { chomp $pwd; my $exclude2 = '-type f'.$exclude; complex_doit("cd '$dir' && " . - "find '$basename' -type f$exclude -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples"); + "find '$basename' -type f$exclude -print0 | LC_ALL=C sort -z | " . + "xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples"); } else { doit("cp", '--reflink=auto', "-a", $example, -- 2.5.1
_______________________________________________ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds