Package: debhelper
Version: 9.20151005
Severity: wishlist
Tags: patch
User: reproducible-bui...@lists.alioth.debian.org
Usertags: toolchain timestamps
X-Debbugs-Cc: reproducible-bui...@lists.alioth.debian.org
As noticed by Santiago Vila [1], timestamps of directories created by
dh_installchangelogs (and probably dh_install and dh_installdocs) can
vary depending on the filesystem readdir() order.
This is because the 'cp --parents -p' calls create subdirectories with
different timestamps depending on the order of files processed. If the
directory gets created as a parent of an installed file, its timestamp
is first preserved, but if other files are later copied under it, the
stamp gets updated. There's a recipe that shows this happening in [2].
The simplest fix I can come up with is to add a 'sort -z' in the find +
xargs pipeline. See the attached proposed patches.
Please note that I've done only light testing of the patches, so review
would be very welcome. The line wrapping changings are because t/size
enforces a maximum of 160 characters per line.
FWIW, it seems to me that using 'cpio -pdm' instead of 'cp --parents -p'
would be cleaner and more efficient, but that's a more intrusive change
with more risk for breakage. And perhaps there's a reason for using 'cp'
that I'm missing?
[1] thread at
http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20151005/003556.html
http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20151012/003574.html
[2]
http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20151005/003567.html
--
Niko Tyni nt...@debian.org
>From 7ad0f54e56e86773e18d7614da53dd76ecf0adfa Mon Sep 17 00:00:00 2001
From: Niko Tyni <nt...@debian.org>
Date: Mon, 12 Oct 2015 14:47:35 +0300
Subject: [PATCH 1/2] dh_installexamples: use find + xargs instead of find
-exec
Commit 71d06996481c6187b016c3b14b7e9faaf8adc432 changed most of the find
-exec invocations in dh_install* programs, but this one remains. Use
xargs instead to prepare for an additional sort command in the pipeline.
---
dh_installexamples | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/dh_installexamples b/dh_installexamples
index c39e899..9ec6914 100755
--- a/dh_installexamples
+++ b/dh_installexamples
@@ -97,7 +97,8 @@ foreach my $package (@{$dh{DOPACKAGES}}) {
my $pwd=`pwd`;
chomp $pwd;
my $exclude2 = '-type f'.$exclude;
- complex_doit("cd '$dir' && find '$basename' -type f$exclude -exec cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples \\;");
+ complex_doit("cd '$dir' && " .
+ "find '$basename' -type f$exclude -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples");
}
else {
doit("cp", '--reflink=auto', "-a", $example,
--
2.5.1
>From 7e962d5d847c7f6d365ad1a9fc256a9b56438a1a Mon Sep 17 00:00:00 2001
From: Niko Tyni <nt...@debian.org>
Date: Mon, 12 Oct 2015 15:05:06 +0300
Subject: [PATCH 2/2] dh_install*: sort file lists passed to 'cp --parents -p'
for reproducibility
The order in which the files are copied can affect directory
timestamps when copying subdirectory hierarchies.
---
dh_install | 6 ++++--
dh_installdocs | 4 +++-
dh_installexamples | 3 ++-
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/dh_install b/dh_install
index 9c8d23c..769d9d4 100755
--- a/dh_install
+++ b/dh_install
@@ -214,11 +214,13 @@ foreach my $package (getpackages()) {
my $pwd=`pwd`;
chomp $pwd;
complex_doit("cd '$dir' && " .
- "find '$basename' $exclude \\( -type f -or -type l \\) -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/$dest/");
+ "find '$basename' $exclude \\( -type f -or -type l \\) -print0 | LC_ALL=C sort -z | " .
+ "xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/$dest/");
# cp is annoying so I need a separate pass
# just for empty directories
complex_doit("cd '$dir' && " .
- "find '$basename' $exclude \\( -type d -and -empty \\) -print0 | xargs -0 -I {} cp --reflink=auto --parents -a {} $pwd/$tmp/$dest/");
+ "find '$basename' $exclude \\( -type d -and -empty \\) -print0 | LC_ALL=C sort -z | " .
+ "xargs -0 -I {} cp --reflink=auto --parents -a {} $pwd/$tmp/$dest/");
}
else {
doit("cp", '--reflink=auto', "-a", $src, "$tmp/$dest/");
diff --git a/dh_installdocs b/dh_installdocs
index 45b5338..7eccab8 100755
--- a/dh_installdocs
+++ b/dh_installdocs
@@ -232,7 +232,9 @@ foreach my $package (@{$dh{DOPACKAGES}}) {
my $pwd=`pwd`;
chomp $pwd;
my $docdir = "$pwd/$tmp/usr/share/doc/$package";
- complex_doit("cd '$dir' && find '$basename' \\( -type f -or -type l \\)$exclude -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $docdir");
+ complex_doit("cd '$dir' && " .
+ "find '$basename' \\( -type f -or -type l \\)$exclude -print0 | LC_ALL=C sort -z | " .
+ "xargs -0 -I {} cp --reflink=auto --parents -dp {} $docdir");
}
else {
doit("cp", '--reflink=auto', "-a", $doc,
diff --git a/dh_installexamples b/dh_installexamples
index 9ec6914..7b28776 100755
--- a/dh_installexamples
+++ b/dh_installexamples
@@ -98,7 +98,8 @@ foreach my $package (@{$dh{DOPACKAGES}}) {
chomp $pwd;
my $exclude2 = '-type f'.$exclude;
complex_doit("cd '$dir' && " .
- "find '$basename' -type f$exclude -print0 | xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples");
+ "find '$basename' -type f$exclude -print0 | LC_ALL=C sort -z | " .
+ "xargs -0 -I {} cp --reflink=auto --parents -dp {} $pwd/$tmp/usr/share/doc/$package/examples");
}
else {
doit("cp", '--reflink=auto', "-a", $example,
--
2.5.1
