Source: openstack-pkg-tools
Version: 108
Severity: wishlist
Tags: patch
User: reproducible-bui...@lists.alioth.debian.org
Usertags: toolchain
X-Debbugs-Cc: reproducible-b...@lists.alioth.debian.org
Hi,
Whilst working on the Reproducible Builds effort [0] we noticed that
openstack-pkg-tools is causing other packages to be built in an
unreproducible manner.
In particular, the "/usr/bin/pkgos-dh_auto_install" script may
nondeterministically create packages with differing shebangs and binary
dependencies. For example, this is from src:redfishtool:
│ -#!/usr/bin/python3.7
│ +#!/usr/bin/python3.8
[…]
│ │ │ │ -Depends: python3-requests, python3.8:any, python3:any
│ │ │ │ +Depends: python3-requests, python3.7:any, python3:any
§
This is caused by a number of layered reasons. First, we are building
all supported Python versions (eg. Python 3.7 and Python 3.8) in
separate directories but then seqeuentially installing them to the
same destination, debian/${TARGET_DIR}.
However, this causes problems because if latter installations complete
in less than one second, distutils may decide to skip copying files in
the shared destination as it incorrectly believes them to be up-to-
date. This will result in a package arbitrarily containing scripts
with different version shebangs depending on the approximate total
execution speed of installation. This is, needless to say,
nondeterminstic.
For example, if we build for both Python 3.7 and Python 3.8 but the
installation of the latter occurs within the same wall clock second of
the former, the Python 3.8 version will not overwrite the Python 3.7
verison and lead to a shebang of #!/usr/bin/python3.7 … whilst if it
does not occur within the same second, the shebang will be overwritten
to #!/usr/bin/python3.8.
A patch is attached that passes --force to `setup.py install [..]`
which will avoid the underlying calls to distutils's `dep_util.newer`
and thus will always update.
[0] https://reproducible-builds.org/
Regards,
--
,''`.
: :' : Chris Lamb
`. `'` la...@debian.org / chris-lamb.co.uk
`-
diff --git a/build-tools/pkgos-dh_auto_install
b/build-tools/pkgos-dh_auto_install
index 130ac59..2c639d9 100755
--- a/build-tools/pkgos-dh_auto_install
+++ b/build-tools/pkgos-dh_auto_install
@@ -37,7 +37,7 @@ fi
if [ "${PKGOS_USE_PY2}" = "yes" ] ; then
for pyvers in ${PYTHONS}; do
- python${pyvers} setup.py install --install-layout=deb --root
$(pwd)/debian/python-${PY_MODULE_NAME}
+ python${pyvers} setup.py install --install-layout=deb --force
--root $(pwd)/debian/python-${PY_MODULE_NAME}
done
fi
@@ -48,7 +48,7 @@ if [ "${PKGOS_USE_PY3}" = "yes" ] ; then
TARGET_DIR=python3-${PY_MODULE_NAME}
fi
for pyvers in ${PYTHON3S}; do
- python${pyvers} setup.py install --install-layout=deb --root
$(pwd)/debian/${TARGET_DIR}
+ python${pyvers} setup.py install --install-layout=deb --force
--root $(pwd)/debian/${TARGET_DIR}
done
fi
rm -rf $(pwd)/debian/python*/usr/lib/python*/dist-packages/*.pth
