Siyao Meng created HDDS-17275:
---------------------------------
Summary: Make release tarballs reproducible across build
environments
Key: HDDS-17275
URL: https://issues.apache.org/jira/browse/HDDS-17275
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Siyao Meng
h3. Problem
The {{repro}} check runs {{artifact:compare}} in the same job, on the same
runner, path, JDK, user, locale and umask as the reference build. It covers
attached Maven artifacts, but not the release tarballs ({{ozone-X.tar.gz}} is
produced by {{dist-tar-stitching}}, {{ozone-X-src.tar.gz}} has
{{attach=false}}), and it cannot detect differences caused by the build
environment. Verifying a release independently (for example with Apache Trusted
Releases, see HADOOP-19976) needs the tarballs to be identical when built
elsewhere.
h3. Observation
Two runs of {{mvn clean package -DskipTests -DskipDocs -Pdist -Psrc}} on the
same machine, the second with {{TZ=Asia/Tokyo LC_ALL=de_DE.UTF-8}} and {{umask
002}}:
* all jars (including javadoc, sources and shaded jars), {{bom.json}},
{{bom.xml}} and the source tarball were identical
* the binary tarball had identical file contents and entry order, but different
archive metadata: entry mtimes, permissions (following umask), owner and group
names of the builder, and the mtime and file name in the gzip header
h3. Gaps
# *Binary tarball metadata.* {{dist-tar-stitching}} runs plain {{tar}} and
{{gzip}}. Entry order also depends on the directory listing order of the
filesystem (sorted on APFS, not on ext4).
# *Unpinned inputs.* The {{hadoop-native-lib}} profile downloads libhadoop from
the {{master}} branch of {{ozone-thirdparty}} without a checksum. The RocksDB
source archive for {{hdds-rocks-native}} is downloaded without a checksum.
# *Optional content depending on the environment.* Docs are included only if
Hugo is available, and {{--enableGitInfo}} is used only in a git checkout, so
building from the source tarball gives different docs than building from git.
libhadoop is included only if it was downloaded. The Hugo docs are being
removed by HDDS-15923 ([PR #10829|https://github.com/apache/ozone/pull/10829]),
which resolves the docs part.
# *Build environment not pinned or recorded.* The release guide uses the
floating {{rockylinux:9}} image and unpinned JDK, cmake and gcc packages. No
{{.buildinfo}} is produced, so a verifier cannot know which toolchain to use.
# *Native library build path.* {{libozone_rocksdb_tools.so}} is built without
{{-ffile-prefix-map}}, so it may embed the build directory. Not verified.
# *rpm and deb packages* (HDDS-13439) may embed build host and build time. Not
verified; only relevant if they become release artifacts.
h3. Proposed changes
* Create the binary tarball with sorted entries, mtime from
{{project.build.outputTimestamp}}, numeric owner and group 0, normalized
permissions, and {{gzip -n}}.
* Pin downloaded inputs to a tag or commit and verify them with {{sha256}}.
* Make tarball content independent of optional downloads such as libhadoop
(include it consistently or fail).
* Generate {{.buildinfo}} ({{artifact:buildinfo}}) and document a pinned build
environment for releases.
* Extend the {{repro}} check (or add a separate one) to compare both tarballs
from an independent build with a different path, user, TZ, locale and umask.
These can be split into separate subtasks if preferred. Nondeterminism within a
single build is tracked separately in HDDS-16309 and HDDS-12862.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
