Siyao Meng created HDDS-17156:
---------------------------------
Summary: Release guide: pin the build environment for native
libraries and verify they load
Key: HDDS-17156
URL: https://issues.apache.org/jira/browse/HDDS-17156
Project: Apache Ozone
Issue Type: Improvement
Components: documentation, website
Reporter: Siyao Meng
Assignee: Siyao Meng
h3. Problem
The release manager guide
({{docs/08-developer-guide/04-project/02-release-guide.md}} in ozone-site) says
only to build the release on Linux (HDDS-15880). It offers either Rocky Linux
or Ubuntu, with no version pinned, and builds with {{-Drocks_tools_native}}.
The native library therefore links against whatever glibc and libstdc++ the
release manager's machine has, and nothing in "Check the Artifacts" verifies
that it loads.
That is how 2.2.1 shipped a {{libozone_rocksdb_tools.so}} built with Debian GCC
14.2. It needs {{GLIBCXX_3.4.32}}, so it cannot load on Rocky Linux 9,
including the official {{apache/ozone:2.2.1}} image (HDDS-17153). Ozone
silently falls back to a full snapshot diff, so nobody noticed.
h3. Proposed change
* *Pin the build environment.*
** Extend the existing note: the native libraries only load on systems with the
same or newer glibc/libstdc++ as the build machine.
** Build on Rocky Linux 9, the base of the {{apache/ozone}} image, or in a
{{rockylinux:9}} container.
** Note that the native library is built only for the build machine's CPU
architecture.
* *Prerequisites.*
** Replace the Ubuntu tab with a {{rockylinux:9}} container tab: a {{docker
run}} that mounts the repository, {{~/.m2}} and {{~/.gnupg}}, plus the same
{{yum install}} line.
** Label the Rocky tab "Rocky Linux 9" and add {{unzip}}, which the new check
needs.
* *New step 8 in "Check the Artifacts"*, run on Rocky Linux 9 or in a
{{rockylinux:9}} container:
** {{bin/ozone debug checknative}} must report {{rocks-tools: true}}. Run it
from a writable working directory until HDDS-17155 is fixed.
** The highest {{GLIBC_}} and {{GLIBCXX_}} versions that
{{libozone_rocksdb_tools.so}} (from
{{share/ozone/lib/hdds-rocks-native-$VERSION.jar}}) requires must not exceed
{{GLIBC_2.34}} / {{GLIBCXX_3.4.29}}.
Rocky Linux 9 as the baseline is a proposal: it matches the {{apache/ozone}}
and {{apache/ozone-runner}} images. If the project wants an older baseline,
only the version numbers in the note and the check change.
h3. Testing done
* The {{yum install}} line works in a fresh {{rockylinux:9}} container: {{svn}}
resolves to {{subversion}}, and {{unzip}} and {{binutils}} ({{objdump}}) are
installed. The container's toolchain is GCC 11.5, glibc 2.34 and
{{GLIBCXX_3.4.29}}, matching the limits in the check.
* Checked the version commands against real libraries:
** The 2.2.1 release library reports {{GLIBC_2.32}} / {{GLIBCXX_3.4.32}}, so
the check flags it.
** A library rebuilt with the HDDS-17153 fix reports {{GLIBC_2.32}} /
{{GLIBCXX_3.4.14}}.
** A combined {{sort -V | tail}} over both prefixes would hide the {{GLIBCXX_}}
maximum, so the step checks each prefix separately.
* {{unzip}} of {{libozone_rocksdb_tools.so}} from
{{hdds-rocks-native-$VERSION.jar}} was verified on a built distribution.
{{bin/ozone debug checknative}} reports {{rocks-tools: true}} on Rocky Linux 9
for a correctly built library.
* {{pnpm run lint}}, {{.github/scripts/spelling.sh}} and {{pnpm build}} pass.
The rendered release-guide page shows the new note, the container tab and step
8.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
