Siyao Meng created HDDS-17153:
---------------------------------
Summary: Native rocks-tools library fails to load on Rocky Linux
9: GLIBCXX_3.4.32 not found
Key: HDDS-17153
URL: https://issues.apache.org/jira/browse/HDDS-17153
Project: Apache Ozone
Issue Type: Bug
Components: Native
Reporter: Siyao Meng
Assignee: Siyao Meng
Attachments: HDDS-17153.001.patch
h3. Problem
The native rocks-tools library in the 2.2.1 release cannot be loaded on Rocky
Linux 9, including the official {{apache/ozone:2.2.1}} image:
{code}
WARN utils.NativeLibraryLoader: Unable to load library: ozone_rocksdb_tools
java.lang.UnsatisfiedLinkError:
/tmp/ozone_rocksdb_tools.../libozone_rocksdb_tools.so: /lib64/libstdc++.so.6:
version `GLIBCXX_3.4.32' not found (required by
/tmp/ozone_rocksdb_tools.../libozone_rocksdb_tools.so)
{code}
{{ozone debug checknative}} reports {{rocks-tools: false}}. The OM logs
{{Native Library for raw sst file reading loading failed. Fallback to
performing a full diff instead.}}, so snapshot diff always takes the full-diff
fallback and the SST pruning in {{RocksDBCheckpointDiffer}} is skipped.
h3. Root cause
* The 2.2.1 {{libozone_rocksdb_tools.so}} was built with {{GCC: (Debian
14.2.0-19) 14.2.0}}.
* Its only dependency on {{GLIBCXX_3.4.32}} is
{{std::ios_base_library_init()}}. GCC 13+ emits a reference to it in every
translation unit that includes {{<iostream>}}.
* {{ManagedRawSSTFileIterator.cpp}} and {{ManagedRawSSTFileReader.cpp}} both
include {{<iostream>}} but use no streams.
* Rocky Linux 9 / RHEL 9 ship libstdc++ from GCC 11, which provides up to
{{GLIBCXX_3.4.29}}. Without that one symbol, the 2.2.1 library needs at most
{{GLIBCXX_3.4.29}} ({{std::__throw_bad_array_new_length()}}), which Rocky 9
provides.
Changing the runner/release images does not help: Rocky 9 has no supported way
to get a system-wide libstdc++ with {{GLIBCXX_3.4.32}}, and RHEL 9 hosts have
the same limit.
h3. Fix
* Remove the unused {{#include <iostream>}} from both native sources.
* {{ManagedRawSSTFileIterator.cpp}} uses {{std::min}} and {{std::memcpy}}, so
include {{<algorithm>}} and {{<cstring>}} explicitly instead of relying on
{{<iostream>}} pulling them in.
h3. Testing done
* Compiled the two sources with the CMake flags of the {{-Drocks_tools_native}}
build, before and after the change. Each original object has one undefined
reference to {{std::ios_base_library_init()}}; the fixed objects have none.
* Native build on master (RocksDB 10.10.1.1, GCC 13.3) with the change:
** The resulting {{libozone_rocksdb_tools.so}} has no {{ios_base_library_init}}
reference and needs at most {{GLIBCXX_3.4.14}}.
** {{hdds-rocks-native}} tests pass: TestNativeLibraryLoader 6,
TestManagedRawSSTFileIterator 28, TestLatestVersionedKWayMergeIterator 10,
TestLatestVersionedKWayMergeIteratorOverSst 2.
* Verified the root cause on the published {{hdds-rocks-native-2.2.1.jar}} from
Maven Central with {{objdump -T}} and {{readelf -p .comment}}.
h3. Follow-ups (not in this patch)
* *Pin the release build environment.* The release guide allows building on any
Linux, so the native library's libstdc++/glibc requirements depend on the
release manager's machine. For example, 2.2.1 also needs {{GLIBC_2.32}}, which
excludes RHEL 8. Building in a container whose toolchain is no newer than the
oldest supported runtime (e.g. {{rockylinux:9}}) would make this deterministic.
* *Guard in CI.* An acceptance check that {{ozone debug checknative}} reports
{{rocks-tools: true}} in the runner image would catch this kind of regression.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
