[
https://issues.apache.org/jira/browse/HDDS-17155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17155:
------------------------------
Fix Version/s: 2.3.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> NativeLibraryLoader falls back to java.io.tmpdir when the working directory
> is not writable
> -------------------------------------------------------------------------------------------
>
> Key: HDDS-17155
> URL: https://issues.apache.org/jira/browse/HDDS-17155
> Project: Apache Ozone
> Issue Type: Bug
> Components: Native
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.0
>
> Attachments: HDDS-17155.001.patch
>
>
> h3. Problem
> {{NativeLibraryLoader}} loads {{libozone_rocksdb_tools.so}} in two steps:
> # It tries {{System.loadLibrary}} from {{java.library.path}}. The
> distribution's {{lib/native}} only contains {{libhadoop}}, so this misses.
> # It extracts the library from {{hdds-rocks-native-*.jar}} into
> {{native.lib.tmp.dir}}. If that property is not set, it uses {{new
> File("").getAbsoluteFile()}}, the *current working directory*.
> No Ozone script sets {{native.lib.tmp.dir}}. Whenever the process cannot
> write to its working directory, extraction fails and the library is not
> loaded:
> {code}
> WARN utils.NativeLibraryLoader: Unable to load library: ozone_rocksdb_tools
> java.nio.file.AccessDeniedException:
> /opt/hadoop/ozone_rocksdb_tools7373108702252003876
> WARN snapshot.SnapshotDiffManager: Native Library for raw sst file reading
> loading failed. Fallback to performing a full diff instead. ...
> {code}
> This was hit in the compose runner image: the OM runs in {{/opt/hadoop}}, the
> mounted distribution, and that directory was not writable by the container
> user. The fallback is silent apart from the WARN, so snapshot diff quietly
> loses the native path. {{ozone debug checknative}} then reports
> {{rocks-tools: false}} for this reason, which would also mask the libstdc++
> problem in HDDS-17153.
> For comparison:
> * Hadoop's {{NativeCodeLoader}} only uses {{System.loadLibrary}} from
> {{lib/native}} and never extracts.
> * RocksJava's own {{NativeLibraryLoader}} extracts {{librocksdbjni}} into
> {{java.io.tmpdir}} by default ({{File.createTempFile}}), or into
> {{ROCKSDB_SHAREDLIB_DIR}}.
> h3. Fix
> Keep the working directory as the default extraction location, and fall back
> to {{java.io.tmpdir}} ({{Files.createTempDirectory(libraryName)}}) only when
> the extraction directory cannot be created under the working directory.
> Deployments that load the library today are unaffected, including ones where
> {{/tmp}} is mounted {{noexec}}. An explicit {{native.lib.tmp.dir}} behaves as
> before and does not fall back. Cleanup is unchanged: the existing shutdown
> hook deletes the directory.
> h3. Testing done
> * {{TestNativeLibraryLoader#testDummyLibrary}} covers the working directory
> default (property {{null}} or empty), an explicit {{native.lib.tmp.dir}}, and
> the fallback to {{java.io.tmpdir}} when the working directory cannot hold the
> extraction directory (property {{null}} or empty). The two fallback cases
> fail without the fix.
> * Checkstyle is clean.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
