yandrey321 opened a new pull request, #12008:
URL: https://github.com/apache/ozone/pull/12008

   ## What changes were proposed in this pull request?
   
   **Problem:** Since 2.2.0, `ozone-filesystem-hadoop3` picked up two libraries 
that each serve a single call, plus two native libraries that cannot load:
   - `commons-io` (512 KB compressed, 428 classes). `hdds-client` declares it 
for one `IOUtils.closeQuietly(sock)` in `DomainSocketFactory`.
   - `kerby-util` (31 KB). `ozone-common` declares it for `Hex.encode` in 
`AWSV4AuthValidator`.
   - 
`liborg_apache_ratis_thirdparty_netty_transport_native_kqueue_aarch_64.jnilib` 
and `..._resolver_dns_native_macos_aarch_64.jnilib`. ratis-thirdparty 1.1.1 
added these, but `ozonefs-shaded` renames and excludes only the x86_64 
variants. They ship under their Ratis names, which the relocated classes don't 
look for, so on Apple-silicon macOS the shaded Ratis netty can't load its 
kqueue transport or its native DNS resolver.
   
   **Implementation idea:**
   - `DomainSocketFactory` uses Ozone's 
`org.apache.hadoop.hdds.utils.IOUtils.closeQuietly`. Remove `commons-io` from 
`hadoop-hdds/client/pom.xml`.
   - `AWSV4AuthValidator` uses Hadoop's `StringUtils.byteToHexString`; the 
class already imports Hadoop `StringUtils`. Remove `kerby-util` from 
`hadoop-ozone/common/pom.xml`. `TestHmacSha256` moves to the same method.
   - In `ozonefs-shaded/pom.xml`, add `copy-rename-maven-plugin` entries and 
shade excludes for the two aarch_64 natives, alongside the existing x86_64 
entries.
   
   **Key decisions:**
   - Hex encoding uses `StringUtils.byteToHexString` rather than Guava's 
`BaseEncoding`, so the change adds no import and no dependency. Its output is 
identical to kerby's `Hex.encode` (lowercase, no separators), checked for all 
256 byte values and for empty input. Ozone's `hdds.StringUtils.bytes2Hex` was 
rejected, because it produces uppercase hex with spaces and would break S3 
signature validation. `java.util.HexFormat` is not available, because 
`ozone-common` targets Java 8.
   - The natives are renamed rather than only excluded, so Apple-silicon 
clients get the native transport, as x86_64 macOS clients already do.
   - No enforcer ban is added for these two libraries. Both stay on client 
classpaths transitively through `hadoop-common`, which the shaded jar already 
excludes.
   
   **Side effects:**
   - `ozone-filesystem-hadoop3` is 543 KB smaller (compressed) and has about 
450 fewer entries.
   - S3 signature validation does not change.
   - On Apple-silicon macOS, the Ozone filesystem client's Ratis netty now 
loads kqueue and the macOS DNS resolver natively, instead of falling back to 
NIO and the JDK resolver. Linux and x86_64 macOS are unaffected.
   - `s3gateway` keeps its own `kerby-util` dependency, and the dist 
`LICENSE.txt` does not change: both libraries still ship with the servers.
   
   Co-authored-by: Claude Opus 5.5 <[email protected]>
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-17775
   
   ## How was this patch tested?
   
   Unit + integration tests
   
   - **Jar contents:** compared the shaded jar before and after the change.
     - No `org/apache/ozone/shaded/org/apache/commons/io` or `.../kerby` 
entries remain.
     - No `liborg_apache_ratis_thirdparty_*` natives remain.
     - Both aarch_64 natives are present under the relocated names.
     - `mvn dependency:tree -pl :ozone-filesystem-shaded 
-Dincludes=commons-io,org.apache.kerby` is empty.
   - **Natives (manual, Apple-silicon macOS):** a probe calls `isAvailable()` 
on the shaded Ratis netty `KQueue` and `MacOSDnsServerAddressStreamProvider`:
   
   | shaded Ratis netty on arm64 macOS | before | after |
   |---|---|---|
   | `KQueue.isAvailable()` | false (`UnsatisfiedLinkError`) | **true** |
   | `MacOSDnsServerAddressStreamProvider.isAvailable()` | false 
(`UnsatisfiedLinkError`) | **true** |
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to