senthh opened a new pull request, #3865: URL: https://github.com/apache/celeborn/pull/3865
## [CELEBORN-2473] Do not relocate zstd-jni in the Tez shaded jar **Problem:** the Maven shade plugin in `client-tez/tez-shaded/pom.xml` and the sbt assembly rules in `project/CelebornBuild.scala` relocate `com.github.luben` (zstd-jni) to the shading prefix. zstd-jni is a JNI library whose bundled native `.so`/`.dll` exports symbols using the original package names (e.g. `Java_com_github_luben_zstd_Zstd_compressBound`). After relocation the JVM binds those symbols from the remapped class names, so the native load fails with `UnsatisfiedLinkError`/`NoClassDefFoundError` and ZSTD compression in the Tez shuffle breaks. **Fix:** keep zstd-jni bundled (it stays in the `artifactSet` includes) but **stop relocating** `com.github.luben`. This matches every other shaded client (`mr`, `spark-*`, `flink-*`), which already include zstd-jni without remapping it. **Files:** - `client-tez/tez-shaded/pom.xml` (−4) - `project/CelebornBuild.scala` (−1) **Validation:** pom remains well-formed XML; zstd-jni still present in the artifact set. cf. HADOOP-17891 / HBASE-23007: never relocate JNI libraries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
