senthh commented on PR #3865: URL: https://github.com/apache/celeborn/pull/3865#issuecomment-6111201818
You're right to question it — my wording was inaccurate. Let me correct it. The accurate picture across the shaded clients: - `mr-shaded`: bundles zstd-jni (artifactSet `<include>com.github.luben:zstd-jni</include>`) and does **not** relocate `com.github.luben`. So its zstd-jni classes ship under the original package. - `tez-shaded`: bundles zstd-jni and *did* relocate `com.github.luben` to the shading prefix — that relocation is what this PR removes. - `spark-3/4-shaded` and `flink-*-shaded`: do **not** bundle zstd-jni in the shaded jar at all (Spark and Flink ship their own `zstd-jni` on the runtime classpath), and they have no `com.github.luben` relocation. So the only apples-to-apples comparison is `mr-shaded`, which already bundles zstd-jni under its original package (no relocation). The real point of the change stands: zstd-jni is a JNI library whose native `.so` exports symbols bound to the original class names, so relocating `com.github.luben` to the shading prefix breaks the native load. `tez-shaded` keeps the artifactSet include (zstd stays bundled) but should not remap it, exactly like `mr-shaded` does. I'll update the PR description to remove the incorrect "spark-* / flink-*" framing and compare against `mr-shaded` only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
