cloud-fan commented on PR #56534: URL: https://github.com/apache/spark/pull/56534#issuecomment-4714602021
Thanks @LuciferYang @pan3793 for the quick look. Based on your feedback I've dropped the shaded-module approach entirely. It turns out the **jcc 12.1.x** driver already bundles a newer lz4-java that has the full `LZ4BlockInputStream.newBuilder()` builder API (including the `LZ4SafeDecompressor` overload `SPARK-54571` uses), so simply upgrading the test-only driver `11.5.9.0` -> `12.1.4.0` resolves the `NoSuchMethodError` with no shading at all. This removes the new module (so no shaded-jar-consumed-across-modules issue and no separate `*-thirdparty` repo needed, @pan3793) and the shade config (@LuciferYang). Updated; PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
