[
https://issues.apache.org/jira/browse/SPARK-54793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wy updated SPARK-54793:
-----------------------
Attachment: image-2025-12-21-22-33-13-539.png
> Shade relocation breaks Scala binary compatibility due to unmodified Scala
> signatures
> -------------------------------------------------------------------------------------
>
> Key: SPARK-54793
> URL: https://issues.apache.org/jira/browse/SPARK-54793
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 4.1.0, 4.2.0
> Reporter: wy
> Priority: Minor
> Attachments: image-2025-12-21-22-33-13-539.png
>
>
> When building Spark with Maven Shade plugin, some dependencies, like Jetty,
> are relocated to some private packages like `org.sparkproject.jetty` in the
> final JAR. However, Scala signatures remain unmodified while JVM bytecode is
> relocated, causing binary compatibility breaks for downstream Scala projects.
> This manifests as runtime linkage errors when Scala clients interact with
> shaded APIs.
> After building Spark, it can be confirmed by the following commands:
>
> {code:java}
> javap -classpath
> ./assembly/target/scala-2.13/jars/spark-core_2.13-4.2.0-SNAPSHOT.jar -verbose
> org.apache.spark.ui.WebUI | grep -n 'attachHandler\|descriptor' | grep
> 'public synchronized void'
> scalap -classpath
> ./assembly/target/scala-2.13/jars/spark-core_2.13-4.2.0-SNAPSHOT.jar
> org.apache.spark.ui.WebUI | grep -n 'attachHandler\|ServletContextHandler' |
> grep 'def attachHandler(handler'{code}
> The outputs shows that the Scala signature is inconsistent with the binary:
> {code:java}
> 2065: public synchronized void
> attachHandler(org.sparkproject.jetty.servlet.ServletContextHandler);
> 20: def attachHandler(handler:
> org.eclipse.jetty.servlet.ServletContextHandler): scala.Unit = { /* compiled
> code */ }
> {code}
>
> This will break the downstream projects who use Spark as a dependency.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
