[
https://issues.apache.org/jira/browse/SPARK-54049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-54049:
----------------------------------
Parent: SPARK-54359
Issue Type: Sub-task (was: Bug)
> spark-network-common no longer shades all of Guava
> --------------------------------------------------
>
> Key: SPARK-54049
> URL: https://issues.apache.org/jira/browse/SPARK-54049
> Project: Spark
> Issue Type: Sub-task
> Components: Build
> Affects Versions: 4.0.0
> Reporter: Yosef Fertel
> Assignee: Vinod KC
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.1.0, 4.0.2
>
>
> Shading of Guava in `spark-network-common` has changed from 3.x to 4.x such
> that conflicting classes prevent downstream code from using Guava.
> In spark jars prior to 4.x,
> `com/google/thirdparty/publicsuffix/PublicSuffixPatterns.class` was not a
> class included in the jar. Post 4.x, Guava is still shaded to `sparkproject`
> but `com/google/thirdparty/publicsuffix/PublicSuffixPatterns.class`, used in
> Guava's `InternetDomainName` and elsewhere, is now present in the jar.
> {code:java}
> $ jar tf spark-network-common_2.13-4.0.0.jar | rg PublicSuffixPatterns
> com/google/thirdparty/publicsuffix/PublicSuffixPatterns.class {code}
> If a library depends on spark but also depends on Guava, calls to
> `InternetDomainName` that call `PublicSuffixPatterns` will fail with
> `Exception in thread "main" java.lang.NoSuchFieldError: EXACT`.
> Inspecting the code locations in such a library via
> `classOf[InternetDomainName].getProtectionDomain.getCodeSource.getLocation`
> and
> `classOf[PublicSuffixPatterns].getProtectionDomain.getCodeSource.getLocation`
> reveals that `InternetDomainName` is sourced from Guava,
> `target/bg-jobs/sbt_2062a9c3/target/5fcb43b5/1685140132000/guava-32.0.0-jre.jar`,
> while `PublicSuffixPatterns` is sourced instead from spark jar,
> `target/bg-jobs/sbt_2062a9c3/target/db746978/1747651686000/spark-network-common_2.13-4.0.0.jar`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
