[
https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen updated SPARK-36835:
-------------------------------
Priority: Blocker (was: Major)
> Spark 3.2.0 POMs are no longer "dependency reduced"
> ---------------------------------------------------
>
> Key: SPARK-36835
> URL: https://issues.apache.org/jira/browse/SPARK-36835
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.2.0
> Reporter: Josh Rosen
> Priority: Blocker
>
> It looks like Spark 3.2.0's POMs are no longer "dependency reduced". As a
> result, applications may pull in additional unnecessary dependencies when
> depending on Spark.
> Spark uses the Maven Shade plugin to create effective POMs and to bundle
> shaded versions of certain libraries with Spark (namely, Jetty, Guava, and
> JPPML). [By
> default|https://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createDependencyReducedPom],
> the Maven Shade plugin generates simplified POMs which remove dependencies
> on artifacts that have been shaded.
> SPARK-33212 /
> [b6f46ca29742029efea2790af7fdefbc2fcf52de|https://github.com/apache/spark/commit/b6f46ca29742029efea2790af7fdefbc2fcf52de]
> changed the configuration of the Maven Shade plugin, setting
> {{createDependencyReducedPom}} to {{false}}.
> As a result, the generated POMs now include compile-scope dependencies on the
> shaded libraries. For example, compare the {{org.eclipse.jetty}} dependencies
> in:
> * Spark 3.1.2:
> [https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.1.2/spark-core_2.12-3.1.2.pom]
> * Spark 3.2.0 RC2:
> [https://repository.apache.org/content/repositories/orgapachespark-1390/org/apache/spark/spark-core_2.12/3.2.0/spark-core_2.12-3.2.0.pom]
> I think we should revert back to generating "dependency reduced" POMs to
> ensure that Spark declares a proper set of dependencies and to avoid "unknown
> unknown" consequences of changing our generated POM format.
> /cc [~csun]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
