[
https://issues.apache.org/jira/browse/SPARK-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611600#comment-14611600
]
Konstantin Shaposhnikov commented on SPARK-8781:
------------------------------------------------
I believe this will affect both released and SNAPSHOT artefacts.
Basically, as part of SPARK-3812 the build was changed to deploy an effective
POMs into maven repository. E.g. in
https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/1.4.0/spark-core_2.11-1.4.0.pom
you won't find {{$\{scala.binary.version}}, it was resolved to 2.11 by the
maven during the build. This is required for Scala 2.11 build to make sure that
jars that are built with Scala 2.11 reference Scala 2.11 jars (e.g.
spark-core_2.11 should depend on spark-launcher_2.11, not on
spark-launcher_2.10). By default {{$\{scala.binary.version}} will be resolved
to 2.10 because scala-2.10 maven profile is the active by default.
Publishing of effective POMs is implemented using maven-shade-plugin. To be
honest I am not sure how exactly it works. However when I removed the following
line from the parent POM
{{<createDependencyReducedPom>false</createDependencyReducedPom>}} the build
started to deploy effective POMs again.
I hope my explanation helps.
> Pusblished POMs are no longer effective POMs
> --------------------------------------------
>
> Key: SPARK-8781
> URL: https://issues.apache.org/jira/browse/SPARK-8781
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 1.3.2, 1.4.1, 1.5.0
> Reporter: Konstantin Shaposhnikov
>
> Published to maven repository POMs are no longer effective POMs. E.g.
> In
> https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-core_2.11/1.4.2-SNAPSHOT/spark-core_2.11-1.4.2-20150702.043114-52.pom:
> {noformat}
> ...
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-launcher_${scala.binary.version}</artifactId>
> <version>${project.version}</version>
> </dependency>
> ...
> {noformat}
> while it should be
> {noformat}
> ...
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-launcher_2.11</artifactId>
> <version>${project.version}</version>
> </dependency>
> ...
> {noformat}
> The following commits are most likely the cause of it:
> - for branch-1.3:
> https://github.com/apache/spark/commit/ce137b8ed3b240b7516046699ac96daa55ddc129
> - for branch-1.4:
> https://github.com/apache/spark/commit/84da653192a2d9edb82d0dbe50f577c4dc6a0c78
> - for master:
> https://github.com/apache/spark/commit/984ad60147c933f2d5a2040c87ae687c14eb1724
> On branch-1.4 reverting the commit fixed the issue.
> See SPARK-3812 for additional details
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
