[
https://issues.apache.org/jira/browse/SEDONA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716940#comment-17716940
]
Kristin Cowalcijk commented on SEDONA-277:
------------------------------------------
One way to solve this problem is to move spark and scala-related dependencies
out of the dependencyManagement of the parent pom, and explicitly define these
dependencies in the submodules. Once we populate the resolved POM by
substituting profile-guided properties, the POMs for the submodules will have
the correct dependencies and won't be affected by profile activations.
I'm resolving this issue in the same patch for supporting Spark 3.4
([https://github.com/apache/sedona/pull/825]). Otherwise, we'll have the same
problem when publishing {{sedona-xxx-3.0_2.xx}} and {{sedona-xxx-3.4_2.xx}}
artifacts.
> Sedona spark artifacts for scala 2.13 do not have proper POMs
> -------------------------------------------------------------
>
> Key: SEDONA-277
> URL: https://issues.apache.org/jira/browse/SEDONA-277
> Project: Apache Sedona
> Issue Type: Bug
> Affects Versions: 1.4.0
> Reporter: Kristin Cowalcijk
> Priority: Major
>
> A strange error happens when launching spark-shell with {{--packages}}
> specified as sedona spark packages:
> {code}
> ::::::::::::::::::::::::::::::::::::::::::::::
> :: UNRESOLVED DEPENDENCIES ::
> ::::::::::::::::::::::::::::::::::::::::::::::
> :: org.apache.sedona#sedona-spark-shaded-3.0_2.13;1.4.0:
> java.text.ParseException: inconsistent module descriptor file found in
> 'https://repo1.maven.org/maven2/org/apache/sedona/sedona-spark-shaded-3.0_2.13/1.4.0/sedona-spark-shaded-3.0_2.13-1.4.0.pom':
> bad module name: expected='sedona-spark-shaded-3.0_2.13'
> found='sedona-spark-shaded-3.0_2.12';
> :: org.scalatest#scalatest_2.13;working@kontinuation: not found
> ::
> org.scala-lang.modules#scala-collection-compat_2.13;working@kontinuation: not
> found
> ::::::::::::::::::::::::::::::::::::::::::::::
> :::: ERRORS
> unknown resolver null
> central: bad module name found in
> https://repo1.maven.org/maven2/org/apache/sedona/sedona-spark-shaded-3.0_2.13/1.4.0/sedona-spark-shaded-3.0_2.13-1.4.0.pom:
> expected='sedona-spark-shaded-3.0_2.13 found='sedona-spark-shaded-3.0_2.12'
> {code}
> It complains that the POM of spark-shaded contains an inconsistent module
> name. Part of the POM file of {{sedona-spark-shaded-3.0_2.13;1.4.0}} looks
> like this:
> {code:xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <project xmlns="http://maven.apache.org/POM/4.0.0";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/maven-v4_0_0.xsd";>
> <parent>
> <artifactId>sedona-parent</artifactId>
> <groupId>org.apache.sedona</groupId>
> <version>1.4.0</version>
> </parent>
> <modelVersion>4.0.0</modelVersion>
>
> <artifactId>sedona-spark-shaded-${spark.compat.version}_${scala.compat.version}</artifactId>
> <name>${project.groupId}:${project.artifactId}</name>
> <description>A cluster computing system for processing large-scale spatial
> data: Spark Shaded.</description>
> <url>http://sedona.apache.org/</url>
> {code}
> The {{artifactId}} was not resolved to constants when publishing, so the
> dependency resolver uses the property definitions in parent POM to resolve
> {{${spark.compat.version}_${scala.compat.version}}}. The parent POM defined
> {{spark.compat.version}} as 3.0 and {{scala.compat.version}} as 2.12 in
> default activated profile, so the artifact name would be
> {{sedona-spark-shaded-3.0_2.12}}, which is inconsistent with
> {{sedona-spark-shaded-3.0_2.13}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
