This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.1 by this push:
new 4bda955 [SPARK-34762][BUILD] Fix the build failure with Scala 2.13
which is related to commons-cli
4bda955 is described below
commit 4bda9558b80ae2cf28c62377c0633012f1189e72
Author: Kousuke Saruta <[email protected]>
AuthorDate: Thu Mar 18 12:31:50 2021 +0900
[SPARK-34762][BUILD] Fix the build failure with Scala 2.13 which is related
to commons-cli
### What changes were proposed in this pull request?
This PR fixes the build failure with Scala 2.13 which is related to
`commons-cli`.
The last few days, build with Scala 2.13 on GA continues to fail and the
error message says like as follows.
```
[error]
/home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/server/HiveServer2.java:26:1:
error: package org.apache.commons.cli does not exist
1278[error] import org.apache.commons.cli.GnuParser;
```
The reason is that `mvn help` in `change-scala-version.sh` downloads the
POM file of `commons-cli` but doesn't download the JAR file, leading the build
failure.
This PR also adds `commons-cli` to the dependencies explicitly because
HiveThriftServer depends on it.
### Why are the changes needed?
Expect to fix the build failure with Scala 2.13.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
I confirmed that build successfully finishes with Scala 2.13 on my laptop.
```
find ~/.m2 -name commons-cli -exec rm -rf {} \;
find ~/.ivy2 -name commons-cli -exec rm -rf {} \;
find ~/.cache/ -name commons-cli -exec rm -rf {} \; // For Linux
find ~/Library/Caches -name commons-cli -exec rm -rf {} \; // For macOS
dev/change-scala-version 2.13
./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver
-Phadoop-cloud -Pkinesis-asl -Pdocker-integration-tests
-Pkubernetes-integration-tests -Pspark-ganglia-lgpl -Pscala-2.13 clean compile
test:compile
```
Closes #31862 from sarutak/commons-cli.
Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit c5cadfefdf9b4c6135355b49366fd9e9d1e3fcd0)
Signed-off-by: HyukjinKwon <[email protected]>
---
dev/change-scala-version.sh | 5 +++++
pom.xml | 6 ++++++
sql/hive-thriftserver/pom.xml | 4 ++++
3 files changed, 15 insertions(+)
diff --git a/dev/change-scala-version.sh b/dev/change-scala-version.sh
index 9cdc7d9..6adfb23 100755
--- a/dev/change-scala-version.sh
+++ b/dev/change-scala-version.sh
@@ -60,6 +60,11 @@ BASEDIR=$(dirname $0)/..
find "$BASEDIR" -name 'pom.xml' -not -path '*target*' -print \
-exec bash -c "sed_i 's/\(artifactId.*\)_'$FROM_VERSION'/\1_'$TO_VERSION'/g'
{}" \;
+# dependency:get is workaround for SPARK-34762 to download the JAR file of
commons-cli.
+# Without this, build with Scala 2.13 using SBT will fail because the help
plugin used below downloads only the POM file.
+COMMONS_CLI_VERSION=`build/mvn help:evaluate -Dexpression=commons-cli.version
-q -DforceStdout`
+build/mvn dependency:get
-Dartifact=commons-cli:commons-cli:${COMMONS_CLI_VERSION} -q
+
# Update <scala.version> in parent POM
# First find the right full version from the profile's build
SCALA_VERSION=`build/mvn help:evaluate -Pscala-${TO_VERSION}
-Dexpression=scala.version -q -DforceStdout`
diff --git a/pom.xml b/pom.xml
index 6f2dfd1..77c4ea8 100644
--- a/pom.xml
+++ b/pom.xml
@@ -199,6 +199,7 @@
<paranamer.version>2.8</paranamer.version>
<maven-antrun.version>1.8</maven-antrun.version>
<commons-crypto.version>1.1.0</commons-crypto.version>
+ <commons-cli.version>1.2</commons-cli.version>
<!--
If you are changing Arrow version specification, please check
./python/pyspark/sql/pandas/utils.py, and ./python/setup.py too.
@@ -2379,6 +2380,11 @@
</exclusion>
</exclusions>
</dependency>
+ <dependency>
+ <groupId>commons-cli</groupId>
+ <artifactId>commons-cli</artifactId>
+ <version>${commons-cli.version}</version>
+ </dependency>
</dependencies>
</dependencyManagement>
diff --git a/sql/hive-thriftserver/pom.xml b/sql/hive-thriftserver/pom.xml
index 7cc9c3b..92893af 100644
--- a/sql/hive-thriftserver/pom.xml
+++ b/sql/hive-thriftserver/pom.xml
@@ -133,6 +133,10 @@
<groupId>net.sf.jpam</groupId>
<artifactId>jpam</artifactId>
</dependency>
+ <dependency>
+ <groupId>commons-cli</groupId>
+ <artifactId>commons-cli</artifactId>
+ </dependency>
</dependencies>
<build>
<outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
