sarutak commented on a change in pull request #31880:
URL: https://github.com/apache/spark/pull/31880#discussion_r598069954
##########
File path: dev/change-scala-version.sh
##########
@@ -60,14 +60,13 @@ BASEDIR=$(dirname $0)/..
find "$BASEDIR" -name 'pom.xml' -not -path '*target*' -print \
-exec bash -c "sed_i 's/\(artifactId.*\)_'$FROM_VERSION'/\1_'$TO_VERSION'/g'
{}" \;
-# dependency:get is workaround for SPARK-34762 to download the JAR file of
commons-cli.
-# Without this, build with Scala 2.13 using SBT will fail because the help
plugin used below downloads only the POM file.
-COMMONS_CLI_VERSION=`build/mvn help:evaluate -Dexpression=commons-cli.version
-q -DforceStdout`
-build/mvn dependency:get
-Dartifact=commons-cli:commons-cli:${COMMONS_CLI_VERSION} -q
-
# Update <scala.version> in parent POM
# First find the right full version from the profile's build
-SCALA_VERSION=`build/mvn help:evaluate -Pscala-${TO_VERSION}
-Dexpression=scala.version -q -DforceStdout`
+# NOTE: We used mvn help:evaluate to fetch the value of scala.version before
but sed is used now.
+# This is a workaround for SPARK-34762.
+ESCAPED_TO_VERSION=$(echo $TO_VERSION | sed -n "s/\./\\\\./gp")
+SCALA_VERSION=$(sed -n "/<id>scala-$ESCAPED_TO_VERSION<\/id>/,/<\/profile>/ \
+ s;^.*<scala\.version>\(.*\)</scala\.version>.*$;\1;p" pom.xml)
Review comment:
I thought what you think too and it can resolve this issue for the time
being.
But I'm afraid that this build failure happens again in the future.
In this case, only `commons-cli` and `comons-io` matters but, actually,
`help` and `dependency` downloads not only them.
I confirmed that `help` downloads pom files but not jar files for 300+
dependencies.
If we use newer maven or upgraded plugins and Spark and those plugins have a
comondependency but plugins use newer version, this problem can happen again.
My worry might be unnecessary or you think we just just fix this problem
when it happens again, I'll close this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
