This is an automated email from the ASF dual-hosted git repository.
sarutak pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 0888622 [SPARK-37302][BUILD][FOLLOWUP] Extract the versions of
dependencies accurately
0888622 is described below
commit 08886223c6373cc7c7e132bfb58f1536e70286ef
Author: Kousuke Saruta <[email protected]>
AuthorDate: Fri Dec 24 11:29:37 2021 +0900
[SPARK-37302][BUILD][FOLLOWUP] Extract the versions of dependencies
accurately
### What changes were proposed in this pull request?
This PR changes `dev/test-dependencies.sh` to extract the versions of
dependencies accurately.
In the current implementation, the versions are extracted like as follows.
```
GUAVA_VERSION=`build/mvn help:evaluate -Dexpression=guava.version -q
-DforceStdout`
```
But, if the output of the `mvn` command includes not only the version but
also other messages like warnings, a following command referring the version
will fail.
```
build/mvn dependency:get -Dartifact=com.google.guava:guava:${GUAVA_VERSION}
-q
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:3.1.1:get (default-cli) on
project spark-parent_2.12: Couldn't download artifact:
org.eclipse.aether.resolution.DependencyResolutionException:
com.google.guava:guava:jar:Falling was not found in
https://maven-central.storage-download.googleapis.com/maven2/ during a previous
attempt. This failure was cached in the local repository and resolution is not
reattempted until the update interval of gcs-maven-cent [...]
```
Actually, this causes the recent linter failure.
https://github.com/apache/spark/runs/4623297663?check_suite_focus=true
### Why are the changes needed?
To recover the CI.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually run `dev/test-dependencies.sh`.
Closes #35006 from sarutak/followup-SPARK-37302.
Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: Kousuke Saruta <[email protected]>
(cherry picked from commit dd0decff5f1e95cedd8fe83de7e4449be57cb31c)
Signed-off-by: Kousuke Saruta <[email protected]>
---
dev/test-dependencies.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev/test-dependencies.sh b/dev/test-dependencies.sh
index 363ba1a..39a11e7 100755
--- a/dev/test-dependencies.sh
+++ b/dev/test-dependencies.sh
@@ -48,9 +48,9 @@ OLD_VERSION=$($MVN -q \
--non-recursive \
org.codehaus.mojo:exec-maven-plugin:1.6.0:exec | grep -E
'[0-9]+\.[0-9]+\.[0-9]+')
# dependency:get for guava and jetty-io are workaround for SPARK-37302.
-GUAVA_VERSION=`build/mvn help:evaluate -Dexpression=guava.version -q
-DforceStdout`
+GUAVA_VERSION=$(build/mvn help:evaluate -Dexpression=guava.version -q
-DforceStdout | grep -E "^[0-9.]+$")
build/mvn dependency:get -Dartifact=com.google.guava:guava:${GUAVA_VERSION} -q
-JETTY_VERSION=`build/mvn help:evaluate -Dexpression=jetty.version -q
-DforceStdout`
+JETTY_VERSION=$(build/mvn help:evaluate -Dexpression=jetty.version -q
-DforceStdout | grep -E "^[0-9.]+v[0-9]+")
build/mvn dependency:get
-Dartifact=org.eclipse.jetty:jetty-io:${JETTY_VERSION} -q
if [ $? != 0 ]; then
echo -e "Error while getting version string from Maven:\n$OLD_VERSION"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
