This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3341021 [SPARK-31041][BUILD] Show Maven errors from within
make-distribution.sh
3341021 is described below
commit 3341021add33a8880392a0968f418ec6733cdefd
Author: Nicholas Chammas <[email protected]>
AuthorDate: Wed Mar 11 08:22:02 2020 -0500
[SPARK-31041][BUILD] Show Maven errors from within make-distribution.sh
### What changes were proposed in this pull request?
This PR makes `dev/make-distribution.sh` a bit easier to use by not hiding
errors thrown by Maven.
As a supporting change, this PR also suppresses progress bar output from
curl and wget that may be output from within `build/mvn`.
### Why are the changes needed?
It's surprising for command-line options to be position-dependent. The
errors that get thrown when you pass the correct option (like `--pip`) but in
the wrong order are confusing.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
I ran a few invocations of `make-distribution.sh` to confirm that, when
passed incorrect options, I can actually see the errors thrown directly by
Maven.
Closes #27800 from nchammas/SPARK-31041-make-distribution.
Authored-by: Nicholas Chammas <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
---
build/mvn | 12 ++----------
dev/make-distribution.sh | 8 ++++----
2 files changed, 6 insertions(+), 14 deletions(-)
diff --git a/build/mvn b/build/mvn
index 3628be9..53a77f6 100755
--- a/build/mvn
+++ b/build/mvn
@@ -35,16 +35,8 @@ install_app() {
local local_tarball="${_DIR}/$2"
local binary="${_DIR}/$3"
- # setup `curl` and `wget` silent options if we're running on Jenkins
- local curl_opts="-L"
- local wget_opts=""
- if [ -n "$AMPLAB_JENKINS" ]; then
- curl_opts="-s ${curl_opts}"
- wget_opts="--quiet ${wget_opts}"
- else
- curl_opts="--progress-bar ${curl_opts}"
- wget_opts="--progress=bar:force ${wget_opts}"
- fi
+ local curl_opts="--silent --show-error -L"
+ local wget_opts="--no-verbose"
if [ -z "$3" -o ! -f "$binary" ]; then
# check if we already have the tarball
diff --git a/dev/make-distribution.sh b/dev/make-distribution.sh
index 9e767ce..571059b 100755
--- a/dev/make-distribution.sh
+++ b/dev/make-distribution.sh
@@ -126,19 +126,19 @@ if [ ! "$(command -v "$MVN")" ] ; then
exit -1;
fi
-VERSION=$("$MVN" help:evaluate -Dexpression=project.version $@ 2>/dev/null\
+VERSION=$("$MVN" help:evaluate -Dexpression=project.version $@ \
| grep -v "INFO"\
| grep -v "WARNING"\
| tail -n 1)
-SCALA_VERSION=$("$MVN" help:evaluate -Dexpression=scala.binary.version $@
2>/dev/null\
+SCALA_VERSION=$("$MVN" help:evaluate -Dexpression=scala.binary.version $@ \
| grep -v "INFO"\
| grep -v "WARNING"\
| tail -n 1)
-SPARK_HADOOP_VERSION=$("$MVN" help:evaluate -Dexpression=hadoop.version $@
2>/dev/null\
+SPARK_HADOOP_VERSION=$("$MVN" help:evaluate -Dexpression=hadoop.version $@ \
| grep -v "INFO"\
| grep -v "WARNING"\
| tail -n 1)
-SPARK_HIVE=$("$MVN" help:evaluate -Dexpression=project.activeProfiles -pl
sql/hive $@ 2>/dev/null\
+SPARK_HIVE=$("$MVN" help:evaluate -Dexpression=project.activeProfiles -pl
sql/hive $@ \
| grep -v "INFO"\
| grep -v "WARNING"\
| fgrep --count "<id>hive</id>";\
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
