This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new afa6aee [SPARK-33237][K8S][TESTS] Use default Hadoop-3.2 profile from
K8s IT Jenkins job
afa6aee is described below
commit afa6aee4f5ea270db5331e48ad08e0b176cdd2a0
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Mon Oct 26 15:29:12 2020 -0700
[SPARK-33237][K8S][TESTS] Use default Hadoop-3.2 profile from K8s IT
Jenkins job
### What changes were proposed in this pull request?
This PR aims to use `hadoop-3.2` profile in K8s IT Jenkins jobs.
- [x] Switch the default value of `HADOOP_PROFILE` from `hadoop-2.7` to
`hadoop-3.2`.
- [x] Remove `-Phadoop2.7` from Jenkins K8s IT job.
-
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/configure
**BEFORE**
```
./dev/make-distribution.sh --name ${DATE}-${REVISION} --r --pip --tgz
-DzincPort=${ZINC_PORT} \
-Phadoop-2.7 -Pkubernetes -Pkinesis-asl -Phive -Phive-thriftserver
```
**AFTER**
```
./dev/make-distribution.sh --name ${DATE}-${REVISION} --r --pip --tgz
-DzincPort=${ZINC_PORT} \
-Pkubernetes -Pkinesis-asl -Phive -Phive-thriftserver
```
### Why are the changes needed?
Since Apache Spark 3.1.0, Hadoop 3 is the default.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Check the Jenkins K8s IT log and result.
-
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34899/
```
+ /home/jenkins/workspace/SparkPullRequestBuilder-K8s/build/mvn clean
package -DskipTests -DzincPort=4021 -Pkubernetes -Pkinesis-asl -Phive
-Phive-thriftserver
Using `mvn` from path:
/home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.6.3/bin/mvn
[INFO] Scanning for projects...
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
```
Closes #30153 from dongjoon-hyun/SPARK-33237.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../kubernetes/integration-tests/dev/dev-run-integration-tests.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh
b/resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh
index 9c03a97..b72a4f7 100755
---
a/resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh
+++
b/resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh
@@ -35,7 +35,7 @@ CONTEXT=
INCLUDE_TAGS="k8s"
EXCLUDE_TAGS=
JAVA_VERSION="8"
-HADOOP_PROFILE="hadoop-2.7"
+HADOOP_PROFILE="hadoop-3.2"
MVN="$TEST_ROOT_DIR/build/mvn"
SCALA_VERSION=$("$MVN" help:evaluate -Dexpression=scala.binary.version
2>/dev/null\
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
