This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 7ba73bf14c3 [SPARK-39119][BUILD] Upgrade Hadoop to 3.3.3 7ba73bf14c3 is described below commit 7ba73bf14c3704a5b341d26c3a68bf35cf1738df Author: Chao Sun <sunc...@apple.com> AuthorDate: Wed May 18 14:05:28 2022 -0700 [SPARK-39119][BUILD] Upgrade Hadoop to 3.3.3 ### What changes were proposed in this pull request? This PR aims to upgrade to Hadoop 3.3.3, which was just announced today. ### Why are the changes needed? Hadoop 3.3.3 has many bug fixes as well as CVE fixes. Please check [release notes](https://hadoop.apache.org/docs/r3.3.3/hadoop-project-dist/hadoop-common/release/3.3.3/RELEASENOTES.3.3.3.html) and [change log](https://hadoop.apache.org/docs/r3.3.3/hadoop-project-dist/hadoop-common/release/3.3.3/CHANGELOG.3.3.3.html). ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. Closes #36474 from sunchao/upgrade-hadoop-3.3.3. Authored-by: Chao Sun <sunc...@apple.com> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> --- dev/deps/spark-deps-hadoop-3-hive-2.3 | 22 +++++++++++----------- pom.xml | 2 +- .../sql/hive/client/IsolatedClientLoader.scala | 2 +- 3 files changed, 13 insertions(+), 13 deletions(-) diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3 index d06f2227631..525c01307a4 100644 --- a/dev/deps/spark-deps-hadoop-3-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-3-hive-2.3 @@ -67,18 +67,18 @@ generex/1.0.2//generex-1.0.2.jar gmetric4j/1.0.10//gmetric4j-1.0.10.jar gson/2.2.4//gson-2.2.4.jar guava/14.0.1//guava-14.0.1.jar -hadoop-aliyun/3.3.2//hadoop-aliyun-3.3.2.jar -hadoop-annotations/3.3.2//hadoop-annotations-3.3.2.jar -hadoop-aws/3.3.2//hadoop-aws-3.3.2.jar -hadoop-azure-datalake/3.3.2//hadoop-azure-datalake-3.3.2.jar -hadoop-azure/3.3.2//hadoop-azure-3.3.2.jar -hadoop-client-api/3.3.2//hadoop-client-api-3.3.2.jar -hadoop-client-runtime/3.3.2//hadoop-client-runtime-3.3.2.jar -hadoop-cloud-storage/3.3.2//hadoop-cloud-storage-3.3.2.jar -hadoop-cos/3.3.2//hadoop-cos-3.3.2.jar -hadoop-openstack/3.3.2//hadoop-openstack-3.3.2.jar +hadoop-aliyun/3.3.3//hadoop-aliyun-3.3.3.jar +hadoop-annotations/3.3.3//hadoop-annotations-3.3.3.jar +hadoop-aws/3.3.3//hadoop-aws-3.3.3.jar +hadoop-azure-datalake/3.3.3//hadoop-azure-datalake-3.3.3.jar +hadoop-azure/3.3.3//hadoop-azure-3.3.3.jar +hadoop-client-api/3.3.3//hadoop-client-api-3.3.3.jar +hadoop-client-runtime/3.3.3//hadoop-client-runtime-3.3.3.jar +hadoop-cloud-storage/3.3.3//hadoop-cloud-storage-3.3.3.jar +hadoop-cos/3.3.3//hadoop-cos-3.3.3.jar +hadoop-openstack/3.3.3//hadoop-openstack-3.3.3.jar hadoop-shaded-guava/1.1.1//hadoop-shaded-guava-1.1.1.jar -hadoop-yarn-server-web-proxy/3.3.2//hadoop-yarn-server-web-proxy-3.3.2.jar +hadoop-yarn-server-web-proxy/3.3.3//hadoop-yarn-server-web-proxy-3.3.3.jar hive-beeline/2.3.9//hive-beeline-2.3.9.jar hive-cli/2.3.9//hive-cli-2.3.9.jar hive-common/2.3.9//hive-common-2.3.9.jar diff --git a/pom.xml b/pom.xml index 22c7b41cd6e..d5618c8f103 100644 --- a/pom.xml +++ b/pom.xml @@ -115,7 +115,7 @@ <slf4j.version>1.7.32</slf4j.version> <log4j.version>2.17.2</log4j.version> <!-- make sure to update IsolatedClientLoader whenever this version is changed --> - <hadoop.version>3.3.2</hadoop.version> + <hadoop.version>3.3.3</hadoop.version> <protobuf.version>2.5.0</protobuf.version> <yarn.version>${hadoop.version}</yarn.version> <zookeeper.version>3.6.2</zookeeper.version> diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala index 15c172a6e75..12bb1b3631c 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala @@ -69,7 +69,7 @@ private[hive] object IsolatedClientLoader extends Logging { // If the error message contains hadoop, it is probably because the hadoop // version cannot be resolved. val fallbackVersion = if (VersionUtils.isHadoop3) { - "3.3.2" + "3.3.3" } else { "2.7.4" } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org