Sakthi created SPARK-51348:
------------------------------
Summary: Upgrade Hive to 4+
Key: SPARK-51348
URL: https://issues.apache.org/jira/browse/SPARK-51348
Project: Spark
Issue Type: Epic
Components: Build, SQL
Affects Versions: 4.0.0
Reporter: Sakthi
It's been a while since Hive 4 was released. It contains lots of new features
and bug fixes. Creating this Epic for us to track updating the built-int hive
version to 4+
As part of quick testing around this, I tried updating the pom file to hive
4.0.1 and ran into the following issue while building the Hive module:
{code:java}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:3.5.0:enforce (enforce-versions)
on project spark-hive_2.13:
[ERROR] Rule 0: org.apache.maven.enforcer.rules.dependency.BannedDependencies
failed with message:
[ERROR] org.apache.spark:spark-hive_2.13:jar:4.1.0-SNAPSHOT
[ERROR] org.apache.hive:hive-common:jar:4.0.1
[ERROR] org.apache.tez:tez-api:jar:0.10.4
[ERROR] org.apache.hadoop:hadoop-yarn-common:jar:3.3.6
[ERROR] org.apache.hadoop:hadoop-hdfs-client:jar:3.3.6 <--- banned
via the exclude/include list
[ERROR] org.apache.hadoop:hadoop-hdfs-client:jar:3.3.6 <--- banned via
the exclude/include list
[ERROR] org.apache.hive:hive-exec:jar:core:4.0.1
[ERROR] org.apache.atlas:atlas-intg:jar:2.3.0
[ERROR] org.apache.hadoop:hadoop-common:jar:3.3.0 <--- banned via the
exclude/include list
[ERROR] org.apache.hadoop:hadoop-yarn-registry:jar:3.3.6
[ERROR] org.apache.hadoop:hadoop-registry:jar:3.3.6
[ERROR] org.apache.hadoop:hadoop-common:jar:3.3.6 <--- banned via
the exclude/include list
[ERROR] org.apache.hive:hive-llap-common:jar:4.0.1
[ERROR] org.apache.hadoop:hadoop-common:jar:3.3.6 <--- banned via the
exclude/include list{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
