This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new b594105f1 [KYUUBI #5830] Test Hive engine with CDH Hive 2.1.1-cdh6.3.2
b594105f1 is described below
commit b594105f149c7c3b2a16279798313e009f329e24
Author: Cheng Pan <[email protected]>
AuthorDate: Fri Dec 8 09:57:08 2023 +0800
[KYUUBI #5830] Test Hive engine with CDH Hive 2.1.1-cdh6.3.2
# :mag: Description
## Issue References ๐
The CDH Hive 2.1.1-cdh6.3.2 is the latest free version of CDH, which is
adopted widely, it would be great if the Kyuubi Hive engine could support this
version.
## Describe Your Solution ๐ง
Actually, the Kyuubi Hive engine could work with CDH Hive 2.1.1-cdh6.3.2,
this pull request just adds an integration test and fixes one test case.
## Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
#### Behavior Without This Pull Request :coffin:
#### Behavior With This Pull Request :tada:
#### Related Unit Tests
---
# Checklists
## ๐ Author Self Checklist
- [x] My code follows the [style
guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html)
of this project
- [x] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature
works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
## ๐ Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes #5830 from pan3793/hive-211-cdh6.
Closes #5830
afa80bd82 [Cheng Pan] nit
89c9ae96e [Cheng Pan] nit
e402e49cd [Cheng Pan] nit
3c126a2a6 [Cheng Pan] Test Hive engine with CDH Hive 2.1.1-cdh6.3.2
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
.github/workflows/master.yml | 3 ++
docs/quick_start/quick_start.rst | 32 +++++++++++-----------
.../scala/org/apache/kyuubi/HiveEngineTests.scala | 7 +++--
3 files changed, 23 insertions(+), 19 deletions(-)
diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml
index 109b21c23..1c8c879c0 100644
--- a/.github/workflows/master.yml
+++ b/.github/workflows/master.yml
@@ -248,6 +248,9 @@ jobs:
- java: 8
hive-archive:
'-Dhive.archive.mirror=https://archive.apache.org/dist/hive/hive-2.3.9
-Dhive.archive.name=apache-hive-2.3.9-bin.tar.gz'
comment: 'verify-on-hive-2.3-binary'
+ - java: 8
+ hive-archive:
'-Dhive.archive.mirror=https://github.com/pan3793/cdh-hive/releases/download/cdh6.3.2-release
-Dhive.archive.name=apache-hive-2.1.1-cdh6.3.2-bin.tar.gz'
+ comment: 'verify-on-hive-2.1-cdh6-binary'
steps:
- uses: actions/checkout@v4
- name: Tune Runner VM
diff --git a/docs/quick_start/quick_start.rst b/docs/quick_start/quick_start.rst
index 9d0a7d30c..85a215aad 100644
--- a/docs/quick_start/quick_start.rst
+++ b/docs/quick_start/quick_start.rst
@@ -36,23 +36,23 @@ For quick start deployment, we need to prepare the
following stuffs:
These essential components are JVM-based applications. So, the JRE needs to be
pre-installed and the ``JAVA_HOME`` is correctly set to each component.
- ================ ============ ===============
===========================================
- Component Role Version Remarks
- ================ ============ ===============
===========================================
- **Java** JRE 8/11/17 Officially released against
JDK8
- **Kyuubi** Gateway \ |release| \ - Kyuubi Server
- Engine lib - Kyuubi Engine
- Beeline - Kyuubi Hive Beeline
- **Spark** Engine >=3.1 A Spark distribution
- **Flink** Engine 1.16/1.17/1.18 A Flink distribution
- **Trino** Engine >=363 A Trino cluster
- **Doris** Engine N/A A Doris cluster
- **Hive** Engine - 3.1.x - A Hive distribution
- Metastore - N/A - An optional and external
metadata store,
- whose version is decided by
engines
+ ================ ============ ====================
===========================================
+ Component Role Version Remarks
+ ================ ============ ====================
===========================================
+ **Java** JRE 8/11/17 Officially released
against JDK8
+ **Kyuubi** Gateway \ |release| \ - Kyuubi Server
+ Engine lib - Kyuubi Engine
+ Beeline - Kyuubi Hive Beeline
+ **Spark** Engine 3.1 to 3.5 A Spark distribution
+ **Flink** Engine 1.16/1.17/1.18 A Flink distribution
+ **Trino** Engine >=363 A Trino cluster
+ **Doris** Engine N/A A Doris cluster
+ **Hive** Engine - 2.1-cdh6/2.3/3.1 - A Hive distribution
+ Metastore - N/A - An optional and
external metadata store,
+ whose version is
decided by engines
**Zookeeper** HA >=3.4.x
- **Disk** Storage N/A N/A
- ================ ============ ===============
===========================================
+ **Disk** Storage N/A N/A
+ ================ ============ ====================
===========================================
The other internal or external parts listed in the above sheet can be used
individually
or all together. For example, you can use Kyuubi, Spark and Flink to build a
streaming
diff --git
a/kyuubi-common/src/test/scala/org/apache/kyuubi/HiveEngineTests.scala
b/kyuubi-common/src/test/scala/org/apache/kyuubi/HiveEngineTests.scala
index 61cb48b4c..29df90724 100644
--- a/kyuubi-common/src/test/scala/org/apache/kyuubi/HiveEngineTests.scala
+++ b/kyuubi-common/src/test/scala/org/apache/kyuubi/HiveEngineTests.scala
@@ -230,11 +230,12 @@ trait HiveEngineTests extends HiveJDBCTestHelper {
assume(SystemUtils.isJavaVersionAtMost(JavaVersion.JAVA_1_8))
withJdbcStatement() { statement =>
val resultSet = statement.getConnection.getMetaData.getTableTypes
- // Hive3 removes support for INDEX_TABLE
- val hive2Expected = Set("TABLE", "VIEW", "MATERIALIZED_VIEW",
"INDEX_TABLE")
+ val hive2_1Expected = Set("TABLE", "VIEW", "INDEX_TABLE")
+ val hive2_3Expected = Set("TABLE", "VIEW", "MATERIALIZED_VIEW",
"INDEX_TABLE")
val hive3Expected = Set("TABLE", "VIEW", "MATERIALIZED_VIEW")
val tableTypes = JdbcUtils.mapResultSet(resultSet) { rs =>
rs.getString(TABLE_TYPE) }.toSet
- assert(tableTypes === hive2Expected || tableTypes === hive3Expected)
+ assert(tableTypes === hive2_1Expected || tableTypes === hive2_3Expected
||
+ tableTypes === hive3Expected)
}
}
