This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch branch-1.8
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/branch-1.8 by this push:
new 0b8809e67 [KYUUBI #5480] Support setting kyuubi hive jdbc client
protocol version
0b8809e67 is described below
commit 0b8809e67233424837d6e97000682cabc9bb1a82
Author: zhouyifan279 <[email protected]>
AuthorDate: Mon Oct 30 23:51:54 2023 +0800
[KYUUBI #5480] Support setting kyuubi hive jdbc client protocol version
### _Why are the changes needed?_
When using Kyuubi Hive JDBC to Hive Server2,
TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V10 is used and can not be changed.
When we connected to Hive Server2 with version lower than 2.2.0, we got the
following error:
```
org.apache.kyuubi.shade.org.apache.thrift.TApplicationException:
Required field 'client_protocol' is unset!
Struct:TOpenSessionReq(client_protocol:null,
configuration:{kyuubi.client.version=1.7.3,
set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000,
kyuubi.client.ipAddress=172.16.19.113, use:database=default})
```
In this PR, we introduced a session conf `clientProtocolVersion`.
By adding `clientProtocolVersion=8` to jdbc url, the error got fixed.
Changes of `kyuubi_jdbc.rst`
<img width="867" alt="image"
src="https://github.com/apache/kyuubi/assets/88070094/8f98edf9-15c4-4d1b-9299-83b24136352b";>
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- [x] Manual tests against Hive Server2 version 2.1.1-cdh6.3.0
- [x] [Run
test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests)
locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes #5480 from zhouyifan279/protocol-version.
Closes #5480
0ee7b1f68 [zhouyifan279] Support setting kyuubi hive jdbc client protocol
version
61b8038d3 [zhouyifan279] Support setting kyuubi hive jdbc client protocol
version
ed4c29f2b [Cheng Pan] Update docs/deployment/migration-guide.md
dc16a05fc [zhouyifan279] Support setting kyuubi hive jdbc client protocol
version
1a6bfd88c [zhouyifan279] Support setting kyuubi hive jdbc client protocol
version
cb00edc37 [zhouyifan279] Support setting kyuubi hive jdbc client protocol
version
c99fc4802 [zhouyifan279] Support setting kyuubi hive jdbc client protocol
version
Lead-authored-by: zhouyifan279 <[email protected]>
Co-authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit d9a26c3b09e7f3cd1394a222928d9c3b07ac6dff)
Signed-off-by: Cheng Pan <[email protected]>
---
docs/client/jdbc/kyuubi_jdbc.rst | 25 +++++++++++++++++++++-
.../kyuubi/jdbc/hive/JdbcConnectionParams.java | 1 +
.../apache/kyuubi/jdbc/hive/KyuubiConnection.java | 12 +++++++++++
.../java/org/apache/kyuubi/jdbc/hive/Utils.java | 7 ++++++
4 files changed, 44 insertions(+), 1 deletion(-)
diff --git a/docs/client/jdbc/kyuubi_jdbc.rst b/docs/client/jdbc/kyuubi_jdbc.rst
index d4270ea8a..7a63dbd98 100644
--- a/docs/client/jdbc/kyuubi_jdbc.rst
+++ b/docs/client/jdbc/kyuubi_jdbc.rst
@@ -147,6 +147,28 @@ Connection URL over Service Discovery
- zookeeper quorum is the corresponding zookeeper cluster configured by
`kyuubi.ha.addresses` at the server side.
- zooKeeperNamespace is the corresponding namespace configured by
`kyuubi.ha.namespace` at the server side.
+HiveServer2 Compatibility
+*************************
+
+.. versionadded:: 1.8.0
+
+JDBC Drivers need to negotiate a protocol version with Kyuubi
Server/HiveServer2 when connecting.
+
+Kyuubi Hive JDBC Driver offers protocol version v10
(`clientProtocolVersion=9`, supported since Hive 2.3.0)
+to server by default.
+
+If you need to connect to HiveServer2 before 2.3.0,
+please set client property `clientProtocolVersion` to a lower number.
+
+.. code-block:: jdbc
+
+ jdbc:subprotocol://host:port[/catalog]/[schema];clientProtocolVersion=9;
+
+
+.. tip::
+ All supported protocol versions and corresponding Hive versions can be
found in `TProtocolVersion.java`_
+ and its git commits.
+
Kerberos Authentication
-----------------------
Since 1.6.0, Kyuubi JDBC driver implements the Kerberos authentication based
on JAAS framework instead of `Hadoop UserGroupInformation`_,
@@ -218,4 +240,5 @@ Authentication by Subject (programing only)
.. _JDBC Applications: ../bi_tools/index.html
.. _java.sql.DriverManager:
https://docs.oracle.com/javase/8/docs/api/java/sql/DriverManager.html
.. _Hadoop UserGroupInformation:
https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/security/UserGroupInformation.html
-.. _krb5.conf instruction:
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html
\ No newline at end of file
+.. _krb5.conf instruction:
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html
+.. _TProtocolVersion.java:
https://github.com/apache/hive/blob/master/service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TProtocolVersion.java
\ No newline at end of file
diff --git
a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/JdbcConnectionParams.java
b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/JdbcConnectionParams.java
index 71949b9df..bcc94e083 100644
---
a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/JdbcConnectionParams.java
+++
b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/JdbcConnectionParams.java
@@ -33,6 +33,7 @@ public class JdbcConnectionParams {
// Client param names:
+ static final String CLIENT_PROTOCOL_VERSION = "clientProtocolVersion";
// Retry setting
static final String RETRIES = "retries";
diff --git
a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
index c23985328..d3fbbeb6d 100644
---
a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
+++
b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
@@ -733,6 +733,18 @@ public class KyuubiConnection implements SQLConnection,
KyuubiLoggable {
if (sessVars.containsKey(HS2_PROXY_USER)) {
openConf.put(HS2_PROXY_USER, sessVars.get(HS2_PROXY_USER));
}
+ String clientProtocolStr =
+ sessVars.getOrDefault(
+ CLIENT_PROTOCOL_VERSION, openReq.getClient_protocol().getValue() +
"");
+ TProtocolVersion clientProtocol =
+ TProtocolVersion.findByValue(Integer.parseInt(clientProtocolStr));
+ if (clientProtocol == null) {
+ throw new IllegalArgumentException(
+ String.format(
+ "Unsupported Hive2 protocol version %s specified by session conf
key %s",
+ clientProtocolStr, CLIENT_PROTOCOL_VERSION));
+ }
+ openReq.setClient_protocol(clientProtocol);
try {
openConf.put("kyuubi.client.ipAddress",
InetAddress.getLocalHost().getHostAddress());
} catch (UnknownHostException e) {
diff --git
a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/Utils.java
b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/Utils.java
index d0167e3e4..135c38d8e 100644
--- a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/Utils.java
+++ b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/Utils.java
@@ -292,6 +292,13 @@ public class Utils {
}
}
}
+ if (!connParams.getSessionVars().containsKey(CLIENT_PROTOCOL_VERSION)) {
+ if (info.containsKey(CLIENT_PROTOCOL_VERSION)) {
+ connParams
+ .getSessionVars()
+ .put(CLIENT_PROTOCOL_VERSION,
info.getProperty(CLIENT_PROTOCOL_VERSION));
+ }
+ }
// Extract user/password from JDBC connection properties if its not
supplied
// in the connection URL
if (!connParams.getSessionVars().containsKey(AUTH_USER)) {
