This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 4bd259afd [KYUUBI #6036] JDBC driver conditional sets fetchSize on
opening session
4bd259afd is described below
commit 4bd259afd84460edea1955c9b217ab31859e93a9
Author: Cheng Pan <[email protected]>
AuthorDate: Wed Jan 31 21:09:36 2024 +0800
[KYUUBI #6036] JDBC driver conditional sets fetchSize on opening session
# :mag: Description
## Issue References ๐
I get reported by a user that using Kyuubi JDBC driver 1.8.0 to access
Spark Thrift Server 2.4 (with Hive 1.2.1) with getting an error on opening the
session even with `clientProtocol=7`
```
org.apache.hive.service.cli.HiveSQLException:
java.lang.IllegalArgumentException: hive configuration
hive.server2.thrift.resultset.default.fetch.size does not exists.
at
org.apache.hive.service.cli.session.HiveSessionImpl.configureSession(HiveSessionImpl.java:220)
at
org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:154)
at
org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:258)
... 13 more
```
## Describe Your Solution ๐ง
When `hive.conf.validation` is `true` (it also is the default value), an
IllegalArgumentException will be thrown if the provided configurations start
with `hive.` and can not be recognized by the server.
One solution is to disable `hive.conf.validation` on the server side, but
we can also address it by avoiding passing this configuration if we know that
the server does not support it.
HIVE-14901 (2.3.0, HIVE_CLI_SERVICE_PROTOCOL_V10) introduces
`hive.server2.thrift.resultset.default.fetch.size`, so we can set this
configuration only when protocol >= HIVE_CLI_SERVICE_PROTOCOL_V10
## Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
Verified locally by connecting to HS2 2.1.1 and HS2 2.3.9
#### Behavior Without This Pull Request :coffin:
IllegalArgumentException throws when using
`jdbc:hive2://localhost:10000/default;clientProtocol=8` to connect HS2 2.1.1
Everything is OK when using `jdbc:hive2://localhost:10000/default` to
connect HS2 2.3.9
#### Behavior With This Pull Request :tada:
Everything is OK when using
`jdbc:hive2://localhost:10000/default;clientProtocol=8` to connect HS2 2.1.1
Everything is OK when using `jdbc:hive2://localhost:10000/default` to
connect HS2 2.3.9
---
# Checklist ๐
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6036 from pan3793/jdbc-fetchsize.
Closes #6036
7ea91f12c [Cheng Pan] nit
e6ea8291f [Cheng Pan] fix
1dbecbb48 [Cheng Pan] JDBC driver conditional sets fetchSize on opening
session
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
.../java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git
a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
index d1d77a26a..87872581c 100644
---
a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
+++
b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java
@@ -723,10 +723,6 @@ public class KyuubiConnection implements SQLConnection,
KyuubiLoggable {
}
// switch the database
openConf.put("use:database", connParams.getDbName());
- // set the fetchSize
- openConf.put(
- "set:hiveconf:hive.server2.thrift.resultset.default.fetch.size",
- Integer.toString(fetchSize));
if (wmPool != null) {
openConf.put("set:hivevar:wmpool", wmPool);
}
@@ -751,6 +747,12 @@ public class KyuubiConnection implements SQLConnection,
KyuubiLoggable {
clientProtocolStr, CLIENT_PROTOCOL_VERSION));
}
openReq.setClient_protocol(clientProtocol);
+ // HIVE-14901: set the fetchSize
+ if
(clientProtocol.compareTo(TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V10) >= 0)
{
+ openConf.put(
+ "set:hiveconf:hive.server2.thrift.resultset.default.fetch.size",
+ Integer.toString(fetchSize));
+ }
try {
openConf.put("kyuubi.client.ipAddress",
InetAddress.getLocalHost().getHostAddress());
} catch (UnknownHostException e) {
