This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch branch-1.9
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/branch-1.9 by this push:
new bcb8f29350 [KYUUBI #6635] Fix Hive 2.3.10 integration tests
bcb8f29350 is described below
commit bcb8f293509321f665de0e2d42244c1a9920be34
Author: Cheng Pan <[email protected]>
AuthorDate: Wed Aug 21 20:22:43 2024 +0800
[KYUUBI #6635] Fix Hive 2.3.10 integration tests
# :mag: Description
## Issue References ๐
The current Hive 2.3.10 integration tests are broken, the root cause is
CALCITE-1224
```
2024-08-20T12:20:55,405 ERROR [HiveTBinaryFrontendHandler-Pool: Thread-49]
org.apache.kyuubi.util.KyuubiUncaughtExceptionHandler - Uncaught exception in
thread HiveTBinaryFrontendHandler-Pool: Thread-49
java.lang.NoSuchMethodError:
com.fasterxml.jackson.core.JsonGenerator.writeStartArray(Ljava/lang/Object;I)V
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:78)
~[jackson-databind-2.12.0.jar:2.12.0]
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
~[jackson-databind-2.12.0.jar:2.12.0]
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
~[jackson-databind-2.12.0.jar:2.12.0]
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
~[jackson-databind-2.12.0.jar:2.12.0]
at
com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4485)
~[jackson-databind-2.12.0.jar:2.12.0]
at
com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3699)
~[jackson-databind-2.12.0.jar:2.12.0]
at
org.apache.hive.service.cli.operation.SQLOperation.getTaskStatus(SQLOperation.java:518)
~[hive-service-2.3.10.jar:2.3.10]
at
org.apache.hive.service.cli.operation.Operation.getStatus(Operation.java:143)
~[hive-service-2.3.10.jar:2.3.10]
at
org.apache.kyuubi.engine.hive.operation.HiveOperation.runInternal(HiveOperation.scala:62)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.operation.AbstractOperation.run(AbstractOperation.scala:175)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.session.AbstractSession.runOperation(AbstractSession.scala:103)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.engine.hive.session.HiveSessionImpl.runOperation(HiveSessionImpl.scala:59)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.session.AbstractSession.$anonfun$executeStatement$1(AbstractSession.scala:133)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.session.AbstractSession.withAcquireRelease(AbstractSession.scala:84)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.session.AbstractSession.executeStatement(AbstractSession.scala:130)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.service.AbstractBackendService.executeStatement(AbstractBackendService.scala:66)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.service.TFrontendService.ExecuteStatement(TFrontendService.scala:253)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.shaded.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1670)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.shaded.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1650)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:35)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
org.apache.kyuubi.shaded.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_422]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_422]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_422]
```
## Describe Your Solution ๐ง
Unpack `avatica-1.8.0.jar` and delete Jackson classes to workaround
CALCITE-1224
## Types of changes :bookmark:
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
This PR should recover CI.
---
# Checklist ๐
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6635 from pan3793/hive-2310-fix.
Closes #6635
334bc5c39 [Cheng Pan] nit
22ae945fb [Cheng Pan] fix
dec03196d [Cheng Pan] Fix Hive 2.3.10 integration tests
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit df8448c95f2a04a4b7005a3032de636ec5555317)
Signed-off-by: Cheng Pan <[email protected]>
---
.github/workflows/master.yml | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml
index ed36ddef4a..3d60bbea8e 100644
--- a/.github/workflows/master.yml
+++ b/.github/workflows/master.yml
@@ -339,6 +339,19 @@ jobs:
run: |
TEST_MODULES="externals/kyuubi-hive-sql-engine,integration-tests/kyuubi-hive-it"
./build/mvn ${MVN_OPT} ${{ matrix.hive-archive }} -pl
${TEST_MODULES} -am clean install -DskipTests
+ # unpack avatica-1.8.0.jar and delete Jackson classes to workaround
CALCITE-1224
+ if [[ "${{ matrix.hive-archive }}" ==
*apache-hive-2.3.10-bin.tar.gz* ]]; then
+
HIVE_2310_LIB="$PWD/externals/kyuubi-download/target/apache-hive-2.3.10-bin/lib"
+ mkdir avatica-1.8.0-unpack
+ cd avatica-1.8.0-unpack
+ jar xf $HIVE_2310_LIB/avatica-1.8.0.jar
+ rm -rf com/fasterxml/jackson org/slf4j
+ jar cf avatica-1.8.0-patched.jar .
+ rm $HIVE_2310_LIB/avatica-1.8.0.jar
+ mv avatica-1.8.0-patched.jar $HIVE_2310_LIB/
+ cd ..
+ rm -rf avatica-1.8.0-unpack
+ fi
./build/mvn ${MVN_OPT} ${{ matrix.hive-archive }} -pl
${TEST_MODULES} test
- name: Upload test logs
if: failure()
