Hi devs,
While working on adapting the external Hive connector for Flink 2.0
(FLINK-37094), we discovered that the Hive 3.1 profile (-Phive3) is
broken on Java 9+ due to a bug in Hive's SessionState constructor.
## Problem
Hive 3.1's SessionState (line 413) casts the system classloader to
URLClassLoader:
(URLClassLoader) Thread.currentThread().getContextClassLoader()
On Java 9+, the application classloader is
jdk.internal.loader.ClassLoaders$AppClassLoader, which does not extend
URLClassLoader. This causes a ClassCastException on any code path that
initializes SessionState — including HiveParser, HiveDialect, and
HiveDynamicTableFactory.
Since Flink 2.0 requires Java 11+, this means Hive 3.1 dialect/parser
features are completely broken on Flink 2.0.
25 tests fail in the Hive 3.1 profile on Java 11+:
- HiveDynamicTableFactoryTest (4)
- HiveTableSourceStatisticsReportTest (8)
- HiveTableCompactSinkParallelismTest (4)
- HiveSourceTest (4)
- HiveDialectSupportSQL11ReservedKeywordAsIdentifierITTest (3)
- HiveDialectQueryPlanTest (1)
- HiveFunctionWrapperTest (1)
The Hive 2.3 profile is unaffected.
## Upstream status
This is a known Hive bug (HIVE-21584 [1]), fixed in Hive 4.0 via
HIVE-27508 [2]. The fix replaces the URLClassLoader cast with
SessionState.class.getClassLoader(). Hive 3.1 is EOL and will not
receive a backport.
## Reproducing
On the current main branch with Java 11+:
JAVA_HOME=/path/to/java-11 mvn verify \
-pl flink-connector-hive -am -Phive3
## Question
Should the Flink Hive connector work around this for Hive 3.1 users?
One approach would be to wrap the thread context classloader in a
URLClassLoader before calling new SessionState() in HiveSessionState.
Alternatively, we could document Hive 3.1 as unsupported on Java 11+
and recommend users upgrade to Hive 4.
Thoughts?
[1] https://issues.apache.org/jira/browse/HIVE-21584
[2] https://issues.apache.org/jira/browse/HIVE-27508
Best,
Jayesh
