beliefer commented on a change in pull request #24575: [SPARK-27670][SQL]Add HA
for HiveThriftServer2 based on HiveServer2.
URL: https://github.com/apache/spark/pull/24575#discussion_r284108565
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala
##########
@@ -107,6 +109,25 @@ object HiveThriftServer2 extends Logging {
} else {
None
}
+
+ // HiveThriftServer2 HA
+ val metaVersion = IsolatedClientLoader.hiveVersion(
+
SparkSQLEnv.sqlContext.conf.getConf(HiveUtils.HIVE_METASTORE_VERSION))
+ metaVersion match {
+ case hive.v12 | hive.v13 =>
+ logWarning("HiveThriftServer2 HA is compatible with Hive metastore
version is " +
+ "0.14.0 or higher.")
+ case _ =>
+ if (executionHive.conf.getBoolVar(
+ ConfVars.HIVE_SERVER2_SUPPORT_DYNAMIC_SERVICE_DISCOVERY)) {
+ val method = server.getClass.getSuperclass.getDeclaredMethod(
+ "addServerInstanceToZooKeeper",
Review comment:
Because `HiveThriftServer2` not support HA based on Hive, so SPARK-11100
occurs.
I have investigated the `addServerInstanceToZooKeeper` method in Hive，it was
supported from version 0.14.0 to version 3.1.1
This change has been running normally for more than a year on our production
environment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
