sunchao commented on pull request #32446: URL: https://github.com/apache/spark/pull/32446#issuecomment-835419237
@dongjoon-hyun @wangyum @viirya @HyukjinKwon I just realized that, due to [HIVE-13149](https://issues.apache.org/jira/browse/HIVE-13149), when a `HiveMetaStoreClient` is initialized, it will always make a copy of the provided `HiveConf`. Therefore, the object reference comparison in `Hive.getWithFastCheck` will always return false and subsequently causing a new `Hive` and new HMS connection to be created in `withHiveState`. @wangyum has added a new API in [HIVE-21563](https://issues.apache.org/jira/browse/HIVE-21563) which disables both factcheck and loading permanent functions. It is going to be shipped in Hive 2.3.9. In addition, this PR itself is not sufficient to make Spark work with HMS service of 1.2 or lower, as [HIVE-24608](https://issues.apache.org/jira/browse/HIVE-24608) is also required (we have enabled this already internally). It will also be part of the Hive 2.3.9 release. Because of this, I think we should revert this PR and resolve this JIRA after Hive 2.3.9 release. What do you think? sorry for missing the issue in the first place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
