lirui-apache commented on a change in pull request #8616: [FLINK-12718][hive]
allow users to specify hive-site.xml location to configure hive metastore
client in HiveCatalog
URL: https://github.com/apache/flink/pull/8616#discussion_r291872326
##########
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/HiveCatalog.java
##########
@@ -109,27 +114,51 @@
private HiveMetastoreClientWrapper client;
- public HiveCatalog(String catalogName, String hivemetastoreURI) {
- this(catalogName, DEFAULT_DB, getHiveConf(hivemetastoreURI));
+ public HiveCatalog(String catalogName, @Nullable String
defaultDatabase, @Nullable String hiveSiteFilePath) {
+ this(catalogName,
+ defaultDatabase == null ? DEFAULT_DB : defaultDatabase,
+ getHiveConf(hiveSiteFilePath));
}
- public HiveCatalog(String catalogName, HiveConf hiveConf) {
- this(catalogName, DEFAULT_DB, hiveConf);
+ public HiveCatalog(String catalogName, @Nullable String
defaultDatabase, @Nullable URL hiveSiteUrl) {
+ this(catalogName,
+ defaultDatabase == null ? DEFAULT_DB : defaultDatabase,
+ getHiveConf(hiveSiteUrl));
}
- public HiveCatalog(String catalogName, String defaultDatabase, HiveConf
hiveConf) {
- super(catalogName, defaultDatabase);
- this.hiveConf = checkNotNull(hiveConf, "hiveConf cannot be
null");
+ public HiveCatalog(String catalogName, @Nullable String
defaultDatabase, @Nullable HiveConf hiveConf) {
+ super(catalogName, defaultDatabase == null ? DEFAULT_DB :
defaultDatabase);
+
+ this.hiveConf = hiveConf == null ? getHiveConf("") : hiveConf;
LOG.info("Created HiveCatalog '{}'", catalogName);
}
- private static HiveConf getHiveConf(String hiveMetastoreURI) {
-
checkArgument(!StringUtils.isNullOrWhitespaceOnly(hiveMetastoreURI),
"hiveMetastoreURI cannot be null or empty");
+ private static HiveConf getHiveConf(String filePath) {
+
+ URL url = null;
+
+ if (!StringUtils.isNullOrWhitespaceOnly(filePath)) {
+ try {
+ url = new File(filePath).toURI().toURL();
+
+ LOG.info("Successfully loaded '{}'", filePath);
+
+ } catch (MalformedURLException e) {
+ throw new CatalogException(
+ String.format("Failed to get
hive-site.xml from the given path '%s'", filePath), e);
+ }
+ }
+
+ return getHiveConf(url);
+ }
+
+ private static HiveConf getHiveConf(URL hiveSiteUrl) {
+ LOG.info("Setting hive-site location as {}", hiveSiteUrl);
+
+ HiveConf.setHiveSiteLocation(hiveSiteUrl);
Review comment:
Wondering will this incur any concurrency issue, e.g. when user access
tables from different Hive catalogs? I think at least some constructors of
HiveConf is no longer safe to use after this change, like `HiveConf()` and
`HiveConf(Class<?> cls)` which automatically load the specified hive-site.xml.
I think it's better not to rely on static fields of HiveConf. Instead, we
can load the hive-site (perhaps `Properties::loadFromXML`?) ourselves and
`HiveConf::hiveSiteURL` should always be set to null.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
