openinx commented on a change in pull request #2799:
URL: https://github.com/apache/iceberg/pull/2799#discussion_r667571844
##########
File path: flink/src/main/java/org/apache/iceberg/flink/CatalogLoader.java
##########
@@ -51,7 +51,9 @@ static CatalogLoader hadoop(String name, Configuration
hadoopConf, Map<String, S
}
static CatalogLoader hive(String name, Configuration hadoopConf, Map<String,
String> properties) {
- return new HiveCatalogLoader(name, hadoopConf, properties);
+ String hiveConfDir = properties.get(FlinkCatalogFactory.HIVE_CONF_DIR);
+ Configuration newHadoopConf =
FlinkCatalogFactory.mergeHiveConf(hadoopConf, hiveConfDir);
Review comment:
You are using flink datastream API to submit the flink jobs right ? In
theory, the `hadoopConf` cloud be passed as any hadoop `Configuration` if your
want (For example, you can add hive-site.xml to `hadoopConf` outside this
method). Another side, we could load the `HiveCatalog` without specifying any
hive-site.xml configurations because we will set all the necessary key-values
configurations from `propeties` to `hiveConf` ( see
[code](https://github.com/apache/iceberg/blob/90225d6c9413016d611e2ce5eff37db1bc1b4fc5/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java#L96)).
So all you need to do is filing the correct hive properties in the
`Map<String, String> properties`:
```java
Map<String, String> properties = Maps.newHashMap();
properties.set("uri", "thrift://localhost:9083");
properties.set("warehouse", "hdfs://nn:9090/path/to/warehouse");
properties.set("clients", "5");
```
Then you cloud load the correct `HiveCatalog`. So I think we don't need
to have this changes in our master codebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
