JingsongLi edited a comment on pull request #1558: URL: https://github.com/apache/iceberg/pull/1558#issuecomment-705938309
Hi @rdblue @zhangjun0x01 - Spark has a built-in hive integration. So its hadoop conf contains `hive-site.xml`. (As @rdblue said, in `SharedState.loadHiveConfFile`) - But in Flink, Hive integration is optional. So the `HadoopUtils.getHadoopConfiguration(..)` not contains `hive-site.xml`. So just like @zhangjun0x01 said, `flink-connector-hive` provides a simple way, users need to pass a `hiveConfDir` to Flink `HiveCatalog`. (Now the `hiveConfDir` must be a local path) I created https://issues.apache.org/jira/browse/FLINK-19541 for track it. Personally, I think we can provide two way in Flink: - From `hiveConfDir`, it should be URI which contains scheme and full path. This means it can be loaded by Hadoop Filesystem. - From the classpath, just like Spark do. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
