kbendick commented on issue #3050: URL: https://github.com/apache/iceberg/issues/3050#issuecomment-908590215
Oh that's great. Thank you @stevenzwu for the info. We are hoping to reduce the tight coupling of Hadoop `Configuration` from the catalogs, as some folks have pointed out that it's difficult (or not presently possible as far as they know) to use things like `GlueCatalog` which don't actually require Hadoop in environments that are difficult to configure (specifically when using `AWS Kinesis Data Analytics`). Further information can be found in this issue: https://github.com/apache/iceberg/issues/3044 Specifically, S3FileIO doesn't actually make use of anything from the `Configurable` interface, so there is an effort to try to make it so that `GlueCatalog` (and also `DynamoDbCatalog`) can be used in environments where Hadoop is not present and that is difficult to configure for them. This will greatly help with that goal. Eventually, it might wind up being beneficial to use the class loader that's provided by the new `Context` interface of Flink 1.13's `CatalogFactory`. I'll leave the rest of the discussion over in that issue (and possibly bring it up in that PR, though it's something that can be done in a follow up PR once this open one has been merged in). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
