lcspinter edited a comment on pull request #2129: URL: https://github.com/apache/iceberg/pull/2129#issuecomment-771698156
@rdblue I followed your recommendations with a few minor changes. This is how it works now: - If the table has an `iceberg.catalog=<catalog_name>` property then use all the properties prefixed with `iceberg.catalog.<catalog_name>` from the hive global config for catalog instantiation. There is one exception, when `iceberg.catalog=location_based_table`. In this case, no catalog is needed, since we are loading the table based on location. - If the table is missing `iceberg.catalog`, fall back to `iceberg.mr.catalog` like you recommended Previously we supported the so-called "custom" catalogs, and these type of catalogs were instantiated through a catalog loader class (`iceberg.mr.catalog.loader.class`). The catalog loader class collected all the relevant properties which are important for that specific catalog type and instantiated the catalog. Right now I'm using CatalogUtil.buildIcebergCatalog() to get a catalog instance. This method expects the mandatory catalog specific properties. In order to assemble this collection, I would need to know what properties are required, but since we are talking about a custom catalog, basically this can be anything. In the end, I decided the remove the support of custom catalog configuration through the `iceberg.mr.catalog.loader.class` property. Of course, it is still possible to configure custom catalogs, but only through the `iceberg.catalog` table property. This way I know that all the catalog specific parameters are prefixed, and I can pass them to the CatalogUtil.buildIcebergCatalog(). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
