rdblue commented on a change in pull request #1737:
URL: https://github.com/apache/iceberg/pull/1737#discussion_r519052825
##########
File path: site/docs/custom-catalog.md
##########
@@ -127,8 +137,111 @@ public class CustomCatalog extends BaseMetastoreCatalog {
// Example service to rename table
CustomService.renameTable(from.namepsace().level(0), from.name(),
to.name());
}
+
+ // implement this method to read catalog name and properties during
initialization
+ public void initialize(String name, Map<String, String> properties) {
+ }
}
```
+
+Catalog implementations can be dynamically loaded in most compute engines.
+For Spark and Flink, you can specify the `catalog-impl` catalog property to
load it.
+Read the [Configuration](../configuration/#catalog-properties) section for
more details.
+For MapReduce, implement `org.apache.iceberg.mr.CatalogLoader` and set Hadoop
property `iceberg.mr.catalog.loader.class` to load it.
+If your catalog must read Hadoop configuration to access certain environment
properties, make your catalog implement `org.apache.hadoop.conf.Configurable`.
+
+### Custom file IO implementation
+
+Extend `FileIO` and provide implementation to read and write data files
+
+Example:
+```java
+public class CustomFileIO implements FileIO {
Review comment:
This isn't really needed for a custom catalog, and it is an option for
the built-in catalogs. I think maybe we should move this to a doc on
customizing tables, instead of here. Or, maybe just make it more clear that
this isn't required. I think it is somewhat common for people to want to
customize catalogs, and I don't want to make it seem like it is harder than it
already is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
