aokolnychyi commented on pull request #1783: URL: https://github.com/apache/iceberg/pull/1783#issuecomment-732410497
> Last, how to handle path URIs: I talked with Anton and Russell about this yesterday and we think that we should make it so that every Spark catalog can load special path identifiers, just like we do today in IcebergSource. To do this, we would need to have a way to pass an identifier that signals this behavior back from SupportsCatalogOptions, like iceberg.hdfs://nn:8020/path/to/table. Then detect those and return Hadoop tables for them. +1 to that. I think we have enough info to get #1306 done. While it definitely makes sense to accept `iceberg.path` in the session catalog, shall we also allow just `path` alone in non-session catalogs? Otherwise, it is going to be weird if the catalog name is iceberg too. The identifier will be `iceberg.iceberg.path` then. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
