rymurr commented on pull request #1783: URL: https://github.com/apache/iceberg/pull/1783#issuecomment-733906396
Thanks all for the input. So I think there is enough info that I can clean up this PR to use `SupportsCatalogOptions` and the process described above to extract a table/catalog from the source. * If there is a catalog in the identifier -> use that catalog * if no catalog in the identifier -> use session/default catalog (favouring Spark3 convention over IcebergSource in 2.4 convention) * if catalog in the identifier isn't an iceberg catalog -> throw * if path then handle with `iceberg.path` option as per above. Seems like should be done as a separate patch? Does that make sense? Am I missing anything? PS Any thoughts on custom catalogs in Spark2.4? I don't know what the migration plan looks like but I feel like Spark2.4 will be a thing for some time. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
