rdblue commented on pull request #1640: URL: https://github.com/apache/iceberg/pull/1640#issuecomment-717448365
@rymurr, for your `IcebergSource` question: the source can implement a trait to return the catalog and identifier to load instead of returning a table itself. There are two reasons we don't do this already: 1. The catalog must be defined in Spark properties, so we either need to have a "default" catalog or have a way to configure a catalog for the source 2. If we use a catalog, then we still need to support path-based tables. We will need to add a way to pass a path as an identifier to the catalog and have it load using `HadoopTables` Figuring out how we want to do this shouldn't be too difficult. We just found it easier to keep the existing behavior for the last release since there weren't other catalogs at the time. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
