alamb commented on issue #10782: URL: https://github.com/apache/datafusion/issues/10782#issuecomment-2226248126
> To move TableProvider out we need to avoid dependency on SessionState for scan function, but there is CatalogProviderList in SessionState 🤔 we could potentially move SessionState into the catalog crate In fact the more I think about it, it seems to make some sort of sense to have a crate with the catalog traits / interfaces such as `SessionState`, `Catalog`, `TableProvider` that can be shared broadly. Then we could make separate crates that actually had the implementations like * `datafusion-catalog`: contains traits * `datafusion-catalog-basic` (not a good name`: `MemoryCatalog`, etc, `InformationCatalog` (potentially `DynamicFileCatalog` / https://github.com/apache/datafusion/pull/11035 from @goldmedal ) * `datafusion-catalog-listing`: `ListingTable` And we could put the individual datasources like * `datafusion-datasource-parquet` * `datafusion-datasource-csv` * `datafusion-datasource-json` * ... 🤔 I think https://github.com/apache/datafusion/issues/11320 and https://github.com/apache/datafusion/pull/11403 from @Omega359 makes this easier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
