alamb opened a new issue, #10782: URL: https://github.com/apache/datafusion/issues/10782
### Is your feature request related to a problem or challenge? As @goldmedal started trying to move the DynamicFileProvider so others could use it in https://github.com/apache/datafusion/pull/10745 I think it is clear that there is not a good way to add additional catalog support in the core without everything being intertwined. Thus I think we should try and extract the different catalog providers out of datafusion core so it it easier ### Describe the solution you'd like I suggest the following final layout: 1. traits like `CatalogProvider`, `SchemaProvider`, etc in a new crate `datafusion-catalog` (since these traits rely on table provider, etc I think this can't be in `datafusion-common` or `datafusion-expr`) 2. The built in `Memory*` providers are in `datafusion-catalog` 3. The bult in `InformationSchema` providers are in `datafusion-catalog` 4. The newly proposed `DynamicFileCatalog` in `datafusion-catalog` 5. (eventually) the LIstingTableProvider (which is by far the most complicated) moved to its own crate `datafusion-catalog-listing` ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
