crepererum commented on issue #3777: URL: https://github.com/apache/arrow-datafusion/issues/3777#issuecomment-1310242554
IMHO it makes sense to make the catalog interfaces async. FWIW IOx gets around this because we "discover" the whole catalog/tables/schemas before we build the DataFusion data structures. This mostly works because our APIs are scoped in a way that allows that. However midterm I think even IOx could be improved by async schema exploration, because this would lower our the amount of data we need to unconditionally prefetech/prepare (e.g. we would only need to construct schemas for tables that are actually touched by a query). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
