alamb commented on issue #19573: URL: https://github.com/apache/datafusion/issues/19573#issuecomment-3710480077
> > I have a draft PR [#19616](https://github.com/apache/datafusion/pull/19616) for one approach I considered, which is to continue using a session level cache as is currently but make the cache key table scoped. The PR contains a short description and some cli testing results to verify that. > Another approach I tried is to attach an independent cache to each table (e.g. add a field in `ListingTable` struct) and have a session level map from table reference to table cache to track and display at session level. However as I was experimenting, I found it a bit tricky to handle user-provided `ListFileCache` implementation, which is currently given as a `Arc<dyn ListFilesCache>`. To support independent table scoped cache, I think we may need some kind of `ListFilesCacheFactory` instead? I agree this one sounds tricky and unecessary -- if we can make https://github.com/apache/datafusion/pull/19616 work let's go with that one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
