milenkovicm commented on issue #18893: URL: https://github.com/apache/datafusion/issues/18893#issuecomment-3598959838
It gets a bit complicated when it comes to ballista implementation of cache. There are few things which needed to be implemented: - query planner extension which would be able to handle custom logical plan extension - there is need for a some kind cache manager which would need to handle cache locations - also there is need for job breakdown, a first job which would materialize cache and second which would read the cache ... It gets event more complicated when cache reconciliation and task colocation are considered. I'm not sure if we can or should implement it all for a showcase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
