akshaychitneni commented on issue #1829:
URL:
https://github.com/apache/datafusion-ballista/issues/1829#issuecomment-5553421084
Planning to start on .cache(). Two things I'd like consensus on first:
- Regarding Keying, the cache_id comes from the client in BallistaCacheNode
— keep the current per-call Uuid to start (already gives reuse when you hold
the cached DataFrame; scheduler keys a registry on (session_id, cache_id) for
hit/miss), and add Spark-style canonical-plan keying for cross-call dedup later.
- I see cache and checkpoint differ only in the materializer, so I'd build
on #1993's plumbing rather than duplicate it
Everything else follows as discussed, in small PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
