nevzheng commented on issue #12254: URL: https://github.com/apache/gravitino/issues/12254#issuecomment-5144790487
Hi @laserninja, thanks for driving this; points 1-4 are a great starting structure. Would you be up for taking it one step further and owning a short design doc? Beyond settling the default, the doc can serve as documentation for future people working on this: the design and the assumptions behind it, in one place. I'm aligned on the direction: cache on by default, with good configurability. What I'd like is for us to be more precise about the specification. Good, correct caches need proper specification. On the four points: 1. **Default on or off.** Aligned. My principle here is good defaults + good configurability: the default serves the common journey with zero configuration, and knobs cover the tail. 2. **Memory footprint.** Agreed, this is a key parameter to cover. I've added it to the cache design questions below. 3. **Correctness.** This matters; we should consider it carefully rather than treat it as settled. Also added to the design questions below. 4. **Multi-replica deployments.** Is there a use case we're serving today? Distributed caching can get tricky and I don't want us caught up on it at the moment, so I've moved it to future considerations. ## Requirements In the doc, could you recap the CUJs the cache is serving? For example: batched scan planning over large tables (https://github.com/apache/gravitino/pull/12194, where redeeming a plan-task replans on a cache miss), repeated identical scans from BI/dashboard refreshes, and the operator running the server on default configuration. Starting from the CUJs and working backwards will sharpen the requirements. From those CUJs, can you define the technical requirements? ## Cache Design Questions I want to make sure the doc covers these; the five I care most about: 1. What is the cache key, and what invariants must it maintain? 2. How is the cache sized and parameterized? Per-catalog or server-wide? 3. One pool, or separate tiers for the batched-plan and repeat-scan journeys? 4. What are the expiry and invalidation policies? 5. What are good defaults, given all of the above? References to caching best practices would be helpful here too. ## Alternatives Can the doc also lay out the implementation option space rather than assuming an in-heap cache? For example: in-process (what we have today), an external store such as Redis/Valkey, or a custom implementation via the existing `scan-plan-cache-impl` plugin point. Even if we land on local-first, having the options and their trade-offs written down makes the decision durable. An external store intersects the multi-replica question below; recording the option here still lets us defer the distributed design. ## Future Considerations - **FGAC** is on the roadmap; I'm fine deferring it, but it can change how caching works in this system: once row/column-level filtering shapes plan output, results are no longer principal-independent. Worth stating that assumption explicitly in the doc so we don't design ourselves into a corner. - **Multi-replica / distributed caching.** Deferring deliberately: getting the single-node story right first will inform a multi-replica design later. I think this is a good framing and a path forward. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
