Hi all, I opened GitHub issue #4874 <https://github.com/apache/polaris/issues/4874> after noticing something while reading through the metastore implementations, and I wanted to get some feedback before diving into a PR.
>From what I can tell, the NoSQL backend currently doesn't use the entity cache at all. Since NoSqlMetaStoreManager doesn't implement change tracking, NoSqlMetaStoreManagerFactory ends up returning null instead of creating an InMemoryEntityCache. That means every Resolver operation - principal lookups, catalog resolution, privilege checks, location validation, and so on - goes directly to the backing store. By comparison, the JDBC and in-memory TreeMap implementations both benefit from the existing cache. The details are in issue #4874 <https://github.com/apache/polaris/issues/4874>, but I was curious about the intent here. A few questions: - Is this a known limitation, or is it something that simply hasn't been addressed yet? - Is the expected long-term solution to add change tracking support for NoSQL? - Has anyone considered a lighter-weight approach for NoSQL caching, or are there consistency concerns that make that undesirable? - More generally, should we expect similar performance characteristics across the supported metastore backends, or is this difference intentional? The NoSQL backend is a supported production backend, so the lack of caching stood out to me as a potentially significant behavioral difference rather than just an implementation detail. I'd appreciate any context before I spend time exploring solutions. Thanks, Vignesh
