Hi, Opened a small fix for NoSQL maintenance where spilled index stripes can get purged even though the catalog still needs them [1].
Once an index goes past maxEmbeddedIndexSize, the entries live in IndexStripeObj (ix). CatalogRetainedIdentifier walks things like nameToObjRef / acls / policy mappings during identify, but only indexRetain’s a subset (stableIdToName, locations, changes). refRetainIndexToSingleObj keeps the entity objs, not the stripe segments. After purge the next read can fail with “references a reference index, which does not exist”. The PR makes refRetainIndexToSingleObj also indexRetain the IndexContainer it walks, and fills in the manual walks (CatalogsObj.nameToObjRef, policy mappings, PrincipalsObj.byClientId). There’s a regression that lowers maxEmbeddedIndexSize so spill actually happens in the test.. Curious if auto-retaining inside refRetainIndexToSingleObj feels right, or if you’d rather keep that explicit at every caller. Also if there are other indexes we should be keeping that identify doesn’t walk today. Thanks *Vignesh* [1] https://github.com/apache/polaris/pull/5701
