Hi Michael, This sounds like a reasonable proposal to me. Looking forward to the PR :)
Cheers, Dmitri. On Fri, Sep 19, 2025 at 2:26 PM Michael Collado <[email protected]> wrote: > Hey folks > > Building on the loadEntities API added in > https://github.com/apache/polaris/pull/2290 , I'd like to propose two new > APIs to support batch loading of entities in the PolarisMetaStoreManager. > The new loadEntities API is great for avoiding the N+1 queries problem > during lists and I'd like to extend that behavior for batch retrieving > entities outside of list operations. > > The two APIs would be > > ResolvedEntitiesResult loadResolvedEntities( > @Nonnull PolarisCallContext callCtx, > @Nonnull List<EntityNameLookupRecord> entityLookupRecords); > > and > > ResolvedEntitiesResult loadResolvedEntities( > @Nonnull PolarisCallContext callCtx, > @Nonnull PolarisEntityType entityType, > @Nonnull List<PolarisEntityId> entityIds); > > The first API is useful for batch loading entities after a list operation, > but may only require loading a subset of entities (e.g., because a subset > is already in the in-memory cache). The latter supports a batch version of > the loadEntity - also useful for loading entities for the cache without the > need to refresh one by one (consider the Resolver, which does this). > > Notably, both APIs return ResolvedEntitiesResult, rather than > EntitiesResult because I'd like to resolve the discrepancy between the > EntityCache API and the PolarisMetaStoreManager API. Right now, it's > important to directly interact with the cache whereas, if we can > consolidate the API, it could be possible to support a cache-backed > PolarisMetaStoreManager implementation that manages the in-memory cache > invisibly. > > I already have a draft PR with some of the work done, but if everyone > agrees on the additions, I can have a PR ready for review next week some > time. Let me know your thoughts. > > Mike >
