XN137 commented on code in PR #2261:
URL: https://github.com/apache/polaris/pull/2261#discussion_r2256952076
##########
runtime/service/src/main/java/org/apache/polaris/service/admin/PolarisAdminService.java:
##########
@@ -200,6 +205,43 @@ private Optional<CatalogRoleEntity>
findCatalogRoleByName(String catalogName, St
.map(path -> CatalogRoleEntity.of(path.getRawLeafEntity()));
}
+ private <T> Stream<T> loadEntities(
+ @Nonnull PolarisEntityType entityType,
+ @Nonnull PolarisEntitySubType entitySubType,
+ @Nullable PolarisEntity catalogEntity,
+ @Nonnull Function<PolarisBaseEntity, T> transformer) {
+ List<PolarisEntityCore> catalogPath;
+ long catalogId;
+ if (catalogEntity == null) {
+ catalogPath = null;
+ catalogId = 0;
+ } else {
+ catalogPath = PolarisEntity.toCoreList(List.of(catalogEntity));
+ catalogId = catalogEntity.getId();
+ }
+ // TODO: add loadEntities method to PolarisMetaStoreManager
Review Comment:
afaict under the hood `listEntities` is already fetching the fully fledged
`PolarisBaseEntity` instances from the database (for the jdbc case):
https://github.com/apache/polaris/blob/af69d9f9699d8a0cae1b70839e1b5c1bfc9ba511/persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java#L507-L513
it includes streaming/pagination.
it just happens that the given `transformer` turns `PolarisBaseEntity` into
`EntityNameLookupRecord`... and then later the record is used to look up the
full entity again.
so afaict, the memory footprint would not be very different if we had a
`loadEntities` function (or changed `listEntities` to not only return a
`EntityNameLookupRecord`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@polaris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
