roryqi commented on code in PR #14194:
URL: https://github.com/apache/gravitino/pull/14194#discussion_r4397450645
##########
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/TableColumnBaseSQLProvider.java:
##########
@@ -141,6 +141,22 @@ public String selectColumnIdByTableIdAndName(
+ " ORDER BY c.table_version DESC, c.id DESC LIMIT 1";
}
+ /** Returns the SQL for a batched active-column lookup. */
+ public String selectColumnIdsByTableIdAndNames(Long tableId, List<String>
columnNames) {
Review Comment:
Addressed in 2cdc4e547d. Added @TestTemplate coverage in
TestTableColumnMetaService and TestTagMetaService for both batch APIs:
499/500/501 boundaries, 5,000-column reads, rename/delete/same-name
replacement, missing columns, and per-source multi-valued tag assignments
across tables. The tests also cover duplicate requests and untagged columns.
All 48 H2 storage-service tests and the four relevant cases on each of MySQL
and PostgreSQL passed. Spotless also passed.
##########
core/src/main/java/org/apache/gravitino/policy/ObjectPolicyResolver.java:
##########
@@ -104,18 +99,52 @@ public Match[] resolveMatches(String metalake,
MetadataObject metadataObject) {
return new Match[0];
}
- Map<String, ResolvedTag> tagsByName =
- Arrays.stream(effectiveTags)
- .collect(
- Collectors.toMap(
- resolvedTag -> resolvedTag.tag().name(),
- resolvedTag -> resolvedTag,
- (left, right) -> left,
- LinkedHashMap::new));
List<NameIdentifier> tagIdentifiers =
- tagsByName.keySet().stream()
+ Arrays.stream(effectiveTags)
+ .map(resolvedTag -> resolvedTag.tag().name())
+ .distinct()
.map(tagName -> NameIdentifierUtil.ofTag(metalake, tagName))
.collect(Collectors.toList());
+ return selectMatches(effectiveTags, loadRelations(tagIdentifiers,
metadataObject));
+ }
+
+ /**
+ * Resolves effective policies for multiple objects using shared tag and
policy reads.
+ *
+ * @param metalake the metalake name
+ * @param metadataObjects the objects to resolve
+ * @return policies keyed by requested object
+ */
+ public Map<MetadataObject, PolicyEntity[]> resolve(
+ String metalake, List<MetadataObject> metadataObjects) {
+ if (metadataObjects.isEmpty()) {
+ return new LinkedHashMap<>();
+ }
+ Map<MetadataObject, ResolvedTag[]> tagsByObject =
+ effectiveTagResolver.resolveWithSources(metalake, metadataObjects);
+ Map<String, NameIdentifier> tagIdentifiers = new LinkedHashMap<>();
+ tagsByObject
+ .values()
+ .forEach(
+ tags -> {
+ for (ResolvedTag resolvedTag : tags) {
+ TagEntity tag = resolvedTag.tag();
+ tagIdentifiers.putIfAbsent(
+ tag.name(), NameIdentifierUtil.ofTag(metalake,
tag.name()));
+ }
+ });
+ List<RelationalEntity<?>> relations =
+ tagIdentifiers.isEmpty()
+ ? Collections.emptyList()
+ : loadRelations(new ArrayList<>(tagIdentifiers.values()),
metadataObjects);
+ Map<MetadataObject, PolicyEntity[]> result = new LinkedHashMap<>();
+ tagsByObject.forEach(
+ (object, tags) -> result.put(object, policies(selectMatches(tags,
relations))));
Review Comment:
Addressed in 2cdc4e547d. Shared policy relations are now indexed by source
tag name, so each object matches only relations for its effective tags.
Original relation order is preserved.
Added a 1,000-column regression test that verifies 2,000 source visits,
including index construction, instead of 1,000,000. Additional assertions cover
policy order, deduplication, disabled policies, and direct-match precedence
over inherited matches. All eight resolver tests passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
