This is an automated email from the ASF dual-hosted git repository.
yuqi1129 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new 3c63c74ff1 [#14681] fix(tag): Normalize metadata object names for tag
operations (#14682)
3c63c74ff1 is described below
commit 3c63c74ff1d19d6592d617035b4b033687825341
Author: Qi Yu <[email protected]>
AuthorDate: Fri Nov 6 20:28:38 2026 +0800
[#14681] fix(tag): Normalize metadata object names for tag operations
(#14682)
### What changes were proposed in this pull request?
Normalize metadata object identifiers before TagManager reads or updates
tag relations. The rules live in one shared helper,
`CapabilityHelpers.normalizeMetadataIdentifier`, which TagManager and
the authorization ID lookup (`MetadataIdConverter`) both use:
- Schema and object names follow the catalog's naming capabilities.
- Tables and the parent table of a column resolve to their physical
names.
- A column keeps the requested name when folding changes it and the
stored table has a column with exactly that name. This keeps exact names
working for catalogs whose custom `normalizeName` folds columns that the
source stored with another case.
- Gravitino-owned Semantic Model names are kept.
Existence checks still use the original request so quoted names are not
normalized twice.
Each operation on a supported non-catalog object loads naming
capabilities after the existence check; table and column operations also
repeat physical table-name resolution. Catalog cache misses can reload
the connector; connectors whose resolver reads the source can add a
source read. A column read from the entity store happens only when
folding changes a column name. Resolution failures propagate before tag
relations are changed.
Authorization change: column authorization now resolves the parent
table's physical name, as table authorization already does, so it checks
the same object the operation changes. In this repository only JDBC
catalogs implement `SupportsTableNameResolution`, and its default
`resolveTableName` returns the name unchanged; a backend that overrides
it adds one call per column ID lookup. A failure while normalizing one
object (including a stored-column read) is remembered for its catalog
for the rest of that authorization request and denies later objects in
it; the `AuthorizationRequestContext` Javadoc now says so.
The separate policy-inheritance lookup in `EffectiveTagResolver`
(#14719) and owner lookups (#14720) are outside this change; both can
reuse the shared helper.
### Why are the changes needed?
An object can pass the catalog's normalized existence check but fail the
relation-store lookup because TagManager uses the original spelling. The
same mismatch affects tag association and disassociation, including
value assignments. The authorization path had its own copy of the rules,
which did not resolve the physical table name for columns.
Fixes #14681
### Does this PR introduce _any_ user-facing change?
Metadata-object tag operations accept equivalent object names according
to the catalog's existing naming rules. Case-sensitive object names,
exact stored column names, tag names, and tag values are preserved.
Column authorization on catalogs with physical table-name resolution now
targets the resolved table. No API, configuration, or storage-schema
changes.
### How was this patch tested?
After merging `main` at `61d5c84753`, all 282 targeted cases passed (122
core and 160 server-common; zero failures, errors, or skips). The
conflict was limited to imports in `TestCapabilityHelpers`; both the
tag-normalization tests and upstream existing-column-name tests are
retained.
- Regression tests exercise real H2 relation storage for
case-insensitive names, custom uppercase normalization, quoted names,
physical table-name resolution, Semantic Model naming, Glue-style rules
(schema and table fold, columns keep case), exact stored column names,
and all ten tag-supported metadata object types. A resolution failure
test checks that add, remove and value assignment fail and leave
existing tags unchanged.
- With the old TagManager, 13 runs of the 6 original regression methods
fail. Removing the exact-column match fails the new column tests in
TestTagManager and TestCapabilityHelpers. Restoring the old
`MetadataIdConverter` fails the new column physical-name test.
- `:core:coreUnitTest` (101 tests): TestTagManager, TestPolicyManager,
TestEffectiveTagResolver, TestTagHookDispatcher, TestCapabilityHelpers,
TestTableNormalizeDispatcher, TestMetadataObjectUtil.
- `:server-common:test` (160 tests): TestMetadataIdConverter (including
the exact stored column name through `normalizeMetadataObject`),
TestJcasbinAuthorizer, TestJcasbinMetadataIdCacheNormalization,
TestJcasbinModelCacheInvalidation, TestJcasbinAuthorizationLookups.
`:core:coreUnitTest` TestAuthorizationRequestContext (21 tests) also
passed.
- Spotless, compile and `javadoc` passed for core and server-common.
Tests use embedded H2; live connector and Docker-backed database tests
were not run.
---
.../authorization/AuthorizationRequestContext.java | 7 +-
.../gravitino/catalog/CapabilityHelpers.java | 76 ++++
.../java/org/apache/gravitino/tag/TagManager.java | 33 +-
.../gravitino/catalog/TestCapabilityHelpers.java | 87 ++++
.../apache/gravitino/policy/TestPolicyManager.java | 16 +-
.../org/apache/gravitino/tag/TestTagManager.java | 505 ++++++++++++++++++++-
.../server/authorization/MetadataIdConverter.java | 38 +-
.../authorization/TestMetadataIdConverter.java | 86 ++++
8 files changed, 804 insertions(+), 44 deletions(-)
diff --git
a/core/src/main/java/org/apache/gravitino/authorization/AuthorizationRequestContext.java
b/core/src/main/java/org/apache/gravitino/authorization/AuthorizationRequestContext.java
index 22c47cf23f..a7c139700f 100644
---
a/core/src/main/java/org/apache/gravitino/authorization/AuthorizationRequestContext.java
+++
b/core/src/main/java/org/apache/gravitino/authorization/AuthorizationRequestContext.java
@@ -284,10 +284,13 @@ public class AuthorizationRequestContext {
}
/**
- * Normalizes a raw name while remembering capability failures for its
catalog in this request.
+ * Normalizes a raw name while remembering normalization failures for its
catalog in this request.
*
* <p>Catalog lookups are serialized so concurrent objects cannot retry a
failed initialization.
- * Missing catalogs are not memoized; a fresh request can retry any
capability failure.
+ * Any failure except not found is remembered for the whole catalog,
including one that only
+ * concerns a single object, such as a physical table-name or stored-column
read. Later objects in
+ * the same catalog then fail in this request, which denies them. Missing
catalogs are not
+ * memoized; a fresh request retries every failure.
*
* @param key the raw metadata object's cache key
* @param catalogIdentifier the containing catalog, including its metalake
diff --git
a/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java
b/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java
index 51f898344e..a46bf623d2 100644
--- a/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java
+++ b/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java
@@ -23,7 +23,12 @@ import static
org.apache.gravitino.rel.expressions.transforms.Transforms.NAME_OF
import static
org.apache.gravitino.utils.NameIdentifierUtil.getCatalogIdentifier;
import com.google.common.base.Preconditions;
+import com.google.common.collect.ObjectArrays;
+import java.io.IOException;
import java.util.Arrays;
+import java.util.function.Supplier;
+import org.apache.gravitino.Entity;
+import org.apache.gravitino.EntityStore;
import org.apache.gravitino.NameIdentifier;
import org.apache.gravitino.Namespace;
import org.apache.gravitino.connector.CatalogOperations;
@@ -31,7 +36,9 @@ import
org.apache.gravitino.connector.SupportsTableNameResolution;
import org.apache.gravitino.connector.capability.Capability;
import org.apache.gravitino.connector.capability.CapabilityResult;
import org.apache.gravitino.exceptions.NoSuchCatalogException;
+import org.apache.gravitino.exceptions.NoSuchEntityException;
import org.apache.gravitino.file.FilesetChange;
+import org.apache.gravitino.meta.TableEntity;
import org.apache.gravitino.model.ModelChange;
import org.apache.gravitino.rel.Column;
import org.apache.gravitino.rel.TableChange;
@@ -108,6 +115,63 @@ public class CapabilityHelpers {
}
}
+ /**
+ * Normalizes the identifier of an existing catalog object to the identifier
stored by Gravitino.
+ * Tag and authorization lookups use it, so they find the same stored object.
+ *
+ * <ul>
+ * <li>Names follow the catalog's case rules for each scope.
+ * <li>A table, or the parent table of a column, is resolved to its
physical name by {@link
+ * #resolvePhysicalTableName}.
+ * <li>A column keeps the requested name when its table has a stored
column with exactly that
+ * name. A custom {@link Capability#normalizeName} can fold a name
that the source stored
+ * with another case, and an exact name must still find that column.
+ * <li>A Semantic Model name is owned by Gravitino, so only its schema is
normalized.
+ * </ul>
+ *
+ * @param ident the entity identifier of the object
+ * @param scope the capability scope of the object
+ * @param catalogManager the catalog manager used to load capabilities and
resolve table names
+ * @param entityStore supplies the entity store that holds the stored
columns of a table. It is
+ * called only for a column whose name the catalog rules change.
+ * @return the normalized identifier
+ * @throws NoSuchCatalogException if the catalog does not exist
+ * @throws RuntimeException if capabilities, the physical table name or the
stored table cannot be
+ * loaded
+ */
+ public static NameIdentifier normalizeMetadataIdentifier(
+ NameIdentifier ident,
+ Capability.Scope scope,
+ CatalogManager catalogManager,
+ Supplier<EntityStore> entityStore) {
+ Capability capability = getCapability(ident, catalogManager);
+ if (scope == Capability.Scope.COLUMN) {
+ NameIdentifier table =
+ resolvePhysicalTableName(
+ applyCaseSensitive(
+ NameIdentifier.of(ident.namespace().levels()),
+ Capability.Scope.TABLE,
+ capability),
+ catalogManager);
+ String column = applyCaseSensitiveOnName(scope, ident.name(),
capability);
+ if (!column.equals(ident.name()) && hasStoredColumn(table, ident.name(),
entityStore.get())) {
+ column = ident.name();
+ }
+ return NameIdentifier.of(
+ Namespace.of(ObjectArrays.concat(table.namespace().levels(),
table.name())), column);
+ }
+
+ if (scope == Capability.Scope.SEMANTIC_MODEL) {
+ return NameIdentifier.of(
+ applyCaseSensitive(ident.namespace(), scope, capability),
ident.name());
+ }
+
+ NameIdentifier normalized = applyCaseSensitive(ident, scope, capability);
+ return scope == Capability.Scope.TABLE
+ ? resolvePhysicalTableName(normalized, catalogManager)
+ : normalized;
+ }
+
public static Column[] applyCapabilities(Column[] columns, Capability
capabilities) {
return Arrays.stream(columns)
.map(c -> applyCapabilities(c, capabilities))
@@ -289,6 +353,18 @@ public class CapabilityHelpers {
return namespace;
}
+ private static boolean hasStoredColumn(
+ NameIdentifier table, String column, EntityStore entityStore) {
+ try {
+ return entityStore.get(table, Entity.EntityType.TABLE,
TableEntity.class).columns().stream()
+ .anyMatch(storedColumn -> storedColumn.name().equals(column));
+ } catch (NoSuchEntityException e) {
+ return false;
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to load stored columns of table: " +
table, e);
+ }
+ }
+
private static boolean hasSchemaParent(Capability.Scope resourceScope) {
switch (resourceScope) {
case TABLE:
diff --git a/core/src/main/java/org/apache/gravitino/tag/TagManager.java
b/core/src/main/java/org/apache/gravitino/tag/TagManager.java
index 63611c1f7c..a8d9f54495 100644
--- a/core/src/main/java/org/apache/gravitino/tag/TagManager.java
+++ b/core/src/main/java/org/apache/gravitino/tag/TagManager.java
@@ -35,6 +35,7 @@ import javax.annotation.Nullable;
import org.apache.gravitino.Entity;
import org.apache.gravitino.EntityAlreadyExistsException;
import org.apache.gravitino.EntityStore;
+import org.apache.gravitino.GravitinoEnv;
import org.apache.gravitino.MetadataObject;
import org.apache.gravitino.NameIdentifier;
import org.apache.gravitino.RelationEdgeTarget;
@@ -42,6 +43,8 @@ import org.apache.gravitino.RelationQuery;
import org.apache.gravitino.RelationUpdate;
import org.apache.gravitino.RelationalEntity;
import org.apache.gravitino.SupportsRelationOperations;
+import org.apache.gravitino.catalog.CapabilityHelpers;
+import org.apache.gravitino.connector.capability.Capability;
import org.apache.gravitino.exceptions.NoSuchEntityException;
import org.apache.gravitino.exceptions.NoSuchMetadataObjectException;
import org.apache.gravitino.exceptions.NoSuchTagException;
@@ -328,11 +331,9 @@ public class TagManager implements TagDispatcher {
public Tag[] listTagsInfoForMetadataObject(String metalake, MetadataObject
metadataObject)
throws NoSuchMetadataObjectException {
- NameIdentifier entityIdent = MetadataObjectUtil.toEntityIdent(metalake,
metadataObject);
+ NameIdentifier entityIdent = checkAndNormalizeMetadataObject(metalake,
metadataObject);
Entity.EntityType entityType =
MetadataObjectUtil.toEntityType(metadataObject);
- MetadataObjectUtil.checkMetadataObject(metalake, metadataObject);
-
try {
checkMetalake(NameIdentifier.of(metalake), entityStore);
List<TagEntity> tags =
@@ -352,12 +353,10 @@ public class TagManager implements TagDispatcher {
public Tag getTagForMetadataObject(String metalake, MetadataObject
metadataObject, String name)
throws NoSuchMetadataObjectException {
- NameIdentifier entityIdent = MetadataObjectUtil.toEntityIdent(metalake,
metadataObject);
+ NameIdentifier entityIdent = checkAndNormalizeMetadataObject(metalake,
metadataObject);
Entity.EntityType entityType =
MetadataObjectUtil.toEntityType(metadataObject);
NameIdentifier tagIdent = NameIdentifierUtil.ofTag(metalake, name);
- MetadataObjectUtil.checkMetadataObject(metalake, metadataObject);
-
try {
checkMetalake(NameIdentifier.of(metalake), entityStore);
return entityStore
@@ -421,11 +420,9 @@ public class TagManager implements TagDispatcher {
validateTagValuesToAdd(tagsToAdd);
validateTagValuesToRemove(tagsToRemove);
- NameIdentifier entityIdent = MetadataObjectUtil.toEntityIdent(metalake,
metadataObject);
+ NameIdentifier entityIdent = checkAndNormalizeMetadataObject(metalake,
metadataObject);
Entity.EntityType entityType =
MetadataObjectUtil.toEntityType(metadataObject);
- MetadataObjectUtil.checkMetadataObject(metalake, metadataObject);
-
Set<TagValue> tagsToAddSet =
tagsToAdd == null ? new LinkedHashSet<>() : new
LinkedHashSet<>(Arrays.asList(tagsToAdd));
Set<TagValue> tagsToRemoveSet =
@@ -482,6 +479,24 @@ public class TagManager implements TagDispatcher {
}
}
+ private NameIdentifier checkAndNormalizeMetadataObject(
+ String metalake, MetadataObject metadataObject) {
+ NameIdentifier ident = MetadataObjectUtil.toEntityIdent(metalake,
metadataObject);
+ // Existence checks use normalizing dispatchers and must see the original
request. Applying
+ // normalization first would fold quoted names again after their quotes
have been removed.
+ MetadataObjectUtil.checkMetadataObject(metalake, metadataObject);
+ if
(!SUPPORTED_METADATA_OBJECT_TYPES_FOR_TAGS.contains(metadataObject.type())
+ || metadataObject.type() == MetadataObject.Type.CATALOG) {
+ return ident;
+ }
+
+ return CapabilityHelpers.normalizeMetadataIdentifier(
+ ident,
+ Capability.Scope.valueOf(metadataObject.type().name()),
+ GravitinoEnv.getInstance().catalogManager(),
+ () -> entityStore);
+ }
+
private static String[] allowedValuesForStorage(TagValueConstraint
valueConstraint) {
TagValueConstraint normalizedConstraint =
valueConstraint == null ? TagValueConstraint.anyValue() :
valueConstraint;
diff --git
a/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
b/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
index ec5db8b006..6cff282079 100644
--- a/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
+++ b/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
@@ -18,13 +18,20 @@
*/
package org.apache.gravitino.catalog;
+import java.io.IOException;
+import java.util.Collections;
import java.util.Locale;
+import org.apache.gravitino.Entity;
+import org.apache.gravitino.EntityStore;
import org.apache.gravitino.NameIdentifier;
import org.apache.gravitino.Namespace;
import org.apache.gravitino.connector.BaseCatalog;
import org.apache.gravitino.connector.capability.Capability;
import org.apache.gravitino.connector.capability.CapabilityResult;
import org.apache.gravitino.exceptions.NoSuchCatalogException;
+import org.apache.gravitino.exceptions.NoSuchEntityException;
+import org.apache.gravitino.meta.ColumnEntity;
+import org.apache.gravitino.meta.TableEntity;
import org.apache.gravitino.rel.TableChange;
import org.apache.gravitino.rel.expressions.literals.Literal;
import org.apache.gravitino.rel.expressions.literals.Literals;
@@ -265,4 +272,84 @@ public class TestCapabilityHelpers {
Assertions.assertTrue(
Capability.DEFAULT.managedStorage(Capability.Scope.SEMANTIC_MODEL).supported());
}
+
+ @Test
+ void testNormalizeMetadataIdentifierKeepsExactStoredColumnName() throws
IOException {
+ CatalogManager catalogManager = foldingCatalogManager();
+ EntityStore store = Mockito.mock(EntityStore.class);
+ NameIdentifier table = NameIdentifier.of("metalake", "catalog", "schema",
"orders");
+ TableEntity tableEntity = Mockito.mock(TableEntity.class);
+ ColumnEntity column = Mockito.mock(ColumnEntity.class);
+ Mockito.when(column.name()).thenReturn("MyCol");
+
Mockito.when(tableEntity.columns()).thenReturn(Collections.singletonList(column));
+ Mockito.when(store.get(table, Entity.EntityType.TABLE, TableEntity.class))
+ .thenReturn(tableEntity);
+
+ Assertions.assertEquals(
+ NameIdentifier.of("metalake", "catalog", "schema", "orders", "MyCol"),
+ CapabilityHelpers.normalizeMetadataIdentifier(
+ NameIdentifier.of("metalake", "catalog", "SCHEMA", "ORDERS",
"MyCol"),
+ Capability.Scope.COLUMN,
+ catalogManager,
+ () -> store));
+ Assertions.assertEquals(
+ NameIdentifier.of("metalake", "catalog", "schema", "orders",
"othercol"),
+ CapabilityHelpers.normalizeMetadataIdentifier(
+ NameIdentifier.of("metalake", "catalog", "SCHEMA", "ORDERS",
"OtherCol"),
+ Capability.Scope.COLUMN,
+ catalogManager,
+ () -> store));
+ Mockito.verify(store, Mockito.times(2)).get(table,
Entity.EntityType.TABLE, TableEntity.class);
+
+ // A name that folding does not change needs no stored-column read.
+ Assertions.assertEquals(
+ NameIdentifier.of("metalake", "catalog", "schema", "orders", "mycol"),
+ CapabilityHelpers.normalizeMetadataIdentifier(
+ NameIdentifier.of("metalake", "catalog", "SCHEMA", "ORDERS",
"mycol"),
+ Capability.Scope.COLUMN,
+ catalogManager,
+ () -> store));
+ Mockito.verify(store, Mockito.times(2)).get(table,
Entity.EntityType.TABLE, TableEntity.class);
+ }
+
+ @Test
+ void testNormalizeMetadataIdentifierHandlesMissingOrFailedStoredTable()
throws IOException {
+ CatalogManager catalogManager = foldingCatalogManager();
+ EntityStore store = Mockito.mock(EntityStore.class);
+ NameIdentifier table = NameIdentifier.of("metalake", "catalog", "schema",
"orders");
+ Mockito.when(store.get(table, Entity.EntityType.TABLE, TableEntity.class))
+ .thenThrow(new NoSuchEntityException("missing"));
+ NameIdentifier column = NameIdentifier.of("metalake", "catalog", "schema",
"ORDERS", "MyCol");
+ Assertions.assertEquals(
+ NameIdentifier.of("metalake", "catalog", "schema", "orders", "mycol"),
+ CapabilityHelpers.normalizeMetadataIdentifier(
+ column, Capability.Scope.COLUMN, catalogManager, () -> store));
+
+ IOException failure = new IOException("store down");
+ Mockito.reset(store);
+ Mockito.when(store.get(table, Entity.EntityType.TABLE,
TableEntity.class)).thenThrow(failure);
+ RuntimeException error =
+ Assertions.assertThrows(
+ RuntimeException.class,
+ () ->
+ CapabilityHelpers.normalizeMetadataIdentifier(
+ column, Capability.Scope.COLUMN, catalogManager, () ->
store));
+ Assertions.assertSame(failure, error.getCause());
+ Assertions.assertTrue(error.getMessage().contains(table.toString()));
+ }
+
+ private static CatalogManager foldingCatalogManager() {
+ CatalogManager catalogManager = Mockito.mock(CatalogManager.class);
+ BaseCatalog<?> catalog = Mockito.mock(BaseCatalog.class);
+ Mockito.when(catalog.capability())
+ .thenReturn(
+ new Capability() {
+ @Override
+ public CapabilityResult caseSensitiveOnName(Scope scope) {
+ return CapabilityResult.unsupported("case insensitive");
+ }
+ });
+ CatalogTestUtils.mockDoWithCatalog(catalogManager, catalog);
+ return catalogManager;
+ }
}
diff --git
a/core/src/test/java/org/apache/gravitino/policy/TestPolicyManager.java
b/core/src/test/java/org/apache/gravitino/policy/TestPolicyManager.java
index 45cbaf2b9c..0c8ed6d87d 100644
--- a/core/src/test/java/org/apache/gravitino/policy/TestPolicyManager.java
+++ b/core/src/test/java/org/apache/gravitino/policy/TestPolicyManager.java
@@ -73,11 +73,15 @@ import org.apache.gravitino.MetadataObject;
import org.apache.gravitino.NameIdentifier;
import org.apache.gravitino.Namespace;
import org.apache.gravitino.catalog.CatalogDispatcher;
+import org.apache.gravitino.catalog.CatalogManager;
+import org.apache.gravitino.catalog.CatalogTestUtils;
import org.apache.gravitino.catalog.FunctionDispatcher;
import org.apache.gravitino.catalog.SchemaDispatcher;
import org.apache.gravitino.catalog.TableDispatcher;
import org.apache.gravitino.catalog.TreeLockTestSupport;
import org.apache.gravitino.catalog.ViewDispatcher;
+import org.apache.gravitino.connector.BaseCatalog;
+import org.apache.gravitino.connector.capability.Capability;
import org.apache.gravitino.exceptions.NoSuchEntityException;
import org.apache.gravitino.exceptions.NoSuchMetalakeException;
import org.apache.gravitino.exceptions.NoSuchPolicyException;
@@ -139,6 +143,7 @@ public class TestPolicyManager {
private static EntityStore entityStore;
private static PolicyManager policyManager;
private static TagManager tagManager;
+ private static CatalogManager previousCatalogManager;
@BeforeAll
public static void setUp() throws IllegalAccessException, IOException {
@@ -148,6 +153,13 @@ public class TestPolicyManager {
entityStore.initialize(config);
policyManager = new PolicyManager(idGenerator, entityStore);
tagManager = new TagManager(idGenerator, entityStore);
+ previousCatalogManager =
+ (CatalogManager) FieldUtils.readField(GravitinoEnv.getInstance(),
"catalogManager", true);
+ CatalogManager catalogManager = mock(CatalogManager.class);
+ BaseCatalog<?> backendCatalog = mock(BaseCatalog.class);
+ CatalogTestUtils.mockDoWithCatalog(catalogManager, backendCatalog);
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ FieldUtils.writeField(GravitinoEnv.getInstance(), "catalogManager",
catalogManager, true);
FieldUtils.writeField(GravitinoEnv.getInstance(), "lockManager", new
LockManager(config), true);
FieldUtils.writeField(
@@ -289,7 +301,9 @@ public class TestPolicyManager {
}
@AfterAll
- public static void tearDown() throws IOException {
+ public static void tearDown() throws IOException, IllegalAccessException {
+ FieldUtils.writeField(
+ GravitinoEnv.getInstance(), "catalogManager", previousCatalogManager,
true);
if (entityStore != null) {
entityStore.close();
entityStore = null;
diff --git a/core/src/test/java/org/apache/gravitino/tag/TestTagManager.java
b/core/src/test/java/org/apache/gravitino/tag/TestTagManager.java
index a57eb35e2e..e32d096ca1 100644
--- a/core/src/test/java/org/apache/gravitino/tag/TestTagManager.java
+++ b/core/src/test/java/org/apache/gravitino/tag/TestTagManager.java
@@ -54,6 +54,7 @@ import java.io.IOException;
import java.time.Instant;
import java.util.Arrays;
import java.util.List;
+import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
@@ -75,12 +76,23 @@ import org.apache.gravitino.NameIdentifier;
import org.apache.gravitino.Namespace;
import org.apache.gravitino.RelationalEntity;
import org.apache.gravitino.catalog.CatalogDispatcher;
+import org.apache.gravitino.catalog.CatalogManager;
+import org.apache.gravitino.catalog.CatalogTestUtils;
+import org.apache.gravitino.catalog.FilesetDispatcher;
import org.apache.gravitino.catalog.FunctionDispatcher;
+import org.apache.gravitino.catalog.ModelDispatcher;
import org.apache.gravitino.catalog.SchemaDispatcher;
import org.apache.gravitino.catalog.SemanticModelDispatcher;
import org.apache.gravitino.catalog.TableDispatcher;
+import org.apache.gravitino.catalog.TopicDispatcher;
import org.apache.gravitino.catalog.TreeLockTestSupport;
import org.apache.gravitino.catalog.ViewDispatcher;
+import org.apache.gravitino.connector.BaseCatalog;
+import org.apache.gravitino.connector.CatalogOperations;
+import org.apache.gravitino.connector.SupportsTableNameResolution;
+import org.apache.gravitino.connector.capability.Capability;
+import org.apache.gravitino.connector.capability.CapabilityResult;
+import org.apache.gravitino.exceptions.IllegalNamespaceException;
import org.apache.gravitino.exceptions.NoSuchMetadataObjectException;
import org.apache.gravitino.exceptions.NoSuchMetalakeException;
import org.apache.gravitino.exceptions.NoSuchTagException;
@@ -88,6 +100,7 @@ import org.apache.gravitino.exceptions.NotFoundException;
import org.apache.gravitino.exceptions.PolicyAlreadyAssociatedException;
import org.apache.gravitino.exceptions.TagAlreadyAssociatedException;
import org.apache.gravitino.exceptions.TagAlreadyExistsException;
+import org.apache.gravitino.file.Fileset;
import org.apache.gravitino.function.FunctionDefinition;
import org.apache.gravitino.function.FunctionDefinitions;
import org.apache.gravitino.function.FunctionImpl;
@@ -102,11 +115,15 @@ import org.apache.gravitino.meta.AuditInfo;
import org.apache.gravitino.meta.BaseMetalake;
import org.apache.gravitino.meta.CatalogEntity;
import org.apache.gravitino.meta.ColumnEntity;
+import org.apache.gravitino.meta.FilesetEntity;
import org.apache.gravitino.meta.FunctionEntity;
+import org.apache.gravitino.meta.ModelEntity;
import org.apache.gravitino.meta.PolicyEntity;
import org.apache.gravitino.meta.SchemaEntity;
import org.apache.gravitino.meta.SchemaVersion;
+import org.apache.gravitino.meta.SemanticModelEntity;
import org.apache.gravitino.meta.TableEntity;
+import org.apache.gravitino.meta.TopicEntity;
import org.apache.gravitino.meta.ViewEntity;
import org.apache.gravitino.metalake.MetalakeDispatcher;
import org.apache.gravitino.policy.AllValuesSelector;
@@ -117,6 +134,8 @@ import org.apache.gravitino.rel.Column;
import org.apache.gravitino.rel.Representation;
import org.apache.gravitino.rel.SQLRepresentation;
import org.apache.gravitino.rel.types.Types;
+import org.apache.gravitino.semantic.Dataset;
+import org.apache.gravitino.semantic.SemanticModelDefinition;
import org.apache.gravitino.storage.IdGenerator;
import org.apache.gravitino.storage.RandomIdGenerator;
import org.apache.gravitino.utils.NameIdentifierUtil;
@@ -126,6 +145,8 @@ import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.EnumSource;
import org.mockito.Mockito;
public class TestTagManager {
@@ -167,6 +188,18 @@ public class TestTagManager {
private static TagManager tagManager;
+ private static final CatalogManager catalogManager =
mock(CatalogManager.class);
+ private static final BaseCatalog<?> backendCatalog = mock(BaseCatalog.class);
+ private static CatalogManager previousCatalogManager;
+
+ private static final Capability CASE_INSENSITIVE_CAPABILITY =
+ new Capability() {
+ @Override
+ public CapabilityResult caseSensitiveOnName(Scope scope) {
+ return CapabilityResult.unsupported("case insensitive");
+ }
+ };
+
@BeforeAll
public static void setUp() throws IOException, IllegalAccessException {
idGenerator = new RandomIdGenerator();
@@ -301,6 +334,11 @@ public class TestTagManager {
entityStore.put(function, false /* overwritten */);
tagManager = new TagManager(idGenerator, entityStore);
+ previousCatalogManager =
+ (CatalogManager) FieldUtils.readField(GravitinoEnv.getInstance(),
"catalogManager", true);
+ FieldUtils.writeField(GravitinoEnv.getInstance(), "catalogManager",
catalogManager, true);
+ CatalogTestUtils.mockDoWithCatalog(catalogManager, backendCatalog);
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
FieldUtils.writeField(
GravitinoEnv.getInstance(), "internalMetalakeDispatcher",
metalakeDispatcher, true);
@@ -329,7 +367,9 @@ public class TestTagManager {
}
@AfterAll
- public static void tearDown() throws IOException {
+ public static void tearDown() throws IOException, IllegalAccessException {
+ FieldUtils.writeField(
+ GravitinoEnv.getInstance(), "catalogManager", previousCatalogManager,
true);
if (entityStore != null) {
entityStore.close();
entityStore = null;
@@ -381,6 +421,382 @@ public class TestTagManager {
Arrays.stream(tagManager.listTags(METALAKE)).forEach(n ->
tagManager.deleteTag(METALAKE, n));
}
+ @ParameterizedTest
+ @EnumSource(
+ value = Entity.EntityType.class,
+ names = {"SCHEMA", "TABLE", "COLUMN", "VIEW", "FUNCTION"})
+ void testTagOperationsNormalizeCaseInsensitiveNames(Entity.EntityType type) {
+ MetadataObject canonical = tagTarget(type, false);
+ MetadataObject request = tagTarget(type, true);
+ when(backendCatalog.capability()).thenReturn(CASE_INSENSITIVE_CAPABILITY);
+ try {
+ assertTagRoundTrip(request, canonical, canonical);
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ }
+ }
+
+ @ParameterizedTest
+ @EnumSource(
+ value = Entity.EntityType.class,
+ names = {"FILESET", "TOPIC", "MODEL"})
+ void testTagOperationsNormalizeOtherCatalogObjects(Entity.EntityType type)
+ throws IOException, IllegalAccessException {
+ NameIdentifier ident = NameIdentifier.of(METALAKE, CATALOG, SCHEMA,
"normalization_object");
+ AuditInfo audit =
AuditInfo.builder().withCreator("test").withCreateTime(Instant.now()).build();
+ Object dispatcher;
+ String dispatcherField;
+ switch (type) {
+ case FILESET:
+ entityStore.put(
+ FilesetEntity.builder()
+ .withId(idGenerator.nextId())
+ .withName(ident.name())
+ .withNamespace(ident.namespace())
+ .withFilesetType(Fileset.Type.EXTERNAL)
+ .withStorageLocations(Map.of("default",
"file:/tmp/normalization_object"))
+ .withAuditInfo(audit)
+ .build(),
+ false);
+ FilesetDispatcher filesetDispatcher = mock(FilesetDispatcher.class);
+ when(filesetDispatcher.filesetExists(any())).thenReturn(true);
+ dispatcher = filesetDispatcher;
+ dispatcherField = "internalFilesetDispatcher";
+ break;
+ case TOPIC:
+ entityStore.put(
+ TopicEntity.builder()
+ .withId(idGenerator.nextId())
+ .withName(ident.name())
+ .withNamespace(ident.namespace())
+ .withAuditInfo(audit)
+ .build(),
+ false);
+ TopicDispatcher topicDispatcher = mock(TopicDispatcher.class);
+ when(topicDispatcher.topicExists(any())).thenReturn(true);
+ dispatcher = topicDispatcher;
+ dispatcherField = "internalTopicDispatcher";
+ break;
+ case MODEL:
+ entityStore.put(
+ ModelEntity.builder()
+ .withId(idGenerator.nextId())
+ .withName(ident.name())
+ .withNamespace(ident.namespace())
+ .withLatestVersion(0)
+ .withAuditInfo(audit)
+ .build(),
+ false);
+ ModelDispatcher modelDispatcher = mock(ModelDispatcher.class);
+ when(modelDispatcher.modelExists(any())).thenReturn(true);
+ dispatcher = modelDispatcher;
+ dispatcherField = "internalModelDispatcher";
+ break;
+ default:
+ throw new IllegalArgumentException("Unsupported test target: " + type);
+ }
+ Object previousDispatcher =
+ FieldUtils.readField(GravitinoEnv.getInstance(), dispatcherField,
true);
+ FieldUtils.writeField(GravitinoEnv.getInstance(), dispatcherField,
dispatcher, true);
+ when(backendCatalog.capability()).thenReturn(CASE_INSENSITIVE_CAPABILITY);
+ try {
+ MetadataObject request =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(
+ METALAKE,
+ CATALOG,
+ SCHEMA.toUpperCase(Locale.ROOT),
+ ident.name().toUpperCase(Locale.ROOT)),
+ type);
+ MetadataObject canonical = NameIdentifierUtil.toMetadataObject(ident,
type);
+ assertTagRoundTrip(request, canonical, canonical);
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ FieldUtils.writeField(GravitinoEnv.getInstance(), dispatcherField,
previousDispatcher, true);
+ entityStore.delete(ident, type);
+ }
+ }
+
+ @Test
+ void testCatalogAndMetalakeTagOperationsDoNotLoadCapabilities() {
+ Mockito.clearInvocations(catalogManager);
+ MetadataObject catalogObject =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(METALAKE, CATALOG), Entity.EntityType.CATALOG);
+ assertTagRoundTrip(catalogObject, catalogObject, catalogObject);
+ MetadataObject metalakeObject =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(METALAKE), Entity.EntityType.METALAKE);
+ // Metalake tags are unsupported: preserve the existing store failure
without loading a catalog.
+ Assertions.assertThrows(
+ IllegalNamespaceException.class,
+ () -> tagManager.listTagsForMetadataObject(METALAKE, metalakeObject));
+ Assertions.assertThrows(
+ IllegalArgumentException.class,
+ () ->
+ tagManager.associateTagsForMetadataObject(
+ METALAKE, metalakeObject, new String[] {"normalization_tag"},
null));
+ Mockito.verifyNoInteractions(catalogManager);
+ }
+
+ @Test
+ void testTagOperationsPreserveCaseSensitiveNames() {
+ MetadataObject canonical = tagTarget(Entity.EntityType.TABLE, false);
+ MetadataObject differentlyCased = tagTarget(Entity.EntityType.TABLE, true);
+ Tag tag = tagManager.createTag(METALAKE, "case_sensitive_tag", null, null);
+ tagManager.associateTagsForMetadataObject(METALAKE, canonical, new
String[] {tag.name()}, null);
+
+ Assertions.assertThrows(
+ NoSuchMetadataObjectException.class,
+ () -> tagManager.listTagsForMetadataObject(METALAKE,
differentlyCased));
+ Assertions.assertThrows(
+ NoSuchMetadataObjectException.class,
+ () -> tagManager.getTagForMetadataObject(METALAKE, differentlyCased,
tag.name()));
+ Assertions.assertThrows(
+ NoSuchMetadataObjectException.class,
+ () ->
+ tagManager.associateTagsForMetadataObject(
+ METALAKE, differentlyCased, null, new String[] {tag.name()}));
+ Assertions.assertArrayEquals(
+ new String[] {tag.name()},
tagManager.listTagsForMetadataObject(METALAKE, canonical));
+ }
+
+ @Test
+ void testTagOperationsPreserveQuotedNamesAndOriginalExistenceCheck() {
+ Capability quoteAware =
+ new Capability() {
+ @Override
+ public String normalizeName(Scope scope, String name) {
+ return name.startsWith("\"") && name.endsWith("\"")
+ ? name.substring(1, name.length() - 1)
+ : name.toUpperCase(Locale.ROOT);
+ }
+ };
+ NameIdentifier quotedTable =
+ NameIdentifier.of(METALAKE, CATALOG, "\"" + SCHEMA + "\"", "\"" +
TABLE + "\"");
+ MetadataObject request =
+ NameIdentifierUtil.toMetadataObject(quotedTable,
Entity.EntityType.TABLE);
+ when(backendCatalog.capability()).thenReturn(quoteAware);
+ when(tableDispatcher.tableExists(any()))
+ .thenAnswer(invocation ->
quotedTable.equals(invocation.getArgument(0)));
+ try {
+ assertTagRoundTrip(request, request, tagTarget(Entity.EntityType.TABLE,
false));
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ when(tableDispatcher.tableExists(any())).thenReturn(true);
+ }
+ }
+
+ @ParameterizedTest
+ @EnumSource(
+ value = Entity.EntityType.class,
+ names = {"TABLE", "COLUMN"})
+ void testTagOperationsResolvePhysicalTableName(Entity.EntityType type) {
+ CatalogOperations ops =
+ mock(
+ CatalogOperations.class,
+
Mockito.withSettings().extraInterfaces(SupportsTableNameResolution.class));
+ Mockito.doReturn(ops).when(backendCatalog).ops();
+ when(((SupportsTableNameResolution) ops).resolveTableName(any()))
+ .thenAnswer(
+ invocation -> {
+ NameIdentifier ident = invocation.getArgument(0);
+ return ident.name().equals("table_alias")
+ ? NameIdentifier.of(ident.namespace(), TABLE)
+ : ident;
+ });
+ when(backendCatalog.capability()).thenReturn(CASE_INSENSITIVE_CAPABILITY);
+ NameIdentifier alias =
+ type == Entity.EntityType.TABLE
+ ? NameIdentifier.of(METALAKE, CATALOG,
SCHEMA.toUpperCase(Locale.ROOT), "TABLE_ALIAS")
+ : NameIdentifier.of(
+ METALAKE,
+ CATALOG,
+ SCHEMA.toUpperCase(Locale.ROOT),
+ "TABLE_ALIAS",
+ COLUMN.toUpperCase(Locale.ROOT));
+ MetadataObject canonical = tagTarget(type, false);
+ try {
+ assertTagRoundTrip(NameIdentifierUtil.toMetadataObject(alias, type),
canonical, canonical);
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ Mockito.doReturn(null).when(backendCatalog).ops();
+ }
+ }
+
+ @Test
+ void testTagOperationsFollowGlueStyleColumnCase() throws IOException {
+ // Glue folds schema and table names but keeps the case of column names.
+ NameIdentifier table = putTableWithColumn("glue_table", "MixedCol");
+ Capability glueStyle =
+ new Capability() {
+ @Override
+ public CapabilityResult caseSensitiveOnName(Scope scope) {
+ return scope == Scope.SCHEMA || scope == Scope.TABLE
+ ? CapabilityResult.unsupported("case insensitive")
+ : CapabilityResult.SUPPORTED;
+ }
+ };
+ when(backendCatalog.capability()).thenReturn(glueStyle);
+ try {
+ MetadataObject request =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(
+ METALAKE, CATALOG, SCHEMA.toUpperCase(Locale.ROOT),
"GLUE_TABLE", "MixedCol"),
+ Entity.EntityType.COLUMN);
+ MetadataObject canonical =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(METALAKE, CATALOG, SCHEMA, table.name(),
"MixedCol"),
+ Entity.EntityType.COLUMN);
+ assertTagRoundTrip(request, canonical, canonical);
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ entityStore.delete(table, Entity.EntityType.TABLE);
+ }
+ }
+
+ @Test
+ void testTagOperationsKeepExactStoredColumnName() throws IOException {
+ // A capability can fold column names that the source stored with another
case. The exact
+ // stored name must still find the column.
+ NameIdentifier table = putTableWithColumn("exact_table", "MyCol");
+ when(backendCatalog.capability()).thenReturn(CASE_INSENSITIVE_CAPABILITY);
+ try {
+ MetadataObject canonical =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(METALAKE, CATALOG, SCHEMA, table.name(),
"MyCol"),
+ Entity.EntityType.COLUMN);
+ MetadataObject request =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(
+ METALAKE, CATALOG, SCHEMA.toUpperCase(Locale.ROOT),
"EXACT_TABLE", "MyCol"),
+ Entity.EntityType.COLUMN);
+ assertTagRoundTrip(request, canonical, canonical);
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ entityStore.delete(table, Entity.EntityType.TABLE);
+ }
+ }
+
+ @ParameterizedTest
+ @EnumSource(
+ value = Entity.EntityType.class,
+ names = {"TABLE", "COLUMN"})
+ void testFailedTableNameResolutionLeavesTagsUnchanged(Entity.EntityType
type) {
+ MetadataObject canonical = tagTarget(type, false);
+ Tag kept = tagManager.createTag(METALAKE, "kept_tag", null, null);
+ Tag added = tagManager.createTag(METALAKE, "added_tag", null, null);
+ tagManager.associateTagsForMetadataObject(
+ METALAKE, canonical, new String[] {kept.name()}, null);
+
+ CatalogOperations ops =
+ mock(
+ CatalogOperations.class,
+
Mockito.withSettings().extraInterfaces(SupportsTableNameResolution.class));
+ when(((SupportsTableNameResolution) ops).resolveTableName(any()))
+ .thenThrow(new IllegalStateException("source unavailable"));
+ Mockito.doReturn(ops).when(backendCatalog).ops();
+ try {
+ RuntimeException addError =
+ Assertions.assertThrows(
+ RuntimeException.class,
+ () ->
+ tagManager.associateTagsForMetadataObject(
+ METALAKE, canonical, new String[] {added.name()}, null));
+ Assertions.assertInstanceOf(IllegalStateException.class,
addError.getCause());
+ Assertions.assertThrows(
+ RuntimeException.class,
+ () ->
+ tagManager.associateTagsForMetadataObject(
+ METALAKE, canonical, null, new String[] {kept.name()}));
+ Assertions.assertThrows(
+ RuntimeException.class,
+ () ->
+ tagManager.associateTagValuesForMetadataObject(
+ METALAKE, canonical, new TagValue[]
{TagValue.of(added.name(), "v")}, null));
+ } finally {
+ Mockito.doReturn(null).when(backendCatalog).ops();
+ }
+ Assertions.assertArrayEquals(
+ new String[] {kept.name()},
tagManager.listTagsForMetadataObject(METALAKE, canonical));
+ }
+
+ @Test
+ void testTagOperationsUseCustomNormalization() throws IOException {
+ NameIdentifier uppercase = NameIdentifier.of(METALAKE, CATALOG, SCHEMA,
"CUSTOM_TABLE");
+ AuditInfo audit =
AuditInfo.builder().withCreator("test").withCreateTime(Instant.now()).build();
+ entityStore.put(
+ TableEntity.builder()
+ .withId(idGenerator.nextId())
+ .withName(uppercase.name())
+ .withNamespace(uppercase.namespace())
+ .withColumns(Lists.newArrayList())
+ .withAuditInfo(audit)
+ .build(),
+ false);
+ Capability uppercaseTables =
+ new Capability() {
+ @Override
+ public String normalizeName(Scope scope, String name) {
+ return scope == Scope.TABLE ? name.toUpperCase(Locale.ROOT) : name;
+ }
+ };
+ when(backendCatalog.capability()).thenReturn(uppercaseTables);
+ try {
+ MetadataObject request =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(uppercase.namespace(), "custom_table"),
Entity.EntityType.TABLE);
+ MetadataObject canonical =
+ NameIdentifierUtil.toMetadataObject(uppercase,
Entity.EntityType.TABLE);
+ assertTagRoundTrip(request, canonical, canonical);
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+ entityStore.delete(uppercase, Entity.EntityType.TABLE);
+ }
+ }
+
+ @Test
+ void testTagOperationsPreserveSemanticModelName() throws IOException {
+ NameIdentifier ident = NameIdentifier.of(METALAKE, CATALOG, SCHEMA,
"SalesModel");
+ AuditInfo audit =
AuditInfo.builder().withCreator("test").withCreateTime(Instant.now()).build();
+ Dataset dataset =
+ Dataset.builder()
+ .withName("sales")
+ .withSource(NameIdentifier.of(CATALOG, SCHEMA, TABLE))
+ .build();
+ entityStore.put(
+ SemanticModelEntity.builder()
+ .withId(idGenerator.nextId())
+ .withName(ident.name())
+ .withNamespace(ident.namespace())
+ .withAuditInfo(audit)
+ .withDefinition(
+ SemanticModelDefinition.builder().withDatasets(new Dataset[]
{dataset}).build())
+ .build(),
+ false);
+ when(backendCatalog.capability()).thenReturn(CASE_INSENSITIVE_CAPABILITY);
+ when(semanticModelDispatcher.semanticModelExists(any())).thenReturn(true);
+ try {
+ MetadataObject request =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(METALAKE, CATALOG,
SCHEMA.toUpperCase(Locale.ROOT), ident.name()),
+ Entity.EntityType.SEMANTIC_MODEL);
+ MetadataObject canonical =
+ NameIdentifierUtil.toMetadataObject(ident,
Entity.EntityType.SEMANTIC_MODEL);
+ assertTagRoundTrip(request, canonical, canonical);
+ MetadataObject wrongCase =
+ NameIdentifierUtil.toMetadataObject(
+ NameIdentifier.of(ident.namespace(), "salesmodel"),
Entity.EntityType.SEMANTIC_MODEL);
+ Assertions.assertThrows(
+ NoSuchMetadataObjectException.class,
+ () -> tagManager.listTagsForMetadataObject(METALAKE, wrongCase));
+ } finally {
+ when(backendCatalog.capability()).thenReturn(Capability.DEFAULT);
+
when(semanticModelDispatcher.semanticModelExists(any())).thenReturn(false);
+ entityStore.delete(ident, Entity.EntityType.SEMANTIC_MODEL);
+ }
+ }
+
@Test
public void testCreateAndGetTag() {
Tag tag = tagManager.createTag(METALAKE, "tag1", null, null);
@@ -1431,4 +1847,91 @@ public class TestTagManager {
Assertions.assertTrue(tag.assignment().isPresent());
Assertions.assertArrayEquals(expectedValues,
tag.assignment().get().values());
}
+
+ private static NameIdentifier putTableWithColumn(String tableName, String
columnName)
+ throws IOException {
+ AuditInfo audit =
AuditInfo.builder().withCreator("test").withCreateTime(Instant.now()).build();
+ NameIdentifier ident = NameIdentifier.of(METALAKE, CATALOG, SCHEMA,
tableName);
+ entityStore.put(
+ TableEntity.builder()
+ .withId(idGenerator.nextId())
+ .withName(tableName)
+ .withNamespace(ident.namespace())
+ .withColumns(
+ Lists.newArrayList(
+ ColumnEntity.builder()
+ .withId(idGenerator.nextId())
+ .withName(columnName)
+ .withPosition(0)
+ .withDataType(Types.IntegerType.get())
+ .withNullable(true)
+ .withAutoIncrement(false)
+ .withAuditInfo(audit)
+ .build()))
+ .withAuditInfo(audit)
+ .build(),
+ false);
+ return ident;
+ }
+
+ private static MetadataObject tagTarget(Entity.EntityType type, boolean
upperCase) {
+ String schema = upperCase ? SCHEMA.toUpperCase(Locale.ROOT) : SCHEMA;
+ String table = upperCase ? TABLE.toUpperCase(Locale.ROOT) : TABLE;
+ NameIdentifier ident;
+ switch (type) {
+ case SCHEMA:
+ ident = NameIdentifier.of(METALAKE, CATALOG, schema);
+ break;
+ case TABLE:
+ ident = NameIdentifier.of(METALAKE, CATALOG, schema, table);
+ break;
+ case COLUMN:
+ String column = upperCase ? COLUMN.toUpperCase(Locale.ROOT) : COLUMN;
+ ident = NameIdentifier.of(METALAKE, CATALOG, schema, table, column);
+ break;
+ case VIEW:
+ case FUNCTION:
+ String name = type == Entity.EntityType.VIEW ? VIEW : FUNCTION;
+ name = upperCase ? name.toUpperCase(Locale.ROOT) : name;
+ ident = NameIdentifier.of(METALAKE, CATALOG, schema, name);
+ break;
+ default:
+ throw new IllegalArgumentException("Unsupported test target: " + type);
+ }
+ return NameIdentifierUtil.toMetadataObject(ident, type);
+ }
+
+ private static void assertTagRoundTrip(
+ MetadataObject request, MetadataObject alias, MetadataObject
storedObject) {
+ Tag tag = tagManager.createTag(METALAKE, "normalization_tag", null, null);
+ String[] names = {tag.name()};
+ tagManager.associateTagsForMetadataObject(METALAKE, alias, names, null);
+ Assertions.assertArrayEquals(names,
tagManager.listTagsForMetadataObject(METALAKE, request));
+ Assertions.assertEquals(
+ tag.name(), tagManager.listTagsInfoForMetadataObject(METALAKE,
request)[0].name());
+ Assertions.assertEquals(
+ tag.name(), tagManager.getTagForMetadataObject(METALAKE, request,
tag.name()).name());
+ Assertions.assertArrayEquals(
+ new MetadataObject[] {storedObject},
+ tagManager.listMetadataObjectsForTag(METALAKE, tag.name()));
+
+ Assertions.assertEquals(
+ 0, tagManager.associateTagsForMetadataObject(METALAKE, request, null,
names).length);
+ Assertions.assertArrayEquals(
+ names, tagManager.associateTagsForMetadataObject(METALAKE, request,
names, null));
+ tagManager.associateTagsForMetadataObject(METALAKE, alias, null, names);
+ TagValue[] values = {TagValue.of(tag.name(), "MixedCaseValue")};
+ Assertions.assertArrayEquals(
+ names, tagManager.associateTagValuesForMetadataObject(METALAKE,
request, values, null));
+ Assertions.assertArrayEquals(
+ new String[] {"MixedCaseValue"},
+ tagManager
+ .getTagForMetadataObject(METALAKE, alias, tag.name())
+ .assignment()
+ .get()
+ .values());
+ Assertions.assertEquals(
+ 0, tagManager.associateTagValuesForMetadataObject(METALAKE, request,
null, values).length);
+ Assertions.assertEquals(0,
tagManager.listTagsInfoForMetadataObject(METALAKE, alias).length);
+ }
}
diff --git
a/server-common/src/main/java/org/apache/gravitino/server/authorization/MetadataIdConverter.java
b/server-common/src/main/java/org/apache/gravitino/server/authorization/MetadataIdConverter.java
index 5a4f44bb59..7040a2ccfc 100644
---
a/server-common/src/main/java/org/apache/gravitino/server/authorization/MetadataIdConverter.java
+++
b/server-common/src/main/java/org/apache/gravitino/server/authorization/MetadataIdConverter.java
@@ -22,7 +22,6 @@ package org.apache.gravitino.server.authorization;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableMap;
-import com.google.common.collect.ObjectArrays;
import java.io.IOException;
import java.util.Map;
import java.util.Optional;
@@ -33,7 +32,6 @@ import org.apache.gravitino.GravitinoEnv;
import org.apache.gravitino.HasIdentifier;
import org.apache.gravitino.MetadataObject;
import org.apache.gravitino.NameIdentifier;
-import org.apache.gravitino.Namespace;
import org.apache.gravitino.catalog.CapabilityHelpers;
import org.apache.gravitino.catalog.CatalogManager;
import org.apache.gravitino.connector.capability.Capability;
@@ -106,8 +104,10 @@ public class MetadataIdConverter {
/**
* Normalizes a metadata object's name using the same catalog rules as ID
resolution.
*
- * <p>Types without a catalog capability scope retain their names. Semantic
model leaves remain
- * case sensitive, while column names and their table/schema parents follow
their own scopes.
+ * <p>Types without a catalog capability scope retain their names. The rules
come from {@link
+ * CapabilityHelpers#normalizeMetadataIdentifier}, which tag operations also
use: Semantic Model
+ * leaves keep their case, tables and column parents resolve to physical
names, and a column keeps
+ * a name that exactly matches a stored column.
*
* @param metadataObject the object whose name will be normalized
* @param metalake the metalake name
@@ -152,33 +152,9 @@ public class MetadataIdConverter {
if (scope == null) {
return ident;
}
-
- Capability capability = CapabilityHelpers.getCapability(ident,
catalogManager);
- if (scope == Capability.Scope.COLUMN) {
- // The NameIdentifier overload applies SCHEMA to the namespace and TABLE
to the leaf.
- NameIdentifier table =
- CapabilityHelpers.applyCaseSensitive(
- NameIdentifier.of(ident.namespace().levels()),
Capability.Scope.TABLE, capability);
- return NameIdentifier.of(
- Namespace.of(ObjectArrays.concat(table.namespace().levels(),
table.name())),
- CapabilityHelpers.applyCaseSensitiveOnName(scope, ident.name(),
capability));
- }
-
- if (scope == Capability.Scope.SEMANTIC_MODEL) {
- return NameIdentifier.of(
- CapabilityHelpers.applyCaseSensitive(ident.namespace(), scope,
capability), ident.name());
- }
-
- NameIdentifier normalized = CapabilityHelpers.applyCaseSensitive(ident,
scope, capability);
-
- // For a table, resolve the normalized name to the physically stored name
through the same
- // shared resolver the table dispatcher uses, so the identifier authorized
here is the
- // identifier the operation later runs on. For catalogs that do not
implement the resolution
- // capability (the vast majority) this is a pure identity call with no
extra source access.
- if (scope == Capability.Scope.TABLE) {
- return CapabilityHelpers.resolvePhysicalTableName(normalized,
catalogManager);
- }
- return normalized;
+ // Tag operations use the same helper, so authorization checks the object
they change.
+ return CapabilityHelpers.normalizeMetadataIdentifier(
+ ident, scope, catalogManager, () ->
GravitinoEnv.getInstance().entityStore());
}
private static Optional<Long> loadId(MetadataObject metadataObject,
NameIdentifier ident) {
diff --git
a/server-common/src/test/java/org/apache/gravitino/server/authorization/TestMetadataIdConverter.java
b/server-common/src/test/java/org/apache/gravitino/server/authorization/TestMetadataIdConverter.java
index c6225e1be1..fe7bfe4f87 100644
---
a/server-common/src/test/java/org/apache/gravitino/server/authorization/TestMetadataIdConverter.java
+++
b/server-common/src/test/java/org/apache/gravitino/server/authorization/TestMetadataIdConverter.java
@@ -32,6 +32,7 @@ import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import java.io.IOException;
import java.time.Instant;
+import java.util.Collections;
import java.util.Optional;
import org.apache.commons.lang3.reflect.FieldUtils;
import org.apache.gravitino.Catalog;
@@ -344,6 +345,91 @@ public class TestMetadataIdConverter {
Assertions.assertEquals(physical, authorized);
}
+ @Test
+ void testColumnAuthorizationResolvesParentPhysicalName() {
+ // Tag operations and authorization share one helper, so a column's parent
table resolves to
+ // the same physical name as a table does.
+ NameIdentifier physical = NameIdentifier.of("metalake", "catalog",
"schema", "t_phys");
+ CatalogOperations resolvingOps =
+ mock(
+ CatalogOperations.class,
+ withSettings().extraInterfaces(SupportsTableNameResolution.class));
+ when(((SupportsTableNameResolution) resolvingOps).resolveTableName(any()))
+ .thenAnswer(
+ invocation -> {
+ NameIdentifier norm = invocation.getArgument(0);
+ return "T_NORM".equals(norm.name()) ? physical : norm;
+ });
+ BaseCatalog<?> mockCatalog = mock(BaseCatalog.class);
+ when(mockCatalog.capability()).thenReturn(new Capability() {});
+ when(mockCatalog.ops()).thenReturn(resolvingOps);
+ CatalogManager mockCatalogManager = mock(CatalogManager.class);
+ doAnswer(
+ invocation -> {
+ ThrowableFunction<BaseCatalog<?>, Object> fn =
invocation.getArgument(1);
+ return fn.apply(mockCatalog);
+ })
+ .when(mockCatalogManager)
+ .doWithCatalog(any(), any());
+
+ Assertions.assertEquals(
+ NameIdentifier.of("metalake", "catalog", "schema", "t_phys", "col"),
+ MetadataIdConverter.normalizeCaseSensitive(
+ NameIdentifier.of("metalake", "catalog", "schema", "T_NORM",
"col"),
+ Capability.Scope.COLUMN,
+ mockCatalogManager));
+ }
+
+ @Test
+ void testColumnAuthorizationKeepsExactStoredColumnName() throws Exception {
+ CatalogManager catalogs = mock(CatalogManager.class);
+ BaseCatalog<?> catalog = mock(BaseCatalog.class);
+ when(catalog.capability())
+ .thenReturn(
+ new Capability() {
+ @Override
+ public CapabilityResult caseSensitiveOnName(Scope scope) {
+ return CapabilityResult.unsupported("lowercase");
+ }
+ });
+ Mockito.doAnswer(
+ invocation -> {
+ ThrowableFunction<BaseCatalog<?>, Object> operation =
invocation.getArgument(1);
+ return operation.apply(catalog);
+ })
+ .when(catalogs)
+ .doWithCatalog(any(), any());
+ EntityStore store = mock(EntityStore.class);
+ TableEntity table = mock(TableEntity.class);
+ ColumnEntity column = mock(ColumnEntity.class);
+ when(column.name()).thenReturn("MyCol");
+ when(table.columns()).thenReturn(Collections.singletonList(column));
+ when(store.get(
+ NameIdentifier.of("metalake", "catalog", "schema", "orders"),
+ Entity.EntityType.TABLE,
+ TableEntity.class))
+ .thenReturn(table);
+ GravitinoEnv env = mock(GravitinoEnv.class);
+ when(env.catalogManager()).thenReturn(catalogs);
+ when(env.entityStore()).thenReturn(store);
+ try (MockedStatic<GravitinoEnv> mocked = mockStatic(GravitinoEnv.class)) {
+ mocked.when(GravitinoEnv::getInstance).thenReturn(env);
+ Assertions.assertEquals(
+ "catalog.schema.orders.MyCol",
+ MetadataIdConverter.normalizeMetadataObject(
+ MetadataObjects.parse("catalog.SCHEMA.ORDERS.MyCol",
MetadataObject.Type.COLUMN),
+ "metalake")
+ .fullName());
+ Assertions.assertEquals(
+ "catalog.schema.orders.othercol",
+ MetadataIdConverter.normalizeMetadataObject(
+ MetadataObjects.parse(
+ "catalog.SCHEMA.ORDERS.OtherCol",
MetadataObject.Type.COLUMN),
+ "metalake")
+ .fullName());
+ }
+ }
+
private void initTestNameIdentifier() {
ident1 = NameIdentifier.of("metalake");
ident2 = NameIdentifier.of("metalake", "catalog");
