This is an automated email from the ASF dual-hosted git repository.
yuqi1129 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new d6c7a8a2e8 [#11701] fix(core): invalidate METADATA_OBJECT_ROLE_REL
cache on role grant/override/create (#11702)
d6c7a8a2e8 is described below
commit d6c7a8a2e8c6d7f9bb87d40e22c8ccf6e60e56c7
Author: Sun Yuhan <[email protected]>
AuthorDate: Wed Jun 17 21:59:08 2026 +0800
[#11701] fix(core): invalidate METADATA_OBJECT_ROLE_REL cache on role
grant/override/create (#11702)
### What changes were proposed in this pull request?
After writing a `RoleEntity`, `RelationalEntityStore.update()` and
`put()` now invalidate the object-keyed `METADATA_OBJECT_ROLE_REL` cache
entry for each of the role's securable objects (private helper
`invalidateMetadataObjectRoleRelationCache`; no-op for non-ROLE entities
or empty securable objects). Covers grant / override-add / createRole.
The invalidation uses a new
`SupportsRelationEntityCache.invalidateRelationEntry`, which drops only
the cached relation result and its index entry **without cascading
through the reverse index**. This matters because the reverse index is
shared across all roles bound to one object; a full `invalidate(ident,
type, relType)` would BFS through it and evict the other roles' mappings
(which would break `testInvalidRelationCache`).
### Why are the changes needed?
Fixes #11701: for a metadata object that was already queried,
`listBindingRoleNames()` returns stale data (missing the newly granted
role) after `grantPrivilegesToRole` (or `overridePrivilegesInRole`
adding an object, or `createRole` with objects), until the relation
cache TTL (default 1h) elapses.
Root cause: the write path only calls `cache.invalidate(roleIdent,
ROLE)` (a role-side BFS). Reaching the object-keyed
`METADATA_OBJECT_ROLE_REL` entry requires the reverse index to already
map `roleIdent -> objectKey`, which is only established when the role
has previously been cached as a binding role of that object (the #11297
patch). A newly granted role was never cached there, so the invalidation
never reaches the entry. The subtraction side (revoke / override-remove
/ deleteRole) is already covered by the same role-side invalidation via
the reverse index (the removed role was previously cached), so this PR
does not touch it.
Fix: #11701
### Does this PR introduce _any_ user-facing change?
No API or configuration change. Behavioral fix: "list binding roles by
object" reflects grant/override/create immediately instead of being
delayed until the cache entry expires.
### How was this patch tested?
Added 5 parameterized tests in `TestEntityStorageRelationCache`
(h2/mysql/postgresql × enableCache true/false):
- `testGrantPrivilegeInvalidatesMetadataObjectRoleRelCache` — reproduces
the bug (red before / green after);
-
`testCreateRoleWithSecurableObjectsInvalidatesMetadataObjectRoleRelCache`
— covers the createRole (`store.put`) path;
- `testRevokeAllPrivileges…` / `testOverrideRemoveObject…` /
`testDeleteRole…` — three subtraction guards (no stale residue; pin
coverage so a future reverse-index rule change cannot silently regress).
Ran: `./gradlew :core:test --tests
'*TestEntityStorageRelationCache.test{Invalid,Grant,Revoke,RevokeAll,OverrideRemove,Delete,Create}*'
-PskipDockerTests=false -PskipITs` → BUILD SUCCESSFUL, `tests=42
skipped=0 failures=0 errors=0` (mysql ~10s/case, postgresql ~3s, h2
~0.1–2s; all three backends actually executed). `:core:spotlessCheck`
passes.
---------
Co-authored-by: Sun Yuhan <[email protected]>
---
.../gravitino/cache/CaffeineEntityCache.java | 22 +
.../org/apache/gravitino/cache/NoOpsCache.java | 7 +
.../cache/SupportsRelationEntityCache.java | 20 +
.../storage/relational/RelationalEntityStore.java | 75 +++
.../storage/TestEntityStorageRelationCache.java | 730 +++++++++++++++++++++
5 files changed, 854 insertions(+)
diff --git
a/core/src/main/java/org/apache/gravitino/cache/CaffeineEntityCache.java
b/core/src/main/java/org/apache/gravitino/cache/CaffeineEntityCache.java
index 952bb1e193..fd785edf5a 100644
--- a/core/src/main/java/org/apache/gravitino/cache/CaffeineEntityCache.java
+++ b/core/src/main/java/org/apache/gravitino/cache/CaffeineEntityCache.java
@@ -195,6 +195,28 @@ public class CaffeineEntityCache extends BaseEntityCache {
});
}
+ /** {@inheritDoc} */
+ @Override
+ public boolean invalidateRelationEntry(
+ NameIdentifier ident, Entity.EntityType type,
SupportsRelationOperations.Type relType) {
+ checkArguments(ident, type, relType);
+ EntityCacheRelationKey key = EntityCacheRelationKey.of(ident, type,
relType);
+ return segmentedLock.withLock(
+ key,
+ () -> {
+ // Drop the cached relation result, its index entry, and the
reverse-index bookkeeping
+ // for this relation key only. Do NOT cascade through the reverse
index to other
+ // entities: the reverse index is shared (e.g. all roles bound to
one metadata object),
+ // and a BFS cascade would evict their mappings.
cacheData.invalidate is explicit so it
+ // bypasses the removal listener; reverseIndex.remove(key) then
cleans up only this
+ // entry's own bookkeeping (entityToReverseIndexMap + reverseIndex
references to it).
+ cacheData.invalidate(key);
+ reverseIndex.remove(key);
+ cacheIndex.remove(key.toString());
+ return true;
+ });
+ }
+
/** {@inheritDoc} */
@Override
public boolean invalidate(NameIdentifier ident, Entity.EntityType type) {
diff --git a/core/src/main/java/org/apache/gravitino/cache/NoOpsCache.java
b/core/src/main/java/org/apache/gravitino/cache/NoOpsCache.java
index a9c9939bc1..fad08c2c2b 100644
--- a/core/src/main/java/org/apache/gravitino/cache/NoOpsCache.java
+++ b/core/src/main/java/org/apache/gravitino/cache/NoOpsCache.java
@@ -125,6 +125,13 @@ public class NoOpsCache extends BaseEntityCache {
return false;
}
+ /** {@inheritDoc} */
+ @Override
+ public boolean invalidateRelationEntry(
+ NameIdentifier ident, Entity.EntityType type,
SupportsRelationOperations.Type relType) {
+ return false;
+ }
+
/** {@inheritDoc} */
@Override
public boolean contains(
diff --git
a/core/src/main/java/org/apache/gravitino/cache/SupportsRelationEntityCache.java
b/core/src/main/java/org/apache/gravitino/cache/SupportsRelationEntityCache.java
index 98619db006..51b12113b5 100644
---
a/core/src/main/java/org/apache/gravitino/cache/SupportsRelationEntityCache.java
+++
b/core/src/main/java/org/apache/gravitino/cache/SupportsRelationEntityCache.java
@@ -56,6 +56,26 @@ public interface SupportsRelationEntityCache {
boolean invalidate(
NameIdentifier ident, Entity.EntityType type,
SupportsRelationOperations.Type relType);
+ /**
+ * Invalidates the cached relation result for the given key and cleans up
the reverse-index
+ * bookkeeping for that entry, without cascading through the reverse index
to other entities.
+ *
+ * <p>Unlike {@link #invalidate(NameIdentifier, Entity.EntityType,
+ * SupportsRelationOperations.Type)}, this does NOT perform a BFS cascade:
it leaves other
+ * entities' caches and the reverse-index mappings of the entities
referenced by this relation
+ * (e.g. all roles bound to one metadata object) intact. It only drops this
relation result and
+ * the reverse-index references that point at this relation key, so the
entry is rebuilt on the
+ * next read. Use it when a relation result is known to be stale and a full
cascade would
+ * incorrectly evict other entities' state.
+ *
+ * @param ident the name identifier
+ * @param type the entity type
+ * @param relType the relation type
+ * @return true if the cache entry was removed
+ */
+ boolean invalidateRelationEntry(
+ NameIdentifier ident, Entity.EntityType type,
SupportsRelationOperations.Type relType);
+
/**
* Checks whether an entity with the given name identifier, type, and
relation type is present in
* the cache.
diff --git
a/core/src/main/java/org/apache/gravitino/storage/relational/RelationalEntityStore.java
b/core/src/main/java/org/apache/gravitino/storage/relational/RelationalEntityStore.java
index 030700eb7e..6f21491173 100644
---
a/core/src/main/java/org/apache/gravitino/storage/relational/RelationalEntityStore.java
+++
b/core/src/main/java/org/apache/gravitino/storage/relational/RelationalEntityStore.java
@@ -41,6 +41,7 @@ import org.apache.gravitino.NameIdentifier;
import org.apache.gravitino.Namespace;
import org.apache.gravitino.RelationalEntity;
import org.apache.gravitino.SupportsRelationOperations;
+import org.apache.gravitino.authorization.SecurableObject;
import org.apache.gravitino.cache.CacheFactory;
import org.apache.gravitino.cache.CachedEntityIdResolver;
import org.apache.gravitino.cache.EntityCache;
@@ -48,8 +49,13 @@ import org.apache.gravitino.cache.EntityCacheKey;
import org.apache.gravitino.cache.EntityCacheRelationKey;
import org.apache.gravitino.cache.NoOpsCache;
import org.apache.gravitino.exceptions.NoSuchEntityException;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.RoleEntity;
+import org.apache.gravitino.meta.UserEntity;
import org.apache.gravitino.storage.relational.service.EntityIdService;
import org.apache.gravitino.utils.Executable;
+import org.apache.gravitino.utils.MetadataObjectUtil;
+import org.apache.gravitino.utils.NamespaceUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -144,6 +150,7 @@ public class RelationalEntityStore
throws IOException, EntityAlreadyExistsException {
backend.insert(e, overwritten);
cache.put(e);
+ invalidateAggregatedRoleRelationCache(e);
}
@Override
@@ -152,6 +159,7 @@ public class RelationalEntityStore
throws IOException, NoSuchEntityException, EntityAlreadyExistsException {
E updatedEntity = backend.update(ident, entityType, updater);
cache.invalidate(ident, entityType);
+ invalidateAggregatedRoleRelationCache(updatedEntity);
return updatedEntity;
}
@@ -451,4 +459,71 @@ public class RelationalEntityStore
cache.put(sourceId, identType, relType, entityList);
}
}
+
+ /**
+ * Invalidates the relation cache entries keyed by the counterpart of a
role-aggregating entity
+ * after that entity is written, so that reverse lookups reflect the change
immediately.
+ *
+ * <p>Three entity types aggregate role relations and are mutated through
{@code store.update} /
+ * {@code store.put}, which only invalidate the entity itself:
+ *
+ * <ul>
+ * <li>{@link RoleEntity} via {@code securableObjects} -> {@code
METADATA_OBJECT_ROLE_REL},
+ * invalidated per metadata object (catalog/schema/table/...);
+ * <li>{@link UserEntity} via {@code roleNames} -> {@code ROLE_USER_REL},
invalidated per role;
+ * <li>{@link GroupEntity} via {@code roleNames} -> {@code
ROLE_GROUP_REL}, invalidated per
+ * role.
+ * </ul>
+ *
+ * <p>The role-side BFS invalidation ({@code invalidate(roleIdent, ROLE)})
only reaches a
+ * counterpart's relation entry when the entity had previously been cached
against it; a freshly
+ * granted binding was never cached there, so without this explicit
invalidation the stale
+ * relation result is served until the entry's TTL elapses. Each entry is
dropped via {@link
+ * EntityCache#invalidateRelationEntry} (no BFS cascade), preserving other
entities' mappings.
+ */
+ private void invalidateAggregatedRoleRelationCache(Entity entity) {
+ if (entity instanceof RoleEntity) {
+ RoleEntity roleEntity = (RoleEntity) entity;
+ List<SecurableObject> securableObjects = roleEntity.securableObjects();
+ if (securableObjects == null || securableObjects.isEmpty()) {
+ return;
+ }
+ String metalake = roleEntity.namespace().level(0);
+ for (SecurableObject securableObject : securableObjects) {
+ cache.invalidateRelationEntry(
+ MetadataObjectUtil.toEntityIdent(metalake, securableObject),
+ MetadataObjectUtil.toEntityType(securableObject.type()),
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL);
+ }
+ } else if (entity instanceof UserEntity) {
+ UserEntity userEntity = (UserEntity) entity;
+ invalidateRoleGranteeRelations(
+ userEntity.namespace().level(0),
+ userEntity.roleNames(),
+ SupportsRelationOperations.Type.ROLE_USER_REL);
+ } else if (entity instanceof GroupEntity) {
+ GroupEntity groupEntity = (GroupEntity) entity;
+ invalidateRoleGranteeRelations(
+ groupEntity.namespace().level(0),
+ groupEntity.roleNames(),
+ SupportsRelationOperations.Type.ROLE_GROUP_REL);
+ }
+ }
+
+ /**
+ * Invalidates the {@code ROLE_USER_REL} / {@code ROLE_GROUP_REL} cache
entries keyed by each role
+ * the grantee (user/group) is aggregated against.
+ */
+ private void invalidateRoleGranteeRelations(
+ String metalake, List<String> roleNames, SupportsRelationOperations.Type
relType) {
+ if (roleNames == null || roleNames.isEmpty()) {
+ return;
+ }
+ for (String roleName : roleNames) {
+ cache.invalidateRelationEntry(
+ NameIdentifier.of(NamespaceUtil.ofRole(metalake), roleName),
+ Entity.EntityType.ROLE,
+ relType);
+ }
+ }
}
diff --git
a/core/src/test/java/org/apache/gravitino/storage/TestEntityStorageRelationCache.java
b/core/src/test/java/org/apache/gravitino/storage/TestEntityStorageRelationCache.java
index 8819d5c028..5bbec6c5e3 100644
---
a/core/src/test/java/org/apache/gravitino/storage/TestEntityStorageRelationCache.java
+++
b/core/src/test/java/org/apache/gravitino/storage/TestEntityStorageRelationCache.java
@@ -58,6 +58,7 @@ import org.apache.gravitino.meta.CatalogEntity;
import org.apache.gravitino.meta.FilesetEntity;
import org.apache.gravitino.meta.FunctionEntity;
import org.apache.gravitino.meta.GenericEntity;
+import org.apache.gravitino.meta.GroupEntity;
import org.apache.gravitino.meta.PolicyEntity;
import org.apache.gravitino.meta.RoleEntity;
import org.apache.gravitino.meta.SchemaEntity;
@@ -1610,6 +1611,735 @@ public class TestEntityStorageRelationCache extends
AbstractEntityStorageTest {
}
}
+ /**
+ * Reproduces the stale-read defect where granting a role access to an
already-cached metadata
+ * object is not reflected by {@code
listEntitiesByRelation(METADATA_OBJECT_ROLE_REL)} until the
+ * cache entry's TTL elapses.
+ *
+ * <p>Flow: create schema + roleA (bound to schema) + roleB (no securable
object, never cached as
+ * a binding role) -> list binding roles (warms cache; the reverse index
maps roleA but not roleB)
+ * -> update roleB to bind the schema (simulates grantPrivilegesToRole) ->
list binding roles
+ * again -> must immediately contain roleB.
+ */
+ @ParameterizedTest
+ @MethodSource("storageProvider")
+ void testGrantPrivilegeInvalidatesMetadataObjectRoleRelCache(String type,
boolean enableCache)
+ throws Exception {
+ Config config = Mockito.mock(Config.class);
+ Mockito.when(config.get(Configs.CACHE_ENABLED)).thenReturn(enableCache);
+ init(type, config);
+
+ AuditInfo auditInfo =
+
AuditInfo.builder().withCreator("creator").withCreateTime(Instant.now()).build();
+
+ try (EntityStore store = EntityStoreFactory.createEntityStore(config)) {
+ try {
+ store.initialize(config);
+
+ BaseMetalake metalake =
+ createBaseMakeLake(RandomIdGenerator.INSTANCE.nextId(),
"metalake", auditInfo);
+ store.put(metalake, false);
+
+ CatalogEntity catalog =
+ createCatalog(
+ RandomIdGenerator.INSTANCE.nextId(),
+ NamespaceUtil.ofCatalog("metalake"),
+ "catalog",
+ auditInfo);
+ store.put(catalog, false);
+
+ SchemaEntity schema =
+ createSchemaEntity(
+ RandomIdGenerator.INSTANCE.nextId(),
+ Namespace.of("metalake", "catalog"),
+ "test_schema",
+ auditInfo);
+ store.put(schema, false);
+
+ SecurableObject catalogObject = SecurableObjects.ofCatalog("catalog",
Lists.newArrayList());
+ SecurableObject schemaObject =
+ SecurableObjects.ofSchema(
+ catalogObject, "test_schema",
Lists.newArrayList(Privileges.UseSchema.allow()));
+
+ // roleA is bound to the schema; roleB has no securable object, so it
was never cached as a
+ // binding role of any metadata object (its identifier is absent from
the reverse index).
+ RoleEntity roleA =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("roleA")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList(schemaObject))
+ .build();
+ store.put(roleA, false);
+
+ RoleEntity roleB =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("roleB")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList())
+ .build();
+ store.put(roleB, false);
+
+ SupportsRelationOperations relationOperations =
(SupportsRelationOperations) store;
+
+ // Warm the METADATA_OBJECT_ROLE_REL cache: schema -> [roleA]. The
reverse index now holds
+ // roleA -> schemaKey but NOT roleB, because roleB has never been
served as a binding role.
+ List<RoleEntity> rolesBeforeGrant =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertEquals(1, rolesBeforeGrant.size());
+ Assertions.assertEquals("roleA", rolesBeforeGrant.get(0).name());
+
+ // Simulate grantPrivilegesToRole: bind roleB to the schema. roleB was
never cached, so the
+ // role-side invalidation cannot reach the schema's relation cache
entry through the reverse
+ // index; RelationalEntityStore must explicitly invalidate it to avoid
a stale read.
+ store.update(
+ roleB.nameIdentifier(),
+ RoleEntity.class,
+ Entity.EntityType.ROLE,
+ existing ->
+ RoleEntity.builder()
+ .withId(existing.id())
+ .withName(existing.name())
+ .withNamespace(existing.namespace())
+ .withProperties(existing.properties())
+ .withAuditInfo(existing.auditInfo())
+ .withSecurableObjects(Lists.newArrayList(schemaObject))
+ .build());
+
+ // listBindingRoleNames(schema) must immediately reflect roleB after
the grant.
+ List<RoleEntity> rolesAfterGrant =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ List<String> roleNames =
+
rolesAfterGrant.stream().map(RoleEntity::name).sorted().collect(Collectors.toList());
+ Assertions.assertEquals(
+ Lists.newArrayList("roleA", "roleB"),
+ roleNames,
+ "grant must be immediately visible via listBindingRoleNames");
+
+ } finally {
+ destroy(type);
+ }
+ }
+ }
+
+ /**
+ * Guards the subtraction side of {@link
+ * #testGrantPrivilegeInvalidatesMetadataObjectRoleRelCache}: after fully
revoking a role's
+ * privilege on an already-warmed metadata object, listBindingRoleNames must
immediately drop the
+ * role. The removed object is no longer in the updated entity, so this
relies on the role-side
+ * {@code invalidate(roleIdent, ROLE)} propagating to the object's relation
cache via the reverse
+ * index; the test pins that behavior so a future change to the
reverse-index rule cannot silently
+ * regress revoke.
+ */
+ @ParameterizedTest
+ @MethodSource("storageProvider")
+ void testRevokeAllPrivilegesInvalidatesMetadataObjectRoleRelCache(
+ String type, boolean enableCache) throws Exception {
+ Config config = Mockito.mock(Config.class);
+ Mockito.when(config.get(Configs.CACHE_ENABLED)).thenReturn(enableCache);
+ init(type, config);
+
+ AuditInfo auditInfo =
+
AuditInfo.builder().withCreator("creator").withCreateTime(Instant.now()).build();
+
+ try (EntityStore store = EntityStoreFactory.createEntityStore(config)) {
+ try {
+ store.initialize(config);
+
+ BaseMetalake metalake =
+ createBaseMakeLake(RandomIdGenerator.INSTANCE.nextId(),
"metalake", auditInfo);
+ store.put(metalake, false);
+
+ CatalogEntity catalog =
+ createCatalog(
+ RandomIdGenerator.INSTANCE.nextId(),
+ NamespaceUtil.ofCatalog("metalake"),
+ "catalog",
+ auditInfo);
+ store.put(catalog, false);
+
+ SchemaEntity schema =
+ createSchemaEntity(
+ RandomIdGenerator.INSTANCE.nextId(),
+ Namespace.of("metalake", "catalog"),
+ "test_schema",
+ auditInfo);
+ store.put(schema, false);
+
+ SecurableObject catalogObject = SecurableObjects.ofCatalog("catalog",
Lists.newArrayList());
+ SecurableObject schemaObject =
+ SecurableObjects.ofSchema(
+ catalogObject, "test_schema",
Lists.newArrayList(Privileges.UseSchema.allow()));
+
+ RoleEntity role =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("test_role")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList(schemaObject))
+ .build();
+ store.put(role, false);
+
+ SupportsRelationOperations relationOperations =
(SupportsRelationOperations) store;
+
+ // Warm the cache: schema -> [test_role].
+ List<RoleEntity> rolesBeforeRevoke =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertEquals(1, rolesBeforeRevoke.size());
+ Assertions.assertEquals("test_role", rolesBeforeRevoke.get(0).name());
+
+ // Revoke all privileges on the schema (the object is removed from the
role). The removed
+ // object is absent from the updated entity, so it must be invalidated
via the role side.
+ store.update(
+ role.nameIdentifier(),
+ RoleEntity.class,
+ Entity.EntityType.ROLE,
+ existing ->
+ RoleEntity.builder()
+ .withId(existing.id())
+ .withName(existing.name())
+ .withNamespace(existing.namespace())
+ .withProperties(existing.properties())
+ .withAuditInfo(existing.auditInfo())
+ .withSecurableObjects(Lists.newArrayList())
+ .build());
+
+ List<RoleEntity> rolesAfterRevoke =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertTrue(
+ rolesAfterRevoke.stream().noneMatch(r ->
r.name().equals("test_role")),
+ "revoke must be immediately visible: the role must no longer bind
the schema");
+
+ } finally {
+ destroy(type);
+ }
+ }
+ }
+
+ /**
+ * Guards the subtraction side: overriding a role so that a
previously-bound, already-warmed
+ * metadata object is dropped must make listBindingRoleNames immediately
drop the role for that
+ * object. The removed object is absent from the updated entity, so
invalidation must come from
+ * the role side via the reverse index.
+ */
+ @ParameterizedTest
+ @MethodSource("storageProvider")
+ void testOverrideRemoveObjectInvalidatesMetadataObjectRoleRelCache(
+ String type, boolean enableCache) throws Exception {
+ Config config = Mockito.mock(Config.class);
+ Mockito.when(config.get(Configs.CACHE_ENABLED)).thenReturn(enableCache);
+ init(type, config);
+
+ AuditInfo auditInfo =
+
AuditInfo.builder().withCreator("creator").withCreateTime(Instant.now()).build();
+
+ try (EntityStore store = EntityStoreFactory.createEntityStore(config)) {
+ try {
+ store.initialize(config);
+
+ BaseMetalake metalake =
+ createBaseMakeLake(RandomIdGenerator.INSTANCE.nextId(),
"metalake", auditInfo);
+ store.put(metalake, false);
+
+ CatalogEntity catalog =
+ createCatalog(
+ RandomIdGenerator.INSTANCE.nextId(),
+ NamespaceUtil.ofCatalog("metalake"),
+ "catalog",
+ auditInfo);
+ store.put(catalog, false);
+
+ SchemaEntity schema =
+ createSchemaEntity(
+ RandomIdGenerator.INSTANCE.nextId(),
+ Namespace.of("metalake", "catalog"),
+ "test_schema",
+ auditInfo);
+ store.put(schema, false);
+
+ SecurableObject catalogObject =
+ SecurableObjects.ofCatalog(
+ "catalog", Lists.newArrayList(Privileges.UseCatalog.allow()));
+ SecurableObject schemaObject =
+ SecurableObjects.ofSchema(
+ catalogObject, "test_schema",
Lists.newArrayList(Privileges.UseSchema.allow()));
+
+ // role binds both catalog and schema.
+ RoleEntity role =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("test_role")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList(catalogObject,
schemaObject))
+ .build();
+ store.put(role, false);
+
+ SupportsRelationOperations relationOperations =
(SupportsRelationOperations) store;
+
+ // Warm the cache: schema -> [test_role].
+ List<RoleEntity> rolesBeforeOverride =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertEquals(1, rolesBeforeOverride.size());
+ Assertions.assertEquals("test_role",
rolesBeforeOverride.get(0).name());
+
+ // Override to keep only the catalog object, dropping the schema
object.
+ store.update(
+ role.nameIdentifier(),
+ RoleEntity.class,
+ Entity.EntityType.ROLE,
+ existing ->
+ RoleEntity.builder()
+ .withId(existing.id())
+ .withName(existing.name())
+ .withNamespace(existing.namespace())
+ .withProperties(existing.properties())
+ .withAuditInfo(existing.auditInfo())
+ .withSecurableObjects(Lists.newArrayList(catalogObject))
+ .build());
+
+ List<RoleEntity> rolesAfterOverride =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertTrue(
+ rolesAfterOverride.stream().noneMatch(r ->
r.name().equals("test_role")),
+ "override-remove must be immediately visible: the role must no
longer bind the schema");
+
+ } finally {
+ destroy(type);
+ }
+ }
+ }
+
+ /**
+ * Guards the subtraction side: deleting a role that bound an already-warmed
metadata object must
+ * make listBindingRoleNames immediately drop the role. deleteRole goes
through {@code
+ * store.delete} (not put/update), so the object-side invalidation must come
from the role-side
+ * {@code invalidate(roleIdent, ROLE)} via the reverse index.
+ */
+ @ParameterizedTest
+ @MethodSource("storageProvider")
+ void testDeleteRoleInvalidatesMetadataObjectRoleRelCache(String type,
boolean enableCache)
+ throws Exception {
+ Config config = Mockito.mock(Config.class);
+ Mockito.when(config.get(Configs.CACHE_ENABLED)).thenReturn(enableCache);
+ init(type, config);
+
+ AuditInfo auditInfo =
+
AuditInfo.builder().withCreator("creator").withCreateTime(Instant.now()).build();
+
+ try (EntityStore store = EntityStoreFactory.createEntityStore(config)) {
+ try {
+ store.initialize(config);
+
+ BaseMetalake metalake =
+ createBaseMakeLake(RandomIdGenerator.INSTANCE.nextId(),
"metalake", auditInfo);
+ store.put(metalake, false);
+
+ CatalogEntity catalog =
+ createCatalog(
+ RandomIdGenerator.INSTANCE.nextId(),
+ NamespaceUtil.ofCatalog("metalake"),
+ "catalog",
+ auditInfo);
+ store.put(catalog, false);
+
+ SchemaEntity schema =
+ createSchemaEntity(
+ RandomIdGenerator.INSTANCE.nextId(),
+ Namespace.of("metalake", "catalog"),
+ "test_schema",
+ auditInfo);
+ store.put(schema, false);
+
+ SecurableObject catalogObject = SecurableObjects.ofCatalog("catalog",
Lists.newArrayList());
+ SecurableObject schemaObject =
+ SecurableObjects.ofSchema(
+ catalogObject, "test_schema",
Lists.newArrayList(Privileges.UseSchema.allow()));
+
+ RoleEntity role =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("test_role")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList(schemaObject))
+ .build();
+ store.put(role, false);
+
+ SupportsRelationOperations relationOperations =
(SupportsRelationOperations) store;
+
+ // Warm the cache: schema -> [test_role].
+ List<RoleEntity> rolesBeforeDelete =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertEquals(1, rolesBeforeDelete.size());
+ Assertions.assertEquals("test_role", rolesBeforeDelete.get(0).name());
+
+ // Delete the role; its binding must vanish from the warmed schema
cache.
+ store.delete(role.nameIdentifier(), Entity.EntityType.ROLE);
+
+ List<RoleEntity> rolesAfterDelete =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertTrue(
+ rolesAfterDelete.stream().noneMatch(r ->
r.name().equals("test_role")),
+ "deleteRole must be immediately visible: the role must no longer
bind the schema");
+
+ } finally {
+ destroy(type);
+ }
+ }
+ }
+
+ /**
+ * Covers the createRole path (which goes through {@code store.put}, not
update): creating a role
+ * that already carries securable objects must make listBindingRoleNames on
those objects
+ * immediately reflect the new role, even when the object's relation cache
was warmed beforehand.
+ */
+ @ParameterizedTest
+ @MethodSource("storageProvider")
+ void testCreateRoleWithSecurableObjectsInvalidatesMetadataObjectRoleRelCache(
+ String type, boolean enableCache) throws Exception {
+ Config config = Mockito.mock(Config.class);
+ Mockito.when(config.get(Configs.CACHE_ENABLED)).thenReturn(enableCache);
+ init(type, config);
+
+ AuditInfo auditInfo =
+
AuditInfo.builder().withCreator("creator").withCreateTime(Instant.now()).build();
+
+ try (EntityStore store = EntityStoreFactory.createEntityStore(config)) {
+ try {
+ store.initialize(config);
+
+ BaseMetalake metalake =
+ createBaseMakeLake(RandomIdGenerator.INSTANCE.nextId(),
"metalake", auditInfo);
+ store.put(metalake, false);
+
+ CatalogEntity catalog =
+ createCatalog(
+ RandomIdGenerator.INSTANCE.nextId(),
+ NamespaceUtil.ofCatalog("metalake"),
+ "catalog",
+ auditInfo);
+ store.put(catalog, false);
+
+ SchemaEntity schema =
+ createSchemaEntity(
+ RandomIdGenerator.INSTANCE.nextId(),
+ Namespace.of("metalake", "catalog"),
+ "test_schema",
+ auditInfo);
+ store.put(schema, false);
+
+ SecurableObject catalogObject = SecurableObjects.ofCatalog("catalog",
Lists.newArrayList());
+ SecurableObject schemaObject =
+ SecurableObjects.ofSchema(
+ catalogObject, "test_schema",
Lists.newArrayList(Privileges.UseSchema.allow()));
+
+ RoleEntity roleA =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("roleA")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList(schemaObject))
+ .build();
+ store.put(roleA, false);
+
+ SupportsRelationOperations relationOperations =
(SupportsRelationOperations) store;
+
+ // Warm the cache: schema -> [roleA].
+ List<RoleEntity> rolesBeforeCreate =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ Assertions.assertEquals(1, rolesBeforeCreate.size());
+ Assertions.assertEquals("roleA", rolesBeforeCreate.get(0).name());
+
+ // Create roleB already bound to the schema (createRole goes through
store.put).
+ RoleEntity roleB =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("roleB")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList(schemaObject))
+ .build();
+ store.put(roleB, false);
+
+ // listBindingRoleNames(schema) must immediately reflect roleB.
+ List<RoleEntity> rolesAfterCreate =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.METADATA_OBJECT_ROLE_REL,
+ schema.nameIdentifier(),
+ Entity.EntityType.SCHEMA,
+ true);
+ List<String> roleNames =
+
rolesAfterCreate.stream().map(RoleEntity::name).sorted().collect(Collectors.toList());
+ Assertions.assertEquals(
+ Lists.newArrayList("roleA", "roleB"),
+ roleNames,
+ "createRole with securable objects must be immediately visible");
+
+ } finally {
+ destroy(type);
+ }
+ }
+ }
+
+ /**
+ * Covers the {@code grantRolesToUser} path: after a role is granted to a
user that was never
+ * cached against that role, listEntitiesByRelation(ROLE_USER_REL,
roleIdent) must immediately
+ * reflect the new user. Same defect shape as {@link
+ * #testGrantPrivilegeInvalidatesMetadataObjectRoleRelCache}, on
user.roleNames instead of
+ * role.securableObjects.
+ */
+ @ParameterizedTest
+ @MethodSource("storageProvider")
+ void testGrantRolesToUserInvalidatesRoleUserRelCache(String type, boolean
enableCache)
+ throws Exception {
+ Config config = Mockito.mock(Config.class);
+ Mockito.when(config.get(Configs.CACHE_ENABLED)).thenReturn(enableCache);
+ init(type, config);
+
+ AuditInfo auditInfo =
+
AuditInfo.builder().withCreator("creator").withCreateTime(Instant.now()).build();
+
+ try (EntityStore store = EntityStoreFactory.createEntityStore(config)) {
+ try {
+ store.initialize(config);
+
+ BaseMetalake metalake =
+ createBaseMakeLake(RandomIdGenerator.INSTANCE.nextId(),
"metalake", auditInfo);
+ store.put(metalake, false);
+
+ RoleEntity roleA =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("roleA")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList())
+ .build();
+ store.put(roleA, false);
+
+ UserEntity user1 =
+ UserEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("user1")
+ .withNamespace(AuthorizationUtils.ofUserNamespace("metalake"))
+ .withRoleNames(Lists.newArrayList("roleA"))
+ .withRoleIds(Lists.newArrayList(roleA.id()))
+ .withAuditInfo(auditInfo)
+ .build();
+ store.put(user1, false);
+
+ UserEntity user2 =
+ UserEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("user2")
+ .withNamespace(AuthorizationUtils.ofUserNamespace("metalake"))
+ .withRoleNames(Lists.newArrayList())
+ .withRoleIds(Lists.newArrayList())
+ .withAuditInfo(auditInfo)
+ .build();
+ store.put(user2, false);
+
+ SupportsRelationOperations relationOperations =
(SupportsRelationOperations) store;
+
+ // Warm the cache: roleA -> [user1].
+ List<UserEntity> usersBeforeGrant =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.ROLE_USER_REL,
+ roleA.nameIdentifier(),
+ Entity.EntityType.ROLE,
+ true);
+ Assertions.assertEquals(1, usersBeforeGrant.size());
+ Assertions.assertEquals("user1", usersBeforeGrant.get(0).name());
+
+ // Simulate grantRolesToUser(roleA -> user2): store.update(user2,
roleNames=[roleA]).
+ store.update(
+ user2.nameIdentifier(),
+ UserEntity.class,
+ Entity.EntityType.USER,
+ existing ->
+ UserEntity.builder()
+ .withId(existing.id())
+ .withName(existing.name())
+ .withNamespace(existing.namespace())
+ .withRoleNames(Lists.newArrayList("roleA"))
+ .withRoleIds(Lists.newArrayList(roleA.id()))
+ .withAuditInfo(auditInfo)
+ .build());
+
+ // roleA -> users must immediately reflect user2.
+ List<UserEntity> usersAfterGrant =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.ROLE_USER_REL,
+ roleA.nameIdentifier(),
+ Entity.EntityType.ROLE,
+ true);
+ List<String> names =
+
usersAfterGrant.stream().map(UserEntity::name).sorted().collect(Collectors.toList());
+ Assertions.assertEquals(
+ Lists.newArrayList("user1", "user2"),
+ names,
+ "grantRolesToUser must be immediately visible via role->users");
+
+ } finally {
+ destroy(type);
+ }
+ }
+ }
+
+ /**
+ * Covers the {@code grantRolesToGroup} path: after a role is granted to a
group that was never
+ * cached against that role, listEntitiesByRelation(ROLE_GROUP_REL,
roleIdent) must immediately
+ * reflect the new group.
+ */
+ @ParameterizedTest
+ @MethodSource("storageProvider")
+ void testGrantRolesToGroupInvalidatesRoleGroupRelCache(String type, boolean
enableCache)
+ throws Exception {
+ Config config = Mockito.mock(Config.class);
+ Mockito.when(config.get(Configs.CACHE_ENABLED)).thenReturn(enableCache);
+ init(type, config);
+
+ AuditInfo auditInfo =
+
AuditInfo.builder().withCreator("creator").withCreateTime(Instant.now()).build();
+
+ try (EntityStore store = EntityStoreFactory.createEntityStore(config)) {
+ try {
+ store.initialize(config);
+
+ BaseMetalake metalake =
+ createBaseMakeLake(RandomIdGenerator.INSTANCE.nextId(),
"metalake", auditInfo);
+ store.put(metalake, false);
+
+ RoleEntity roleA =
+ RoleEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("roleA")
+ .withNamespace(AuthorizationUtils.ofRoleNamespace("metalake"))
+ .withProperties(null)
+ .withAuditInfo(auditInfo)
+ .withSecurableObjects(Lists.newArrayList())
+ .build();
+ store.put(roleA, false);
+
+ GroupEntity group1 =
+ GroupEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("group1")
+ .withNamespace(AuthorizationUtils.ofGroupNamespace("metalake"))
+ .withRoleNames(Lists.newArrayList("roleA"))
+ .withRoleIds(Lists.newArrayList(roleA.id()))
+ .withAuditInfo(auditInfo)
+ .build();
+ store.put(group1, false);
+
+ GroupEntity group2 =
+ GroupEntity.builder()
+ .withId(RandomIdGenerator.INSTANCE.nextId())
+ .withName("group2")
+ .withNamespace(AuthorizationUtils.ofGroupNamespace("metalake"))
+ .withRoleNames(Lists.newArrayList())
+ .withRoleIds(Lists.newArrayList())
+ .withAuditInfo(auditInfo)
+ .build();
+ store.put(group2, false);
+
+ SupportsRelationOperations relationOperations =
(SupportsRelationOperations) store;
+
+ // Warm the cache: roleA -> [group1].
+ List<GroupEntity> groupsBeforeGrant =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.ROLE_GROUP_REL,
+ roleA.nameIdentifier(),
+ Entity.EntityType.ROLE,
+ true);
+ Assertions.assertEquals(1, groupsBeforeGrant.size());
+ Assertions.assertEquals("group1", groupsBeforeGrant.get(0).name());
+
+ // Simulate grantRolesToGroup(roleA -> group2): store.update(group2,
roleNames=[roleA]).
+ store.update(
+ group2.nameIdentifier(),
+ GroupEntity.class,
+ Entity.EntityType.GROUP,
+ existing ->
+ GroupEntity.builder()
+ .withId(existing.id())
+ .withName(existing.name())
+ .withNamespace(existing.namespace())
+ .withRoleNames(Lists.newArrayList("roleA"))
+ .withRoleIds(Lists.newArrayList(roleA.id()))
+ .withAuditInfo(auditInfo)
+ .build());
+
+ // roleA -> groups must immediately reflect group2.
+ List<GroupEntity> groupsAfterGrant =
+ relationOperations.listEntitiesByRelation(
+ SupportsRelationOperations.Type.ROLE_GROUP_REL,
+ roleA.nameIdentifier(),
+ Entity.EntityType.ROLE,
+ true);
+ List<String> names =
+
groupsAfterGrant.stream().map(GroupEntity::name).sorted().collect(Collectors.toList());
+ Assertions.assertEquals(
+ Lists.newArrayList("group1", "group2"),
+ names,
+ "grantRolesToGroup must be immediately visible via role->groups");
+
+ } finally {
+ destroy(type);
+ }
+ }
+ }
+
private FunctionEntity createFunctionEntity(
Long id, Namespace namespace, String name, AuditInfo auditInfo) {
FunctionParam param1 = FunctionParams.of("param1",
Types.IntegerType.get());
