yuqi1129 opened a new issue, #13621: URL: https://github.com/apache/gravitino/issues/13621
### Describe the proposal `EntityStore` is the only way core, catalogs and the server reach Gravitino metadata, and `gravitino.entity.store` accepts any implementation class. Recent concurrency fixes keep adding one-off capability interfaces (`SupportsIdentityFencedDelete` in #13198, `SupportsConditionalCatalogDelete` in #13499) or bypass the store with direct mapper calls. The root cause is the interface: parts of its contract are unwritten or wrong, and some capabilities callers need cannot be expressed. This epic fixes the interface first; #13172 and #13176 then become ordinary users of it. Principles: - The interface states semantics in domain terms: identity, write intent, read freshness, and the outcome of a conflict. How they are achieved (row versions, CAS, row locks, transactions) stays inside the implementation, as `OccWriteSupport` does today. - Additions are `default` methods, so existing implementations keep working and reject what they cannot honor. - Checks the backend can make on its own (relation endpoint fences, import id checks, statistic CAS) stay inside the backend and are out of scope. #### Audit 1: the written contract differs from the behavior callers rely on | # | Problem | Evidence | |---|---|---| | C1 | No concurrency contract. `update` throws `OptimisticLockException` on a conflict, but the Javadoc lists only `NoSuchEntityException` and `EntityAlreadyExistsException`. Which methods are atomic, and what "should handle concurrent store" means for `put`, is unstated. | `OperationDispatcher`, `TableOperationDispatcher`, `JobManager`, `TagManager` and Lance catch it; a Lance test re-implements the behavior in a `CasEntityStore` test double. | | C2 | `put(e)` is documented as overwrite but calls `put(e, false)`, a strict insert. | `EntityStore.put(E)` | | C3 | `delete` is documented to return `false` for a missing entity, and `RelationalEntityStore` maps `NoSuchEntityException` to `false`, yet callers still catch `NoSuchEntityException` around it. | `CatalogManager`, `ManagedFunctionOperations` and others | | C4 | `batchGet` is documented to throw `NoSuchEntityException` for a missing entity, but skips it; USER, ROLE and VIEW are fetched one by one. Callers cannot tell a missing entity from a failed read. | `JDBCBackend.batchGet`; the JCasbin authorizer falls back to per-entity `get` when the batch comes back empty. | | C5 | `cascade` semantics per entity type are undocumented: which children make a non-cascade delete fail, and which relations are removed. | Only visible in each meta service. | | C6 | `update` may rename, but it is not stated which fields may change (id, parent) or who invalidates the old and new names' cache entries. | #13303 (stale cache entry after a rename by import); #13197 adds new-name invalidation. | | C7 | `executeInTransaction` is declared, but the only implementation throws `UnsupportedOperationException` and nothing calls it. | `RelationalEntityStore.executeInTransaction` | | C8 | Reads may be served from the per-node cache, which is not stated, and callers cannot request the current stored state. | #13198 adds `getEntityId` to bypass the cache. | #### Audit 2: capabilities callers need but cannot express | # | Missing capability | Current workaround | Related | |---|---|---|---| | G1 | Identity addressing. Every operation resolves the name when it runs, so a delayed write can hit a newer entity under the same name. | #13198 adds `deleteIfIdMatches`; `MetadataIdConverter` loads a whole entity to get its id; owner-by-object-id and id→name lookups go around the store. | E1/F5, E2/F6 in the concurrency review | | G2 | Write intent. One `overwrite` flag stands for create, import and reconcile. | All 8 `put(..., true)` call sites are external create/import paths; a natural-key conflict keeps the old id on MySQL and fails on PostgreSQL. | M7, #13303 | | G3 | Lifecycle preconditions. A delete has only a `cascade` flag; conditions such as "non-force, only these children may go" or "parent must be in use/enabled" cannot be decided with the write. | #13499 adds `deleteCatalogWithAllowedSchemas`; managers check first and write later. | M4, M5, #13176 | | G4 | Version and change awareness. Callers cannot get an entity's version or last update, so derived caches cannot tell when they are stale. | The JCasbin authorizer probes `user_meta`, `group_meta` and `role_meta.updated_at` and prefetches subjects with its own SQL. | Authorization caches | | G5 | Relation change notifications. The entity change log covers entities, not relations such as owners and grants. | `JcasbinChangeListener` polls `owner_meta` on its own. | Multi-node authorization | | G6 | Pagination and counting. | `UserGroupManager` calls `UserMetaService` / `GroupMetaService` directly. | — | | G7 | Multi-entity atomicity. | Not possible (see C7). | M5 in-use propagation; replacing a stale registration in #13198 | ### Task list Sub-tasks are being created and will be linked here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
