yuqi1129 opened a new issue, #12166:
URL: https://github.com/apache/gravitino/issues/12166
### Describe the subtask
Implement real optimistic concurrency control (version CAS) on the Gravitino
store's entity write paths, per the OCC requirements in the design doc (PR
#12157). Today no entity does a clean version CAS: `alter` (UPDATE) carries
`current_version` in its `WHERE` but **freezes** the version for
metalake/catalog/schema/topic (only `table` increments) and leans on a fragile
full-row compare; `drop` (soft-delete) has **no** version predicate at all
(`WHERE id=? AND deleted_at=0`, and the mapper takes only the id).
### Scope
1. **alter — always raise the version.** Every OCC-covered entity's
successful UPDATE must set `current_version = old + 1`, and the UPDATE
predicate is reduced to `id = ? AND current_version = ? AND deleted_at = 0`
(drop the `audit_info`/name/... full-row compare). Frozen today: metalake,
catalog, schema, topic; conditional: fileset, policy; already increments: table
(still has a full-row `WHERE` to slim).
2. **drop — CAS delete.** Add an expected-version parameter to the
soft-delete path; change the SQL to `UPDATE ... SET deleted_at=<now> WHERE id=?
AND current_version=? AND deleted_at=0`. (Today `softDelete...ById(Long id)`
takes only the id.)
3. **drop — report 0 vs 1 rows.** The soft-delete returns the affected-row
count so the service can tell **1 = deleted** from **0 = version changed /
already gone**. Map 0 rows by re-reading: a version mismatch -> OCC conflict
(retry -> HTTP 409); an already-deleted row -> `NoSuchEntity` / idempotent
`false`, per the API contract.
4. **Backward compatibility with existing data.**
- **No schema migration**: `current_version`/`last_version` already exist
(DEFAULT 1). Existing rows keep their value; the first OCC write
reads-then-CAS's on it (frozen rows sitting at `current_version=1` bump to 2 on
first write — no backfill needed).
- **Rolling upgrade / mixed nodes**: an old-code node's write does
**not** bump the version, so a new-code node's CAS would not detect it. Gate
the change (feature flag, or a single-release cutover with a documented note)
so a cluster is never half-CAS / half-frozen.
- **Ordering**: land "always increment version" first, then slim the
`WHERE`; never remove the full-row compare while any path still depends on it.
- **Out of scope of CAS**: `put(overwrite=true)` on
create/import/reconcile stays a blind upsert — CAS applies only to update and
soft-delete.
### Acceptance criteria
- alter/alter lost-update: two concurrent alters — one wins (`N->N+1`), the
other matches 0 rows -> OCC conflict.
- drop CAS: stale-version drop returns 0 rows -> reported as conflict;
current-version drop returns 1 row -> success.
- alter/drop: current-version drop wins; the concurrent alter then matches 0
rows (via `deleted_at`) — no lost write, no stale row.
- change-then-change-back (A->B->A) is caught by version, not by JSON byte
equality.
- existing rows (`current_version=1`) upgrade cleanly with no migration.
- tests pass on MySQL and PostgreSQL.
### Relevant code
- `POConverters.update*POWithVersion` — version freeze/increment
- `*MetaBaseSQLProvider.update*Meta` (alter `WHERE`) and `softDelete*ById`
(drop `WHERE`)
- `*MetaService.delete*` — pass the version, interpret the row count
### Parent issue
https://github.com/apache/gravitino/issues/10238
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
