nevzheng opened a new issue, #12258:
URL: https://github.com/apache/gravitino/issues/12258
### Describe the proposal
This is the purge-execution child epic of #12219. It builds on the
relational deletion model in
#12220 and the Iceberg REST delete and UNDROP lifecycle in #12221. The
implementation follows the
API lifecycle stack ending in #12227.
The goal of this epic is to complete Iceberg REST deletion V2: retained
tables can be handed off to
durable, asynchronous purge jobs after their retention deadline, without
deleting Iceberg files on
the `DELETE` request thread.
#### End-to-end behavior
1. `DELETE` durably marks a table as deleted and records its retention
deadline. While retained,
the table remains hidden, its name remains reserved, and it may be
restored through UNDROP.
2. A periodic collector finds a bounded number of eligible `DELETED` actions
whose retention has
expired.
3. In one short relational transaction, the collector revalidates the exact
table and deletion
generation, creates its linked purge job, and moves the action from
`DELETED` to `PURGING`.
This transition closes the UNDROP window.
4. Existing asynchronous cleanup workers claim the job, renew its heartbeat
lease, and delete the
Iceberg registration and referenced files outside the request path.
5. After cleanup succeeds, one transaction completes the linked job and
removes only the exact
retained metadata generation. The table name may then be reused, and
ordinary lookup returns
`404` because the deletion has been fully consumed.
#### Consistency and recovery
- `table_meta.deletion_id` and `entity_deletion` remain the lifecycle source
of truth. Purge-job
state records execution and progress; it does not replace the deletion
lifecycle.
- The deletion action and linked job are handed off atomically, so there is
no durable `PURGING`
action without its job.
- Every linked job carries the immutable table ID and exact deletion ID.
Name snapshots are used
for routing, never as the sole deletion predicate.
- UNDROP and purge claim the same retained row under a fine-grained
relational lock. Exactly one can
move the action out of `DELETED`.
- Existing claim, compare-and-set, heartbeat, lease-reclaim, and durable
retry behavior make jobs
restart-safe across worker and server failures.
- External cleanup is repeatable: missing files are treated as already
complete, children are
deleted before parents, and the root Iceberg metadata file is deleted last.
- Finalization verifies the linked table ID, deletion ID, lifecycle state,
and purge job ID before
deleting relational metadata. An older deletion can never purge a later
same-name table.
- Manifest progress is advisory operator information, not a checkpoint. It
may restart from zero
after a lease reclaim without changing correctness.
#### Reuse and compatibility
This work extends the existing `iceberg_cleanup_job`,
`IcebergCleanupJobStore`, and
`IcebergCleanupManager` table-as-queue implementation. The existing
scheduled executor supplies
periodic wake-ups; correctness comes from the relational claim and heartbeat
lease rather than a
new scheduler, generic job framework, or saga.
Legacy V1 cleanup jobs continue through their current path. When deletion V2
is enabled, linked V2
jobs share the same durable queue and workers. Disabling new V2 collection
does not abandon work
already in `PURGING`; those jobs continue to drain safely.
#### Backward-compatibility contract
- Database changes are additive and must be deployed before deletion V2 is
enabled.
- An unlinked cleanup job is a V1 job and retains its existing execution and
completion behavior. A
V2 purge job is explicitly linked by both table ID and deletion ID;
partial linkage is rejected.
- V2-capable workers may process V1 and V2 jobs from the same queue.
Enabling V2 does not migrate,
reinterpret, or rewrite existing V1 jobs.
- Disabling V2 stops the creation of new linked purge jobs. Existing
`PURGING` jobs continue to
completion, while retained `DELETED` tables remain hidden and
name-reserved rather than falling
back to the V1 cleanup path.
- Rollout must upgrade all cleanup workers before V2 begins creating linked
jobs. Once linked jobs
or retained deletion actions exist, running or rolling back to a V1-only
worker is unsupported
because it cannot enforce the V2 generation predicates.
#### After this epic
Iceberg REST deletion V2 will support durable purge jobs end to end: bounded
expiry collection,
atomic ownership transfer, asynchronous cleanup, visible progress,
crash-safe retry, exact
generation finalization, and compatibility with existing V1 cleanup.
#### Out of scope
- A generic public Jobs API or operator control plane.
- Purge handlers for entity types other than Iceberg REST tables.
- A new scheduler framework or external queue service.
- Long-term audit storage or terminal-record retention policy.
- Full deployment-scale and multi-node performance validation, which belongs
to the integration and
scale-testing epic.
### Task list
TBD
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
