nevzheng opened a new issue, #12221:
URL: https://github.com/apache/gravitino/issues/12221
### Describe the proposal
This is an Iceberg REST API-focused child epic of #12219 and depends on the
deletion storage foundation in #12220.
Add opt-in soft deletion and recovery for Iceberg REST tables while
preserving existing behavior when soft deletion is disabled.
## DELETE behavior
The existing Iceberg REST DELETE endpoint remains the entry point for
deleting a table.
When the deletion lifecycle applies, DELETE should:
- Create one immutable deletion generation.
- Mark the existing table metadata as deleted in one relational transaction.
- Reserve the canonical table name.
- Return `204 No Content` only after the deletion metadata is durable.
- Never delete Iceberg files on the request thread.
A repeated DELETE against the same active deletion should be idempotent: it
returns success without creating another deletion generation or changing the
original deletion time.
Normal table list and load operations must continue to hide deleted tables.
Creating or registering another table under a reserved name must fail until the
original deletion is restored or finalized.
## Deleted-table discovery
Reuse the existing Iceberg REST table resources rather than introducing
specialized deletion-discovery routes:
- Omitting `deleted`, or using `deleted=false`, returns live tables.
- Using `deleted=true` returns deleted tables only.
- Existing `pageToken` and `pageSize` behavior applies to deleted-table
listings.
- Loading a deleted table joins its retained table identity with the current
deletion action.
- Deleted-table responses include the deletion ID, lifecycle state,
revision, deletion time, retention deadline, recoverability, and a strong
deletion-action ETag.
- Responses must not expose Iceberg metadata locations, FileIO
configuration, credentials, or storage secrets.
- Deleted-table responses should prevent unsafe caching.
## UNDROP behavior
Add an explicit action endpoint:
`POST /management/v1/{prefix}/namespaces/{namespace}/tables/{table}/undrop`
The request contains the exact `deletionId` and must include the strong
deletion-action ETag in `If-Match`.
The deletion ID identifies the deletion generation. The ETag validates that
the client is acting on its current representation.
A successful UNDROP must:
- Verify that the exact deletion generation remains recoverable.
- Reactivate the original table and related metadata rows in one relational
transaction.
- Preserve the original table identity.
- Clear the table's active deletion pointer and name reservation.
- Record the lifecycle transition.
- Return the ordinary live Iceberg table response and live-table ETag.
UNDROP is an action, not a generic metadata PATCH. It does not create a
replacement table or perform an unregister/register saga.
An exact replay of a successful UNDROP may return the current restored
table. An UNDROP request must not affect an unrelated live table or a later
deletion generation using the same name.
## Concurrency and errors
DELETE, create, discovery, and UNDROP must use the exact deletion
generation, optimistic revisions, and transaction-scoped row-level coordination.
The API should distinguish malformed requests, missing preconditions, stale
ETags, conflicting current state, expired or purge-boundary deletions, missing
objects, and authorization failures with consistent HTTP status codes.
Unauthorized callers must not be able to use the deleted-table APIs to
discover metadata they could not otherwise access.
## Testing
Add tests covering:
- Soft deletion enabled and disabled.
- Retention boundaries, including zero retention.
- DELETE retry behavior.
- Live-only and deleted-only list and load behavior.
- Deleted-table pagination.
- Strong ETag generation and conditional requests.
- Missing, malformed, weak, wildcard, and stale `If-Match` values.
- Successful UNDROP and exact replay.
- Same-name creation conflicts.
- Concurrent lifecycle changes.
- Authorization and concealment of inaccessible deletions.
- Exclusion of credentials and internal cleanup data from API responses.
This epic does not implement garbage-collector selection, physical Iceberg
file deletion, purge workers, job progress APIs, or large-scale cleanup
testing. Those capabilities remain in the pending purge and validation epics.
### Task list
- [ ] Define implementation subtasks and reviewable PR boundaries — pending
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
