nevzheng opened a new issue, #12219: URL: https://github.com/apache/gravitino/issues/12219
### Describe the proposal Add a durable soft-deletion lifecycle to Gravitino, beginning with Iceberg REST tables. Soft deletion should allow an object to be hidden from normal use while its metadata is retained for a configurable recovery period. During that period, an authorized user can restore the original object. After the recovery period ends, Gravitino can asynchronously complete any required physical cleanup. The implementation should provide: - A durable identity for each deletion generation. - Retention of the original metadata identity during the recovery period. - Discovery of deleted objects through the appropriate metadata APIs. - Restoration of the original object rather than creation of a replacement. - Reservation of the deleted object's name until restoration or final cleanup. - Durable, asynchronous physical cleanup with observable progress and retry behavior. - Safe and deterministic behavior for retries, concurrent restore and cleanup requests, server restarts, and repeated deletion requests. - Lifecycle audit records that do not expose credentials or storage secrets. - Compatibility with existing behavior when soft deletion is not enabled. The initial implementation targets Iceberg REST tables. The underlying deletion model should allow additional Gravitino entity types and cleanup handlers to be added later without changing the identity model. Detailed API contracts, metadata relationships, lifecycle transitions, and cleanup execution semantics will be documented in a separate public design document. Related work: - #12187 tracks management-API semantics for metadata undelete. - #11265 introduced asynchronous hard deletion for the Iceberg REST catalog. ### Task list - [ ] Add durable deletion metadata and storage integration - [ ] Add Iceberg REST soft-delete discovery and recovery APIs - [ ] Add durable asynchronous purge execution — pending design - [ ] Add cleanup progress, observability, and operator controls — pending design - [ ] Add integration, concurrency, restart, and scale validation — pending design - [ ] Extend soft deletion to additional entity types — pending design -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
