nevzheng opened a new issue, #14391: URL: https://github.com/apache/gravitino/issues/14391
### What would you like to be improved? **Priority: P2** (scale in the parent epic; impact if hit, not scheduling) The [async drop](https://github.com/apache/gravitino/blob/bb634defdc5663a8124be5fca1c8d9e52ac096ac/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergTableOperationExecutor.java#L151-L169) writes to two stores in sequence: it removes the backend catalog entry, then inserts the cleanup row. Nothing ties the two together, so a failure between them leaves a state neither side records. Observed symptoms (each reproduced by a test in the parent epic): | Symptom | Effect | | ------- | ------ | | If `addJob` (or the catalog id lookup) throws after the catalog drop, or the server stops between the steps, the table is gone and no row exists. | Files are never deleted and nothing records them. The name is free at once, and the client gets an error for a drop that already happened. | | The metadata location is read before the drop. A commit landing in between adds files and a newer `metadata.json` the job cannot reach. | Files remain while the job ends `SUCCEEDED`. Sync purge has a similar window, so this is low severity. | ### How should we improve? Write the row first, in a state that reserves the name but that workers do not claim (for example `PREPARING`). Then drop the catalog entry, then promote the row to `PENDING`. If the drop fails, delete the row. A row left `PREPARING` by a crash can be reconciled at startup against whether the table still exists. The read-then-drop race would also need a conditional drop from the backend catalog. Filed for discussion and prioritization; no fix is scheduled as part of this report. Parent: the async purge audit epic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
