mengw15 opened a new issue, #7742:
URL: https://github.com/apache/texera/issues/7742
### Task Summary
Found while testing the flag-gated per-user warehouse feature (#6870); no
deployment is affected because the flag defaults to off.
`LakekeeperClient.deleteWarehouseEmptyFirst` (used by `DELETE
/warehouse/{whid}`, introduced in #7473) empties the warehouse by dropping
every table with `purgeRequested=true`, then immediately deletes the warehouse
entity. Lakekeeper runs those purges as **asynchronous tasks** (queue
`tabular_purge`), so the warehouse delete races the queue: while any purge task
is still pending, Lakekeeper answers
```
HTTP 409 {"error":{"message":"Warehouse has unfinished tasks. Cannot delete
warehouse until all tasks are
finished.","type":"WarehouseHasUnfinishedTasks","code":409}}
```
and the client surfaces it as a failure. In practice the first delete of any
warehouse that has ever stored execution results always fails; a second attempt
after the queue drained succeeds.
To reproduce: create a warehouse, run one workflow execution into it (this
creates three tables: result, stats, console), then delete the warehouse — the
first attempt fails with the 409 above (`"Unfinished tasks: 3 Tasks in queue
'tabular_purge'"`), and a retry a few seconds later succeeds. Deterministic
unit-level repro: a `LakekeeperClientSpec` case whose stubbed management
endpoint answers the warehouse delete with 409 `WarehouseHasUnfinishedTasks`
twice and then 204 — `deleteWarehouseEmptyFirst` should succeed, but currently
throws on the first 409.
Expected: `deleteWarehouseEmptyFirst` treats 409
`WarehouseHasUnfinishedTasks` as transient and retries the final warehouse
delete with a bounded backoff (the purge normally drains within seconds),
failing only if the queue never drains within the bound. All other error
responses still fail immediately.
Part of #6870.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
