mengw15 opened a new pull request, #7744: URL: https://github.com/apache/texera/pull/7744
### What changes were proposed in this PR? `LakekeeperClient.deleteWarehouseEmptyFirst` empties the warehouse by dropping every table with `purgeRequested=true`, then immediately deletes the warehouse entity. Lakekeeper purges the dropped tables' **data files asynchronously** (task queue `tabular_purge`) and refuses to delete the warehouse while any purge is pending — the tasks need the warehouse's storage profile to reach S3, so deleting it first would orphan them and leak the files. It answers `409 WarehouseHasUnfinishedTasks` until the queue drains, so in practice the first delete of any warehouse that has ever stored execution results always failed; a retry seconds later succeeded (details in #7742). - **Retry the final warehouse delete on exactly this conflict**: `409` with `error.type == "WarehouseHasUnfinishedTasks"` is treated as transient and retried with a bounded pause (default 10 retries × 2s ≈ 20s; the queue normally drains within seconds). **Every other error — including any other 409 — still fails immediately**, and 404 stays the idempotent goal state. - The retry bound and delay are constructor parameters with defaults (`unfinishedTasksRetries = 10`, `unfinishedTasksRetryDelayMillis = 2000`), so production call sites are unchanged and the spec injects a zero delay — the tests contain no real sleeps and stay deterministic. Found while testing the flag-gated per-user warehouse feature (#6870); no deployment is affected because the flag defaults to off. ### Any related issues, documentation, discussions? Closes #7742. Part of #6870; `deleteWarehouseEmptyFirst` introduced in #7473. Verified against the local Lakekeeper (0.11.0) that its management API exposes no task-query endpoint or force-delete option, so waiting out the documented conflict type is the only client-side path. ### How was this PR tested? - `LakekeeperClientSpec` (in-process HTTP stub, no external infra) gains three cases mirroring the issue's repro: the stub answers the warehouse delete with `409 WarehouseHasUnfinishedTasks` twice and then 204 — **verified failing before the fix** (the first 409 threw) and passing after, with the stub asserting exactly 3 delete attempts; a warehouse whose queue never drains fails after the bounded 1+3 attempts; a 409 of any other type fails on the first attempt with no retry. - Full spec run locally: 7/7 passed; `WorkflowExecutionService/scalafmtCheck` (main + Test) passes. - The retry delay is injected as 0 in all tests — no real sleeps, deterministic across runs. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (claude-opus-4-8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
