Hi Huaxin. Thanks again for starting this proposal.
After a quick initial read of the docs, this feels like a proposal to treat each REST change request as an identifiable "task" or "operation" where the ID is allocated by the client and the execution status is tracked by the server. Essentially, with the proposed feature implemented, it would be possible to provide a "get status" API for each "idempotency key". Does that make sense? Do you think the proposal could be reformulated in terms of "tasks/operations"? From my POV it might be clearer and easier to understand. In that regard, as far as Polaris is concerned, I think it connected well and could build on top of the tasks proposal [1]. [1] https://lists.apache.org/thread/gg0kn89vmblmjgllxn7jkn8ky2k28f5l Thanks, Dmitri. On Thu, Sep 11, 2025 at 2:37 PM huaxin gao <[email protected]> wrote: > Hi all, > > I’d like to propose adding *optional idempotent retries* to Polaris REST > mutation endpoints via an Idempotency-Key header. > > *The Problem* > When a mutation (e.g., POST /v1/tables/{name}/commit) succeeds but the > client doesn’t receive the response (timeout, restart, etc.), a retry can > hit 409 Conflict. Some clients then treat this as a failed commit and clean > up files that actually belong to the committed snapshot, causing > catalog/storage inconsistency. > > *The Proposed Solution* > Accept Idempotency-Key on mutation routes. The server binds the key to a > canonical payload hash and enforces: > > - > > *Same key + same payload* -> return the original result (no > re-execution). > - > > *Same key + different payload* -> 422 Unprocessable Content. > - > > *In-flight duplicate* -> 409 Conflict. > Transient 5xx are never cached. > > *Discovery & Compatibility* > When serving Iceberg REST, Polaris can advertise support and retention via > GET > /v1/config, e.g.: > > { "properties": { "idempotency-key-supported": "true", > "idempotency-key-lifetime": "PT30M" } } > > Fully backward compatible: servers may ignore the header; clients enable > retries only when discovery indicates support. > > This is the *server-side counterpart* to the Iceberg REST Catalog > Idempotency proposal > < > https://docs.google.com/document/d/1WyiIk08JRe8AjWh63txIP4i2xcIUHYQWFrF_1CCS3uw/edit?tab=t.0#heading=h.jfecktgonj1i > > > I sent to the Iceberg community. > Here is the Polaris proposal > < > https://docs.google.com/document/d/1L5A8Cspugsk1dW4Ij4dy5wB5FKa8KnaXT0LHBJdgq9w/edit?usp=sharing > >. > Feedback welcome! > > Thanks, > > Huaxin >
