Hi all,
I’d like to propose adding *optional idempotent retries* to Polaris REST
mutation endpoints via an Idempotency-Key header.
*The Problem*
When a mutation (e.g., POST /v1/tables/{name}/commit) succeeds but the
client doesn’t receive the response (timeout, restart, etc.), a retry can
hit 409 Conflict. Some clients then treat this as a failed commit and clean
up files that actually belong to the committed snapshot, causing
catalog/storage inconsistency.
*The Proposed Solution*
Accept Idempotency-Key on mutation routes. The server binds the key to a
canonical payload hash and enforces:
-
*Same key + same payload* -> return the original result (no
re-execution).
-
*Same key + different payload* -> 422 Unprocessable Content.
-
*In-flight duplicate* -> 409 Conflict.
Transient 5xx are never cached.
*Discovery & Compatibility*
When serving Iceberg REST, Polaris can advertise support and retention via GET
/v1/config, e.g.:
{ "properties": { "idempotency-key-supported": "true",
"idempotency-key-lifetime": "PT30M" } }
Fully backward compatible: servers may ignore the header; clients enable
retries only when discovery indicates support.
This is the *server-side counterpart* to the Iceberg REST Catalog
Idempotency proposal
<https://docs.google.com/document/d/1WyiIk08JRe8AjWh63txIP4i2xcIUHYQWFrF_1CCS3uw/edit?tab=t.0#heading=h.jfecktgonj1i>
I sent to the Iceberg community.
Here is the Polaris proposal
<https://docs.google.com/document/d/1L5A8Cspugsk1dW4Ij4dy5wB5FKa8KnaXT0LHBJdgq9w/edit?usp=sharing>.
Feedback welcome!
Thanks,
Huaxin
