AynSiddiqui opened a new pull request, #2679: URL: https://github.com/apache/iceberg-go/pull/2679
Fixes #2443. Follows #2492, which added the key and header. ## Motivation #2492 sends an `Idempotency-Key` on REST mutations when the server advertises `idempotency-key-lifetime`, but nothing resends a failed request or limits how late a resend may be. `net/http` already replays a keyed POST on a dropped reused connection, even when the lifetime is `PT0S`, and the scan plan and task POSTs have a separate retry loop that ignores the advertised lifetime. This implements the retry rules agreed in #2443. ## Changes - One key and body per call, reused for every resend. A resend happens only after a 500, 502, 503 or 504, a timeout or a connection error. A 4xx is never resent, and a transaction refreshed after a 409 is a new call with a new key. - Keyed resends happen only when `idempotency-key-lifetime` parses (ISO-8601 `PnDTnHnMnS`) and is positive. The window starts at the first attempt, and a resend stops when its delay would reach the lifetime. Backoff is jittered, honors `Retry-After` and the context, and is capped at three resends. - A commit (`CommitTable`, `UpdateTable`, `CommitTransaction`) that cannot be resent after an ambiguous failure returns `ErrCommitStateUnknown`. Other calls return the last error. - `net/http`'s own replay of a keyed request with a body, including 307/308 redirects, is held to the same window through `GetBody`. A keyed `DELETE` has no body, so its transport replay cannot be bounded this way. This is documented and covered by a test. - Scan plan and task POSTs use the same code and the old retry loop is removed. They no longer resend on 408 or 429, now resend after a timeout or connection error, and send a generated key only when a lifetime is advertised. A caller-supplied `IdempotencyKey` is always sent; its age from the UUIDv7 timestamp counts against the lifetime (a future timestamp is clamped), and without an advertised lifetime it is sent once and not resent. Scan GET polling is unchanged. - When `GetBody` refuses a replay, `Do` returns the `GetBody` error, not the original connection error (seen on go1.26.3 and go1.27.1). - `configuration.md` and the unreleased compatibility notes in `releases.md` describe the final behavior, replacing the "lifetime is not enforced" warning from #2492. ### Differences from Java - Java stops bounding resends when the lifetime is malformed (it falls back to no limit). Here a missing, zero, negative or malformed lifetime allows no keyed resends. The key is still sent whenever the lifetime is present. - Java also retries 408 and 429 on keyed requests; this does not, as agreed in #2443. - Go also has `net/http`'s implicit replay, which is why the window is applied there too, with the DELETE gap above. ## For reviewers - a. Backoff is 3 resends with 100 ms to 5 s jitter (the old scan constants); Java defaults to 5 retries from 1 s. Which do you want? - b. 408 and 429 are not resent on keyed requests, per "never a 4xx"; Java does resend them. - c. Only 500, 502, 503 and 504 count as retryable 5xx. - d. A resend must start before the window ends (`elapsed + wait < lifetime`); it can finish after. - e. `UpdateView` returns the last error, not `ErrCommitStateUnknown`, as it has no unknown-state mapping today. - f. A network error from the auth token fetch is also resent. ## Testing `go build ./...`, `go vet ./catalog/rest/...`, `golangci-lint run ./catalog/rest/...` and `go test ./catalog/rest/... -race -count=1` pass. New `httptest` tests cover matching keys and bodies on resends (5xx, connection error, DELETE), fresh keys per call and after a 409, no resend on 4xx, no resend with a missing, zero, negative, malformed or empty lifetime, expiry (`Retry-After` past the window, slow first attempt), `ErrCommitStateUnknown`, cancellation, the resend budget, real-transport replay inside the window, after a slow first attempt and with `PT0S`, redirects, the keyed DELETE gap, the lifetime parser, and both scan POSTs including caller-key age. The tests for resends, transport replay bounds and scan POST changes fail on `main`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
