AynSiddiqui opened a new pull request, #2679:
URL: https://github.com/apache/iceberg-go/pull/2679

   Fixes #2443. Follows #2492, which added the key and header.
   
   ## Motivation
   
   #2492 sends an `Idempotency-Key` on REST mutations when the server 
advertises `idempotency-key-lifetime`, but nothing resends a failed request or 
limits how late a resend may be. `net/http` already replays a keyed POST on a 
dropped reused connection, even when the lifetime is `PT0S`, and the scan plan 
and task POSTs have a separate retry loop that ignores the advertised lifetime. 
This implements the retry rules agreed in #2443.
   
   ## Changes
   
   - One key and body per call, reused for every resend. A resend happens only 
after a 500, 502, 503 or 504, a timeout or a connection error. A 4xx is never 
resent, and a transaction refreshed after a 409 is a new call with a new key.
   - Keyed resends happen only when `idempotency-key-lifetime` parses (ISO-8601 
`PnDTnHnMnS`) and is positive. The window starts at the first attempt, and a 
resend stops when its delay would reach the lifetime. Backoff is jittered, 
honors `Retry-After` and the context, and is capped at three resends.
   - A commit (`CommitTable`, `UpdateTable`, `CommitTransaction`) that cannot 
be resent after an ambiguous failure returns `ErrCommitStateUnknown`. Other 
calls return the last error.
   - `net/http`'s own replay of a keyed request with a body, including 307/308 
redirects, is held to the same window through `GetBody`. A keyed `DELETE` has 
no body, so its transport replay cannot be bounded this way. This is documented 
and covered by a test.
   - Scan plan and task POSTs use the same code and the old retry loop is 
removed. They no longer resend on 408 or 429, now resend after a timeout or 
connection error, and send a generated key only when a lifetime is advertised. 
A caller-supplied `IdempotencyKey` is always sent; its age from the UUIDv7 
timestamp counts against the lifetime (a future timestamp is clamped), and 
without an advertised lifetime it is sent once and not resent. Scan GET polling 
is unchanged.
   - When `GetBody` refuses a replay, `Do` returns the `GetBody` error, not the 
original connection error (seen on go1.26.3 and go1.27.1).
   - `configuration.md` and the unreleased compatibility notes in `releases.md` 
describe the final behavior, replacing the "lifetime is not enforced" warning 
from #2492.
   
   ### Differences from Java
   
   - Java stops bounding resends when the lifetime is malformed (it falls back 
to no limit). Here a missing, zero, negative or malformed lifetime allows no 
keyed resends. The key is still sent whenever the lifetime is present.
   - Java also retries 408 and 429 on keyed requests; this does not, as agreed 
in #2443.
   - Go also has `net/http`'s implicit replay, which is why the window is 
applied there too, with the DELETE gap above.
   
   ## For reviewers
   
   - a. Backoff is 3 resends with 100 ms to 5 s jitter (the old scan 
constants); Java defaults to 5 retries from 1 s. Which do you want?
   - b. 408 and 429 are not resent on keyed requests, per "never a 4xx"; Java 
does resend them.
   - c. Only 500, 502, 503 and 504 count as retryable 5xx.
   - d. A resend must start before the window ends (`elapsed + wait < 
lifetime`); it can finish after.
   - e. `UpdateView` returns the last error, not `ErrCommitStateUnknown`, as it 
has no unknown-state mapping today.
   - f. A network error from the auth token fetch is also resent.
   
   ## Testing
   
   `go build ./...`, `go vet ./catalog/rest/...`, `golangci-lint run 
./catalog/rest/...` and `go test ./catalog/rest/... -race -count=1` pass. New 
`httptest` tests cover matching keys and bodies on resends (5xx, connection 
error, DELETE), fresh keys per call and after a 409, no resend on 4xx, no 
resend with a missing, zero, negative, malformed or empty lifetime, expiry 
(`Retry-After` past the window, slow first attempt), `ErrCommitStateUnknown`, 
cancellation, the resend budget, real-transport replay inside the window, after 
a slow first attempt and with `PT0S`, redirects, the keyed DELETE gap, the 
lifetime parser, and both scan POSTs including caller-key age. The tests for 
resends, transport replay bounds and scan POST changes fail on `main`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to