luma-shangma opened a new issue, #901: URL: https://github.com/apache/arrow-rs-object-store/issues/901
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Callers that adapt their request rate need to tell "the store asked me to slow down" apart from other failures. Today a 429, or a 503 carrying `SlowDown` / `ServerBusy`, that still fails after retries comes back as `Error::Generic`, the same as a connection reset, a DNS failure, a timeout or a plain 500. The status is held by `RetryError`, which is not reachable from outside the crate, so callers parse the `Display` text. Lance does this in its AIMD throttle ([`is_throttle_error`](https://github.com/lance-format/lance/blob/692d6f829c57415328d7d651bb2af07143c198b4/rust/lance-io/src/object_store/throttle.rs#L58-L95)). It treats any error whose text contains `retries, max_retries` as a throttle, but that text is on every error that was retried, so transport failures and ordinary 5xx responses also reduce its request rate. #618 made the 5xx body available after retries are exhausted, but only through the same private type. **Describe the solution you'd like** Following `PermissionDenied` and `Unauthenticated` (apache/arrow-rs#5345), add a variant such as `Error::Throttled { path, source }`, produced by `RetryError::error`: - 429 → `Throttled`; - 503 whose body carries a throttle code (S3 `SlowDown`, Azure `ServerBusy`) → `Throttled`; - other 503s and other 5xx → unchanged. `Error` is `#[non_exhaustive]`, so the variant is additive. The same mapping could apply to list requests alongside #851. Open questions: 1. Should a 503 without a throttle code count? A HEAD response has no body, so a HEAD 503 `SlowDown` cannot be told apart from an outage. 2. Name: `Throttled`, `TooManyRequests` or `RateLimited`? 3. Errors that are `Generic` today would change variant, and callers matching on `Generic` would silently stop seeing them. Is that acceptable in a minor release, or should it wait for the next breaking one? **Describe alternatives you've considered** - Expose the HTTP status or make `RetryError` public: declined in apache/arrow-rs#5345 because it breaks the store abstraction. - Keep parsing error text: fragile, and the only stable marker today matches every retried error. **Additional context** Happy to send a PR once the mapping is agreed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
