kszlim commented on code in PR #5383:
URL: https://github.com/apache/arrow-rs/pull/5383#discussion_r1486906379
##########
object_store/src/client/retry.rs:
##########
@@ -263,7 +262,7 @@ impl RetryExt for reqwest::RequestBuilder {
do_retry = true
} else if let Some(source) = e.source() {
if let Some(e) =
source.downcast_ref::<hyper::Error>() {
- if e.is_connect() || e.is_closed() ||
e.is_incomplete_message() {
+ if !(e.is_parse() || e.is_parse_status() ||
e.is_parse_too_large() || e.is_user() || e.is_canceled()) {
Review Comment:
I'm curious why you think it's a rate limit issue, I thought s3 should 503
on rate limit as documented here?
https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
Based off:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance-design-patterns.html#optimizing-performance-timeouts-retries
(Though this might also imply that object_store should have a configurable
retry strategy on a per cloud provider basis?)
It seems like these errors are probably caused by an underlying issue in
object_store/hyper/reqwest/polars usage of object_store?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
