rtyler opened a new pull request, #513:
URL: https://github.com/apache/arrow-rs-object-store/pull/513
This error in the logic of the backoff configuration meant that in
practice when an object store (e.g. S3) would produce an error, the
steps of the backoff _may_ be incredibly small and not exponentially
backoff properly.
Before this change, a few runs of the test case I added had backoff
steps far exceeding 20 steps for the default configuration, e.g.:
```
left: 20
right: 70
left: 20
right: 194
left: 20
right: 31
left: 20
right: 407
left: 20
right: 407
```
Due to these steps, a rate-limiting or other error by an object store
would result in an _explosion_ in repeated requests to the object store.
In the case of a rate-limiting situation this further exacerbated the
failure mode.
A rather simple fix however. I did also remove some logic from the
existing test because I do not believe repeating the logic of the
`next()` function in the test is useful from a testing perspective.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
