Barre commented on PR #445: URL: https://github.com/apache/arrow-rs-object-store/pull/445#issuecomment-3135816157
> object_store will retry PUT requests provided they don't have preconditions - https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/mod.rs#L182. My suspicion is slatedb is specifying a precondition and therefore the retry machinery doesn't retry - https://github.com/apache/arrow-rs-object-store/blob/main/src/client/retry.rs#L439. > > _Whilst I am aware that technically a put with preconditions is idempotent as idempotency refers to the server-side state not the client-side result, in practice few clients would handle it correctly if we retried such requests._ > > > Perhaps they shouldn't and just implement their own retry policies? > > Slatedb should likely not be solely relying on object_store retries, there will almost certainly be error scenarios where recovery would require retrying at a higher level. > > > elapsed: 30.000417786s > > As an aside, this would suggest that slatedb is using PUT for very large payloads where it should possibly be using a multipart upload instead. Thank you. On my end, I patched `object_store` with https://github.com/Barre/arrow-rs-object-store/commit/077d0ff07ee5fb4c2dedcdeb8069ad6a28778761 which seems to work great with SlateDB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
