wjones127 opened a new pull request, #4305:
URL: https://github.com/apache/arrow-rs/pull/4305
# Which issue does this PR close?
Closes #4304.
# Rationale for this change
Previously, when a call to `poll_write()` was made, we would write the full
buffer provided unless it was too small (when combined with previous values).
So if 100MB of data was provided, we would write a part that was 100MB.
This meant in part that part sizes could be uneven. This is fine with S3 and
Minio, but it rejected by R2. In R2, once you upload your first part, you are
committing to only uploading parts of exactly equal size (except for the final
part).
TBH, I think it's probably an improvement we don't blindly write the full
buffer in one go. This should lead to more predictable performance.
# What changes are included in this PR?
The behavior change is described above.
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking
change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
