crepererum commented on issue #7171: URL: https://github.com/apache/arrow-rs/issues/7171#issuecomment-2678157929
> Could you expand on what you mean by chunking? If you have a large object (like 100MB), instead of using a single GET request for the whole range, using multiple requests in -- let's say -- 16MB chunks. Our internal testing showed that while the throughput and hence latency of a single S3 GET request is somewhat limited, you can reduce the overall latency and increase the throughput using that method. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
