mapleFU commented on code in PR #41564:
URL: https://github.com/apache/arrow/pull/41564#discussion_r1600112929
##########
cpp/src/arrow/filesystem/s3fs.cc:
##########
@@ -1545,6 +1559,13 @@ class ObjectInputFile final : public
io::RandomAccessFile {
// (for rational, see: https://github.com/apache/arrow/issues/34363)
static constexpr int64_t kPartUploadSize = 10 * 1024 * 1024;
+// Above this threshold, use a multi-part upload instead of a single request
upload.
+static constexpr int64_t kMultiPartUploadThresholdSize = 5 * 1024 * 1024;
Review Comment:
Both is ok for me, here S3OutputStream might manage these part of memory in
memory buffer, so I think it shouldn't be too large? changing it default 10MB
or making it tunable later is both ok for me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
