mapleFU commented on code in PR #41564:
URL: https://github.com/apache/arrow/pull/41564#discussion_r1599921602
##########
cpp/src/arrow/filesystem/s3fs.cc:
##########
@@ -1545,6 +1559,13 @@ class ObjectInputFile final : public
io::RandomAccessFile {
// (for rational, see: https://github.com/apache/arrow/issues/34363)
static constexpr int64_t kPartUploadSize = 10 * 1024 * 1024;
+// Above this threshold, use a multi-part upload instead of a single request
upload.
+static constexpr int64_t kMultiPartUploadThresholdSize = 5 * 1024 * 1024;
Review Comment:
Should we config hard limit like this, or Switch to multipart when there're
multi parts? (size > 10MiB during append)
##########
cpp/src/arrow/filesystem/s3fs_test.cc:
##########
@@ -521,9 +521,6 @@ class TestS3FS : public S3TestMixin {
void TestOpenOutputStream() {
std::shared_ptr<io::OutputStream> stream;
- // Nonexistent
- ASSERT_RAISES(IOError,
fs_->OpenOutputStream("nonexistent-bucket/somefile"));
Review Comment:
Why this is being removed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
