thisisnic commented on code in PR #47599:
URL: https://github.com/apache/arrow/pull/47599#discussion_r2425590842
##########
cpp/src/arrow/dataset/dataset_writer.cc:
##########
@@ -383,8 +383,11 @@ class DatasetWriterDirectoryQueue
}
init_future_ = Future<>::Make();
auto create_dir_cb = [this] {
- return
DeferNotOk(write_options_.filesystem->io_context().executor()->Submit(
- [this]() { return write_options_.filesystem->CreateDir(directory_);
}));
+ return DeferNotOk(
+ write_options_.filesystem->io_context().executor()->Submit([this]() {
+ return write_options_.filesystem->CreateDir(directory_,
+
write_options_.create_dir);
Review Comment:
This looks like it's passing in the value of `create_dir` as the `recursive`
argument. I'm wondering if it might make more sense to expose
`check_directory_existence_before_creation` in R instead of updating the
dataset writer?
https://github.com/apache/arrow/blob/b8eed8c9666c1280f92053291c61c59dd7ed19b7/cpp/src/arrow/filesystem/s3fs.h#L173-L180
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
