nealrichardson commented on a change in pull request #8187:
URL: https://github.com/apache/arrow/pull/8187#discussion_r489598925
##########
File path: cpp/src/arrow/filesystem/filesystem.cc
##########
@@ -448,6 +449,13 @@ Status CopyFiles(const std::vector<FileLocator>& sources,
destinations.size(), " paths.");
}
+ RETURN_NOT_OK(::arrow::internal::OptionalParallelFor(
+ use_threads, static_cast<int>(sources.size()), [&](int i) {
+ auto dest_dir =
internal::GetAbstractPathParent(destinations[i].path).first;
+ return dest_dir.empty() ? Status::OK()
+ :
destinations[i].filesystem->CreateDir(dest_dir);
Review comment:
`cp file.ext new/path` fails if `new/path` doesn't exist. `cp dir
newdir` fails if `dir` is a directory. But `cp -r dir newdir` works and makes
nested directories.
So maybe the idiomatic solution is either (1) add a "recursive" argument, or
(2) have a version that takes a single fs+FileSelector and a single destination
fs+path, and that does the `cp -r` case.
The use case I'm interested in supporting is "copy the contents of this S3
bucket locally" and similar things.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
