pitrou commented on a change in pull request #8187:
URL: https://github.com/apache/arrow/pull/8187#discussion_r494464587
##########
File path: cpp/src/arrow/filesystem/filesystem.cc
##########
@@ -487,15 +489,27 @@ Status CopyFiles(const std::shared_ptr<FileSystem>&
source_fs,
internal::ConcatAbstractPath(destination_base_dir,
relative->to_string());
if (source_info.IsDirectory()) {
- destination_dirs.insert(destination_path);
+ dirs.push_back(destination_path);
} else if (source_info.IsFile()) {
sources.push_back({source_fs, source_info.path()});
destinations.push_back({destination_fs, destination_path});
}
}
- std::vector<std::string> dirs(destination_dirs.size());
- std::move(destination_dirs.begin(), destination_dirs.end(), dirs.begin());
+ // remove directories with descendants since recursive directory creation
will create
+ // them automatically
+ std::sort(dirs.begin(), dirs.end());
+ for (auto ancestor = dirs.begin(); ancestor != dirs.end(); ++ancestor) {
+ auto descendants_end = ancestor + 1;
+
+ while (descendants_end != dirs.end() &&
+ internal::IsAncestorOf(*ancestor, *descendants_end)) {
+ ++descendants_end;
+ }
+
+ dirs.erase(ancestor, descendants_end - 1);
Review comment:
Another problem is that you're iterating on `dirs` while erasing from
it. That's not safe, I think, unless you write `ancestor = dirs.erase(ancestor,
descendants_end - 1)`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
