jorisvandenbossche commented on a change in pull request #10955:
URL: https://github.com/apache/arrow/pull/10955#discussion_r718483532
##########
File path: cpp/src/arrow/dataset/file_base.h
##########
@@ -343,6 +343,18 @@ class ARROW_DS_EXPORT FileWriter {
fs::FileLocator destination_locator_;
};
+/// \brief Controls what happens if files exist in an output directory during
a dataset
+/// write
+enum ExistingDataBehavior : int8_t {
+ /// Deletes all files in a directory the first time that directory is
encountered
+ kDeleteMatchingPartitions,
+ /// Ignores existing files, overwriting any that happen to have the same
name as an
+ /// output file
+ kOverwriteOrIgnore,
+ /// Returns an error if there are any files or subdirectories in the output
directory
+ kError,
Review comment:
Ah, because `kDeleteMatchingPartitions` deletes data and doesn't abort
the operation. While for erroring in this case, you could have already written
a file to a partition that didn't exist yet, before writing a file to an
existing partition. In which case you indeed expect to roll back the part that
already was written.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
