pitrou commented on a change in pull request #11632:
URL: https://github.com/apache/arrow/pull/11632#discussion_r744804659
##########
File path: python/pyarrow/dataset.py
##########
@@ -736,7 +736,8 @@ def _ensure_write_partitioning(part, schema, flavor):
def write_dataset(data, base_dir, basename_template=None, format=None,
partitioning=None, partitioning_flavor=None, schema=None,
filesystem=None, file_options=None, use_threads=True,
- max_partitions=None, file_visitor=None):
+ max_partitions=None, file_visitor=None,
+ existing_data_behavior='error'):
Review comment:
Unrelated to this PR, but I would expect most of the parameters to be
declared keyword-only. @jorisvandenbossche Thoughts?
##########
File path: python/pyarrow/_dataset.pyx
##########
@@ -3381,6 +3382,19 @@ def _filesystemdataset_write(
c_options.partitioning = partitioning.unwrap()
c_options.max_partitions = max_partitions
c_options.basename_template = tobytes(basename_template)
+ if existing_data_behavior == 'error':
+ c_options.existing_data_behavior = ExistingDataBehavior_ERROR
+ elif existing_data_behavior == 'overwrite_or_ignore':
+ c_options.existing_data_behavior =\
+ ExistingDataBehavior_OVERWRITE_OR_IGNORE
+ elif existing_data_behavior == 'delete_matching':
+ c_options.existing_data_behavior = ExistingDataBehavior_DELETE_MATCHING
+ else:
+ raise ValueError(
+ ('existing_data_behavior must be one of error, ',
+ 'overwrite_or_ignore or delete_matching')
Review comment:
Nit: add quotes around each possible value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
