jorisvandenbossche commented on code in PR #12838:
URL: https://github.com/apache/arrow/pull/12838#discussion_r855156736
##########
python/pyarrow/parquet/__init__.py:
##########
@@ -3116,6 +3136,14 @@ def file_visitor(written_file):
"'basename_template' parameter instead. For "
"usage see `pyarrow.dataset.write_dataset`"),
FutureWarning, stacklevel=2)
+ if existing_data_behavior is not None:
Review Comment:
```suggestion
if existing_data_behavior != "overwrite_or_ignore":
```
? (since that is the default value. Or otherwise we need to change the
default value to None and turn that into "overwrite_or_ignore" in the
use_legacy_dataset=False code path)
##########
python/pyarrow/parquet/__init__.py:
##########
@@ -3116,6 +3136,14 @@ def file_visitor(written_file):
"'basename_template' parameter instead. For "
"usage see `pyarrow.dataset.write_dataset`"),
FutureWarning, stacklevel=2)
+ if existing_data_behavior is not None:
+ warnings.warn(
+ _DEPR_MSG.format("existing_data_behavior", " Specify "
+ "'use_legacy_dataset=False' while constructing "
+ "the ParquetDataset, and then use the "
+ "'basename_template' parameter instead. For "
Review Comment:
this last part is not relevant here I think (copy paste from
partition_filename_cb?)
##########
python/pyarrow/parquet/__init__.py:
##########
@@ -3116,6 +3136,14 @@ def file_visitor(written_file):
"'basename_template' parameter instead. For "
"usage see `pyarrow.dataset.write_dataset`"),
FutureWarning, stacklevel=2)
+ if existing_data_behavior is not None:
+ warnings.warn(
+ _DEPR_MSG.format("existing_data_behavior", " Specify "
+ "'use_legacy_dataset=False' while constructing "
+ "the ParquetDataset, and then use the "
+ "'basename_template' parameter instead. For "
Review Comment:
This could also be an error like the schema/partitioning/use_threads
handling above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
