Joris Van den Bossche created ARROW-17829:
---------------------------------------------
Summary: [Python] Avoid pandas groupby deprecation warning
write_to_dataset
Key: ARROW-17829
URL: https://issues.apache.org/jira/browse/ARROW-17829
Project: Apache Arrow
Issue Type: Bug
Components: Python
Reporter: Joris Van den Bossche
I noticed the following warnings in our test builds:
{code}
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_dataset.py::test_make_fragment
/opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_dataset.py:197:
FutureWarning: In a future version of pandas, a length 1 tuple will be
returned when iterating over a groupby with a grouper equal to a list of length
1. Don't supply a list with a single grouper to avoid this warning.
for part, chunk in df_d.groupby(["color"]):
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_dataset.py::test_legacy_write_to_dataset_drops_null
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/parquet/test_pandas.py::test_write_to_dataset_pandas_preserve_extensiondtypes[True]
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/parquet/test_pandas.py::test_write_to_dataset_pandas_preserve_index[True]
/opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/parquet/core.py:3326:
FutureWarning: In a future version of pandas, a length 1 tuple will be
returned when iterating over a groupby with a grouper equal to a list of length
1. Don't supply a list with a single grouper to avoid this warning.
for keys, subgroup in data_df.groupby(partition_keys):
{code}
I suppose those are coming from pandas 1.5.0. We should investigate whether
this is something to fix in our code (or just in the tests)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
