jorisvandenbossche opened a new issue, #34404: URL: https://github.com/apache/arrow/issues/34404
We have several failing tests in the nightly build (https://github.com/ursacomputing/crossbow/actions/runs/4277727973/jobs/7446784501) because of a change in pandas 2.0: the Index can now store all numeric dtypes, and not just int64/uint64/float64, see https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html#index-can-now-hold-numpy-numeric-dtypes. Failing tests because of this: ``` FAILED python/pyarrow/tests/test_pandas.py::test_table_from_pandas_schema_index_columns - AssertionError: DataFrame.index are different FAILED python/pyarrow/tests/parquet/test_dataset.py::test_read_partitioned_directory[False] - AssertionError: Attributes of DataFrame.iloc[:, 2] (column name="foo") are different FAILED python/pyarrow/tests/parquet/test_dataset.py::test_read_partitioned_directory_s3fs[False] - AssertionError: Attributes of DataFrame.iloc[:, 2] (column name="foo") are different ``` I think all those cases are where now an int32 dtype is preserved, while before it would have been cast to int64 by pandas. But the expected result still uses int64, causing the test failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
