kou commented on PR #36745: URL: https://github.com/apache/arrow/pull/36745#issuecomment-1644788980
@jorisvandenbossche Do you have any idea why the following failure was happen? https://github.com/apache/arrow/actions/runs/5615711782/job/15216554355?pr=36745#step:6:6074 ```text =================================== FAILURES =================================== _______________________ test_cloudpickle_dataset[False] ________________________ tempdir = PosixPath('/tmp/pytest-of-root/pytest-0/test_cloudpickle_dataset_False0') datadir = PosixPath('/opt/conda/envs/arrow/lib/python3.8/site-packages/pyarrow/tests/data/parquet') use_legacy_dataset = False @pytest.mark.pandas @parametrize_legacy_dataset def test_cloudpickle_dataset(tempdir, datadir, use_legacy_dataset): cp = pytest.importorskip('cloudpickle') dataset = _make_dataset_for_pickling(tempdir, use_legacy_dataset) > _assert_dataset_is_picklable( dataset, pickler=cp, use_legacy_dataset=use_legacy_dataset) opt/conda/envs/arrow/lib/python3.8/site-packages/pyarrow/tests/parquet/test_dataset.py:1572: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ opt/conda/envs/arrow/lib/python3.8/site-packages/pyarrow/tests/parquet/test_dataset.py:1540: in _assert_dataset_is_picklable assert is_pickleable(dataset) opt/conda/envs/arrow/lib/python3.8/site-packages/pyarrow/tests/parquet/test_dataset.py:1538: in is_pickleable return obj == pickler.loads(pickler.dumps(obj)) opt/conda/envs/arrow/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py:73: in dumps cp.dump(obj) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <cloudpickle.cloudpickle_fast.CloudPickler object at 0x7f25aa27aa10> obj = <pyarrow.parquet.core._ParquetDatasetV2 object at 0x7f25a9ec9690> def dump(self, obj): try: > return Pickler.dump(self, obj) E _pickle.PicklingError: Can't pickle <cyfunction ParquetFragmentScanOptions._reconstruct at 0x7f2660177140>: it's not the same object as pyarrow._dataset_parquet.ParquetFragmentScanOptions._reconstruct opt/conda/envs/arrow/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py:632: PicklingError ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
