jonded94 commented on issue #7973: URL: https://github.com/apache/arrow-rs/issues/7973#issuecomment-3490383054
Bumping this issue again, even appears with public data (https://huggingface.co/datasets/HuggingFaceTB/dclm-edu): ``` >>> ParquetFile("/ceph-hdd-fal/home/fmt/downloaded_data/HuggingFaceTB/dclm-edu/data/000_00001.parquet").read_row_group(0) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: Arrow read/write error: Parquet argument error: Parquet error: index overflow decoding byte array >>> ParquetFile("/ceph-hdd-fal/home/fmt/downloaded_data/HuggingFaceTB/dclm-edu/data/000_00002.parquet").read_row_group(0) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: Arrow read/write error: Parquet argument error: Parquet error: index overflow decoding byte array ``` (using our own PyO3 layer around `arrow-rs` here) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
