[jira] [Created] (ARROW-6713) [Python] Getting "ArrowIOError: Corrupted file, smaller than file footer" when reading large number of parquet files through ParquetDataset()

Thu, 26 Sep 2019 11:38:07 -0700 

Harini Kannan created ARROW-6713:
------------------------------------

             Summary: [Python] Getting "ArrowIOError: Corrupted file, smaller 
than file footer" when reading large number of parquet files through 
ParquetDataset()
                 Key: ARROW-6713
                 URL: https://issues.apache.org/jira/browse/ARROW-6713
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: Harini Kannan
         Attachments: Screen Shot 2019-09-26 at 2.30.49 PM.png

When trying to read a large number of parquet files (> 600) into 
ParquetDataset(), getting the error: 

ArrowIOError: Corrupted file, smaller than file footer.

 

Note:

-This works fine for small number of (10-20) parquet files.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to