Caleb Overman created ARROW-8245:
------------------------------------
Summary: [Python] Skip hidden directories when reading partitioned
parquet files
Key: ARROW-8245
URL: https://issues.apache.org/jira/browse/ARROW-8245
Project: Apache Arrow
Issue Type: Bug
Components: Python
Reporter: Caleb Overman
When writing a partitioned parquet file Spark can create a temporary hidden
`.spark-staging` directory within the parquet file. Because it is a directory
and not a file, it is not skipped when trying to read the parquet file. Pyarrow
currently only skips directories prefixed with `_`.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
