[
https://issues.apache.org/jira/browse/ARROW-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394118#comment-17394118
]
Joris Van den Bossche commented on ARROW-5825:
----------------------------------------------
I _suppose_ erroring on first exception is fine, as there should be no errors
at all in a normal (successful) call to this.
> [Python] Exceptions swallowed in ParquetManifest._visit_directories
> -------------------------------------------------------------------
>
> Key: ARROW-5825
> URL: https://issues.apache.org/jira/browse/ARROW-5825
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: George Sakkis
> Priority: Major
> Labels: dataset-parquet-read, parquet
>
> {{ParquetManifest._visit_directories}} uses a {{ThreadPoolExecutor}} to visit
> partitioned parquet datasets concurrently, it waits for them to finish but
> doesn't check if the respective futures have failed or not. This is quite
> tricky to detect and debug as an exception is either raised later as a a
> side-effect or (perhaps worse) it passes silently.
> Observed on 0.12.1 but appears to be on latest master too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
