Steven Phillips created HADOOP-11751:
----------------------------------------
Summary: Parquet scan fails when directory contains _SUCCESS or
_logs
Key: HADOOP-11751
URL: https://issues.apache.org/jira/browse/HADOOP-11751
Project: Hadoop Common
Issue Type: Bug
Reporter: Steven Phillips
Assignee: Steven Phillips
_SUCCESS and _logs are often created from map reduce jobs, and typically
ignored. This is commonly done using the OutputFilesFilter in hadoop.
The new FooterGatherer class, which is used to read parquet footers in
parallel, does not use this filter. So it attempts to read these as parquet
files and fails.
The fix is to use the DrillPathFilter, which extends OutputFilesFilter, in the
FooterGather code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
