[
https://issues.apache.org/jira/browse/IMPALA-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-3900.
-----------------------------------
Resolution: Duplicate
> Add per-split runtime filtering to HdfsParquetScanner::ProcessSplit()
> ---------------------------------------------------------------------
>
> Key: IMPALA-3900
> URL: https://issues.apache.org/jira/browse/IMPALA-3900
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.6.0
> Reporter: Henry Robinson
> Assignee: Henry Robinson
> Priority: Minor
> Labels: runtime-filters
>
> If a partition filter arrives after a footer scan range for a Parquet has
> been issued, but before {{HdfsParquetScanner::ProcessSplit()}}, there's an
> opportunity to filter out all the scan ranges that would otherwise be issued
> when reading that footer, by adding a call to {{ScanRangeIsFilteredOut()}} at
> the top of that method.
> Care must be taken to ensure that all scan ranges are marked as done, since
> they won't be processed by their own scanner instances. This will avoid a
> recurrence of IMPALA-3804.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
