[
https://issues.apache.org/jira/browse/CRUNCH-299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830054#comment-13830054
]
Josh Wills commented on CRUNCH-299:
-----------------------------------
So we could either a) add an option to give a ColumnRecordFilter to the Parquet
Source, or b) create a subclass of FilterFn that knows how to express
ColumnRecordFilters.
Right now, the planner doesn't generally do anything special to handle the
children of a DoFn, with the exception of CombineFns around GBK operations. If
we go with b), we'd need to add some planner functionality to support it.
> Support predicate pushdown for Parquet sources
> ----------------------------------------------
>
> Key: CRUNCH-299
> URL: https://issues.apache.org/jira/browse/CRUNCH-299
> Project: Crunch
> Issue Type: Improvement
> Components: Core
> Reporter: Tom White
> Assignee: Josh Wills
>
> We should be able to push Crunch FilterFn down to a Parquet
> ColumnRecordFilter.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
