[
https://issues.apache.org/jira/browse/SPARK-5296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14320417#comment-14320417
]
Corey J. Nolet edited comment on SPARK-5296 at 2/13/15 5:33 PM:
----------------------------------------------------------------
bq. I would only pass down ORs though, as we already split up ANDs and
filter/pass them down already implicitly.
I could understand wanting to treat the Array as a conjunction but how would
you support arbitrarily nested AND and OR filters if you don't have AND as part
of the tree? This is really what interests me about the CatalystScan- I can
unroll the parts of the tree that i am able to support and those can be
arbitrarily nested.
was (Author: sonixbp):
bq. I would only pass down ORs though, as we already split up ANDs and
filter/pass them down already implicitly.
I could understand wanting to treat the Array as a conjunction but how would
you support arbitrarily nested AND and OR filters if you don't have AND as part
of the tree?
> Predicate Pushdown (BaseRelation) to have an interface that will accept OR
> filters
> ----------------------------------------------------------------------------------
>
> Key: SPARK-5296
> URL: https://issues.apache.org/jira/browse/SPARK-5296
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Corey J. Nolet
> Assignee: Cheng Lian
> Priority: Critical
>
> Currently, the BaseRelation API allows a FilteredRelation to handle an
> Array[Filter] which represents filter expressions that are applied as an AND
> operator.
> We should support OR operations in a BaseRelation as well. I'm not sure what
> this would look like in terms of API changes, but it almost seems like a
> FilteredUnionedScan BaseRelation (the name stinks but you get the idea) would
> be useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
