[
https://issues.apache.org/jira/browse/IMPALA-10098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183434#comment-17183434
]
Tim Armstrong commented on IMPALA-10098:
----------------------------------------
[~superdupershant] do we have an idea of what cardinality these might need to
support? Up to a certain cardinality you could push down a full list of values.
> Runtime Filters for Set Exclusion or Compliment
> -----------------------------------------------
>
> Key: IMPALA-10098
> URL: https://issues.apache.org/jira/browse/IMPALA-10098
> Project: IMPALA
> Issue Type: New Feature
> Reporter: Shant Hovsepian
> Priority: Major
> Labels: runtime-filters
>
> It would be beneficial to extend runtime filters to push set exclusion down
> to scan nodes. This would be used to optimize NOT IN, EXCEPT style queries or
> more generally ANTI JOINS, as well as OUTER JOINs which filter out non null
> attributes from the nullable side.
> This is almost the inverse operation of a traditional bloom filter, other
> data structures might be more efficient.
> This would also compliment Impala's left deep pipelined query planning very
> well for what otherwise would require complex query plans due to reordering
> restrictions with ANTI/OUTER joins.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
