[
https://issues.apache.org/jira/browse/SPARK-60694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60694:
-----------------------------------
Labels: pull-request-available (was: )
> Push predicates through external lookup joins
> ---------------------------------------------
>
> Key: SPARK-60694
> URL: https://issues.apache.org/jira/browse/SPARK-60694
> Project: Spark
> Issue Type: Task
> Components: SQL, Structured Streaming
> Affects Versions: 4.4.0
> Reporter: Jungtaek Lim
> Priority: Major
> Labels: pull-request-available
>
> Part of SPARK-60031. `ExternalLookupJoin` is not a `Join`, so
> `PushPredicateThroughJoin` does not apply to it.
> In scope:
> - A standalone rule, `PushDownPredicatesThroughExternalLookupJoin`, that
> matches only `ExternalLookupJoin`.
> - It moves only deterministic conjuncts that cannot throw, following
> `PushPredicateThroughJoin`'s semantics per join type:
> - stream-side filters move below the join;
> - lookup-side conjuncts move onto the lookup side, where they reach the
> connector or are evaluated on its results.
> - Conjuncts without references stay where they are.
> - No existing join rule is changed.
> Out of scope:
> - Join reordering.
> - Constraint inference.
> - Outer-join elimination for lookup joins.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
