[
https://issues.apache.org/jira/browse/FLINK-24139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413870#comment-17413870
]
godfrey he commented on FLINK-24139:
------------------------------------
[~trushev] Thanks for reporting this improvement
> Push down more predicates through Join in stream mode
> -----------------------------------------------------
>
> Key: FLINK-24139
> URL: https://issues.apache.org/jira/browse/FLINK-24139
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Alexander Trushev
> Priority: Minor
> Attachments: q13_after.json, q13_after.png, q13_after.txt,
> q13_before.json, q13_before.png, q13_before.txt
>
>
> h3. Context
> Rule {{JoinDependentConditionDerivationRule}} is introduced in FLINK-12509.
> This rule rewrites join condition in such way that more predicates can be
> pushed down through join. For example,
> # Source A = [a0, a1, a2], source B = [b0, b1]
> # {code:sql}select * from A join B on a0 = b0 where (a1 = 0 and b1 = 0) or
> a2 = 0{code}
> # {{JoinDependentConditionDerivationRule}} transforms condition ((a1 and b1)
> or a2) to (((a1 and b1) or a2) and (a1 or a2))
> # {{JoinConditionPushRule}} pushes (a1 or a2) to A source
> It is a good optimization that can lead to performance improvement of query
> execution.
> Currently, {{JoinDependentConditionDerivationRule}} is used only in batch
> mode.
> h3. Proposal
> Enable {{JoinDependentConditionDerivationRule}} in stream mode.
> h3. Benefit
> Experiment based on [https://github.com/ververica/flink-sql-benchmark]
> Cluster – 4 nodes each 2 slots
> Dataset – tpcds_bin_orc_20
> Before – 1.14.0-rc0
> After – 1.14.0-rc0 + patched {{FlinkStreamProgram}} including
> {{JoinDependentConditionDerivationRule}}
> ||TPC-DS 20 GB||Before||After||
> |q13 stream mode|83 s|8 s|
> Query plan, stream graph, dashboard visualization before and after the patch
> are in the attachment
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
