[
https://issues.apache.org/jira/browse/FLINK-39340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger updated FLINK-39340:
-----------------------------------
Component/s: Table SQL / Planner
> Transform binary multi join back to regular join
> ------------------------------------------------
>
> Key: FLINK-39340
> URL: https://issues.apache.org/jira/browse/FLINK-39340
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Dmitriy Linevich
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2026-03-27-13-33-32-140.png
>
>
> It has been observed that multi-join optimization leads to performance
> regressions when the multi join has only two inputs.
> !image-2026-03-27-13-33-32-140.png!
> *Cluster information (through docker):*
> * Job manager has 1 cpu and 4gb memory
> * Task manager has 2 cpu and 4gb memory
> *Flink Configuration:*
> {code:java}
> jobmanager.rpc.address: jobmanager
> parallelism.default: 1
> pipeline.max-parallelism: 24
> taskmanager.memory.process.size: 3900m
> taskmanager.numberOfTaskSlots: 1
> process.working-dir: /data/flink-tmp
> execution.checkpointing.dir: file:///data/flink-checkpoints
> execution.checkpointing.incremental: true
> execution.checkpointing.interval: 3min
> execution.checkpointing.mode: EXACTLY_ONCE
> execution.checkpointing.unaligned.enabled: true
> execution.checkpointing.local-backup.enabled: true
> execution.checkpointing.checkpoints-after-tasks-finish.enabled: false
> execution.checkpointing.tolerable-failed-checkpoints: 20
> state.backend.type: rocksdb
> table.exec.mini-batch.enabled: true
> table.exec.mini-batch.allow-latency: 2s
> table.exec.mini-batch.size: 50000
> table.exec.async-state.enabled: false
> table.optimizer.multi-join.enabled: false
> pipeline.object-reuse: true {code}
> *Nexmark config:*
> {code:java}
> nexmark.workload.suite.100m.events.num: 100000000
> nexmark.workload.suite.100m.tps: 1000000
> nexmark.workload.suite.100m.queries:
> "q0,q1,q2,q3,q4,q5,q7,q8,q9,q10,q11,q12,q13,q14,q15,q16,q17,q18,q19,q20,q21,q22,q23"
> nexmark.workload.suite.100m.queries.cep: "q0,q1,q2,q3"
> nexmark.workload.suite.100m.warmup.duration: 0s
> nexmark.workload.suite.100m.warmup.events.num: 0
> nexmark.workload.suite.100m.warmup.tps: 0
> nexmark.metric.monitor.delay: 10s {code}
>
> *As a result:*
> * Q4 - 17% regression
> * Q9 - 17% regression
> * mini-batch optimization for regular join, but not for multi join
> * window mofication for regular join, but not for multi join (Q5)
>
> *Suggestions:*
> # Add rule (BinaryMultiJoinToJoinRule) after multi join, that will transform
> multi join with 2 inputs back to regular join
> # Modify JoinToMultiJoinRule, it shouldn't to generate MultiJoin with 2
> inputs
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
