Hi Gustavo, the idea and approach LGTM. +1 to proceed.
Best, Arvid On Thu, Apr 24, 2025 at 4:58 PM Gustavo de Morais <gustavopg...@gmail.com> wrote: > Hi everyone, > > I'd like to propose FLIP-516: Multi-Way Join Operator [1] for discussion. > > Chained non-temporal joins in Flink SQL often cause a "big state issue" due > to large intermediate results, impacting performance and stability. This > FLIP introduces a StreamingMultiJoinOperator to tackle this by joining > multiple inputs (that need to share a common key) simultaneously within one > operator. > > The main goal is achieving zero intermediate state for these common join > patterns, significantly reducing state size. This initial version requires > a common partitioning key and focuses on INNER/LEFT joins, with plans for > future expansion. The operator is opt-in via > table.optimizer.multi-join.enabled (default false). PR with the initial > version of the operator is available [2]. > > Happy to be contributing to this community, and looking forward to your > feedback and thoughts. > > Kind regards, > Gustavo de Morais > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-516%3A+Multi-Way+Join+Operator > > [2] https://github.com/apache/flink/pull/26313 >
