Thanks for driving this! +1 for the FLIP given there is a solid user case behind.
Shuai and I had a discussion and we agree that `KeyedAsyncWaitOperator` in current FLIP shares similar functionality with the `Asynchronous Execution Model (AEC)` introduced in FLIP-425[1]. We think it is better to generalize the AEC for all keyed ordered cases, not only for state access. So I'd make this happen after the approval of this FLIP. Hope this helps all the similar operators to implement. [1] https://cwiki.apache.org/confluence/x/S4p3EQ Best, Zakelly On Tue, Apr 8, 2025 at 10:00 AM shuai xu <xushuai...@gmail.com> wrote: > Hi devs, > > I'd like to start a discussion on FLIP-519: Introduce async lookup key > ordered mode[1]. > > The Flink system currently supports both record-level ordered and > unordered output modes for asynchronous lookup joins. However, it does > not guarantee the processing order of records sharing the same key. > > As highlighted in [2], there are two key requirements for enhancing > async io operations: > 1. Ensuring the processing order of records with the same key is a > common requirement in DataStream. > 2. Sequential processing of records sharing the same upsertKey when > performing lookup join in Flink SQL is essential for maintaining > correctness. > > This optimization aims to balance correctness and performance for > stateful streaming workloads.Then the FLIP introduce a new operator > KeyedAsyncWaitOperator to supports the optimization. Besides, a new > option is added to control the behaviour avoid influencing existing > jobs. > > please find more details in the FLIP wiki document[1]. Looking forward > to your feedback. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-519%3A++Introduce+async+lookup+key+ordered+mode > [2] https://lists.apache.org/thread/wczzjhw8g0jcbs8lw2jhtrkw858cmx5n > > Best, > Xu Shuai >
