FYI: After an offline discussion with Feng Jin(CC‘ed & Thanks), I have added some additional candidate designs along with a comparative table of advantages and disadvantages to facilitate better evaluation and discussion of the various candidate designs.
Looking forward to any comments. Best regards, Yuepeng Pan Yuepeng Pan <[email protected]> 于2025年12月16日周二 12:02写道： > Thanks Mang for the review. > > Since the names of the subtitles are somewhat ambiguous and may lead to > misunderstandings about not preserving the original interface, > I have made some corresponding change[1]. > > Hope this helps. Please take a look if you had the free time. > Thank you. > > [1] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425181#FLIP339:SupportAdaptivePartitionSelectionforStreamPartitioner-IntroducethenewmethodsatAPIlevel > : > > Best regards, > Yuepeng Pan > > Mang Zhang <[email protected]> 于2025年12月16日周二 10:53写道： > >> hi Yuepeng >> Thank you for continuing to drive this FLIP forward. >> Regarding the changes to the DataStream API in FLIP, I haven't observed >> any forward compatibility with the original API. Could you explain the >> rationale behind this design choice? >> Forward-compatible APIs reduce the cost for users to upgrade Flink >> versions. >> >> >> >> >> >> -- >> >> Best regards, >> Mang Zhang >> >> >> >> At 2025-12-15 12:40:00, "Pan Yuepeng" <[email protected]> wrote: >> >Hi devs, >> > >> >I re-sorted out and supplemented the 'FLIP-339[1] Support Adaptive >> >Partition Selection for StreamPartitioner' based on Flink JIRA[2]. >> > >> >Flink offers multiple partition strategies, some of which bind data to >> >downstream subtasks, while others do not (e.g., shuffle, rescale, >> >rebalance). >> >For [Data not bound to subtasks] scenarios, overloaded sub-task-nodes may >> >slow down the processing of Flink jobs, leading to backpressure and data >> >lag. Dynamically adjusting the partition of data to subtasks based on the >> >processing load of downstream operators helps achieve a peak-shaving and >> >valley-filling effect, thereby striving to maintain the throughput of >> Flink >> >jobs. >> > >> >The raw discussions could be found in the Flink JIRA[2]. >> >I really appreciate developers involved in the discussion for the >> valuable >> >help and suggestions in advance. >> > >> >Please refer to the FLIP[1] wiki for more details about the proposed >> design >> >and implementation. >> > >> >Welcome any feedback and opinions on this proposal. >> > >> >[1] https://cwiki.apache.org/confluence/x/nYyzDw >> >[2] https://issues.apache.org/jira/browse/FLINK-31655 >> > >> >Best regards, >> >Yuepeng Pan >> >
