Hi Dong and Xuannan, I'm excited to see this FLIP. I think support for processing-time temporal joins is something that the Flink users will greatly benefit off. I specifically want to call-out that it's great to see the use cases that this enables. From a technical implementation perspective, I defer to the opinion of others with expertise on this topic.
Best regards, Martijn On Sun, Jun 25, 2023 at 9:03 AM Xuannan Su <suxuanna...@gmail.com> wrote: > > Hi all, > > Dong(cc'ed) and I are opening this thread to discuss our proposal to > enhance the watermark to properly support processing-time temporal > join, which has been documented in FLIP-326 [1]. > > We want to support the use case where the records from the probe side > of the processing-time temporal join need to wait until the build side > finishes the snapshot phrase by enhancing the expressiveness of the > Watermark. Additionally, these changes lay the groundwork for > simplifying the DataStream APIs, eliminating the need for users to > explicitly differentiate between event-time and processing-time, > resulting in a more intuitive user experience. > > Please refer to the FLIP document for more details about the proposed > design and implementation. We welcome any feedback and opinions on > this proposal. > > Best regards, > > Dong and Xuannan > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-326%3A+Enhance+Watermark+to+Support+Processing-Time+Temporal+Join
