Hi everyone, I'd like to start a discussion on FLIP-579: LATERAL SNAPSHOT Join [1].
Enriching a stream with data from a (slowly changing) dynamic table is a super common use case. Flink SQL features Temporal Joins [2] to address these use cases. However, SQL users can only use the event-time variant which has many limitations (heavy dependency on frequent WM updates on both inputs, build-side table requires a PK, the join predicate must include the build-side PK, etc). The processing-time temporal join is disabled (due to build-side initialization issues [3]) and temporal table function joins are only available in Table API. FLIP-579 proposes a new temporal join operator that operates in processing-time and addresses the limitations of the existing implementations: * initialization of the build-side before joining * no requirement of continuous, frequent build-side WMs (after the initialization completed) * no requirement for a PK on the build-side * table function-based syntax [4] via a built-in SNAPSHOT function (proposed in FLIP-517 [4]) Looking forward to your feedback. Best, Fabian [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-579%3A+LATERAL+SNAPSHOT+Join [2] https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/joins/#temporal-joins [3] https://issues.apache.org/jira/browse/FLINK-19830 [4] https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/joins/#temporal-table-function-join [5] https://cwiki.apache.org/confluence/display/FLINK/FLIP-517%3A+Better+Handling+of+Dynamic+Table+Primitives+with+PTFs#FLIP517:BetterHandlingofDynamicTablePrimitiveswithPTFs-SNAPSHOTfortemporaljoins
