Re: Handle late message with flink SQL

2021-03-17 Thread Yi Tang
Thanks Timo. The whole idea is also based on the side output and output tag. Let me explain it in detail: 1. Introduce a VirtualTableScan(or SideOutputTableScan), which can be optimized as Physical RelNode. Then we can create a source catalog table which will be converted to a VirtualTableScan, a

Re: Handle late message with flink SQL

2021-03-16 Thread Timo Walther
Hi, your explanation makes sense but I'm wondering how the implementation would look like. This would mean bigger changes in a Flink fork, right? Late data handling in SQL is a frequently asked question. Currently, we don't have a good way of supporting it. Usually, we recommend to use DataS