Thanks Congxian, I made a JIRA to track this request. https://issues.apache.org/jira/browse/FLINK-15202
On Wed, Dec 11, 2019 at 12:56 AM Congxian Qiu <qcx978132...@gmail.com> wrote: > Hi Chris > > From the code[1], currently, IntervalJoin will ignore the late data > silently, maybe you can create an issue to track this. > > [1] > https://github.com/apache/flink/blob/5c89d12849ea2aa332126b32808e363f12d436a2/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/co/IntervalJoinOperator.java#L222 > <https://github.com/apache/flink/blob/5c89d12849ea2aa332126b32808e363f12d436a2/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/co/IntervalJoinOperator.java#L222> > Best, > Congxian > > > Chris Gillespie <cgilles...@zendesk.com> 于2019年12月11日周三 上午8:09写道： > >> Hello Flink users, first time poster here. >> >> I'm using an interval join in my Flink project, however I haven't found >> where late records get logged in metrics. Window Joins have >> "numLateRecordsDropped" implemented >> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/windowing/WindowOperator.java#L141>, >> but is >> there an equivalent within an interval join? >> >> My main use case is to track how often a record falls outside of the >> lower and upper bounds when trying to join two streams. Interval Join looks >> like it simply short circuits >> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/co/IntervalJoinOperator.java#L222>when >> there is >> a late record? Maybe I am not understanding what defines a late record in >> this situation. >> >> Is there a good way to monitor when an interval join fails to join two >> streams? Currently I'm looking at the delta between two operator metrics, >> but it hasn't looked that reliable so far. >> >> Thanks, >> Chris Gillespie >> >
