Hi Chris >From the code[1], currently, IntervalJoin will ignore the late data silently, maybe you can create an issue to track this.
[1] https://github.com/apache/flink/blob/5c89d12849ea2aa332126b32808e363f12d436a2/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/co/IntervalJoinOperator.java#L222 Best, Congxian Chris Gillespie <cgilles...@zendesk.com> 于2019年12月11日周三 上午8:09写道： > Hello Flink users, first time poster here. > > I'm using an interval join in my Flink project, however I haven't found > where late records get logged in metrics. Window Joins have > "numLateRecordsDropped" implemented > <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/windowing/WindowOperator.java#L141>, > but is there an equivalent within an interval join? > > My main use case is to track how often a record falls outside of the lower > and upper bounds when trying to join two streams. Interval Join looks like > it simply short circuits > <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/co/IntervalJoinOperator.java#L222>when > there is a late record? Maybe I am not understanding what defines a late > record in this situation. > > Is there a good way to monitor when an interval join fails to join two > streams? Currently I'm looking at the delta between two operator metrics, > but it hasn't looked that reliable so far. > > Thanks, > Chris Gillespie >
