Although flatMap() is a valid choice, it would be more idiomatic to use filter(). I'd apply that even before running TimestampAssigner, except when extracting the timestamp is rather complicated. But if it's a simple field, then it feels better to first filter bad data, and then apply any kind of logic.
On Fri, May 29, 2020 at 1:13 PM Robert Metzger <rmetz...@apache.org> wrote: > Hi Joe, > > my gut feeling is that a flatMap() is what you are looking for. > > Best, > Robert > > On Thu, May 28, 2020 at 7:21 PM Joe Malt <jm...@yelp.com> wrote: > >> Hi, >> >> I'm working on a custom TimestampAssigner which will do different things >> depending on the value of the extracted timestamp. One of the actions I >> want to take is to drop messages entirely if their timestamp meets certain >> criteria. >> >> Of course there's no direct way to do this in the TimestampAssigner, but >> I'd like to keep this logic as close to the TimestampAssigner as possible >> since this is going to be a pluggable component used in a bunch of >> different Flink apps. >> >> What would be the best way to implement this? >> >> Thanks, >> Joe >> >> >> -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng