Thanks for the reply. As mentioned before the data comes from the database. Timestams are from one months ago. And I’m searching a way on how to dump this data into a working flink application which already processed this data (watermarks are far away from those dates).
On Fri 18. Jan 2019 at 03:22, Jamie Grier <jgr...@lyft.com> wrote: > I don't think I understood all of your question but with regard to the > watermarking and keys.. You are correct that watermarking (event time > advancement) is not per key. Event-time is a local property of each Task > in an executing Flink job. It has nothing to do with keys. It has only to > do with the input data timestamps seen by each task and the watermarking > function (which isn't per-key). > > I hope that helps. > > With regard to how to play historical data.. Well there are many ways to > approach that. Can you narrow down your constraints? Where does the > historical data live? > > -Jamie > > > > > On Thu, Jan 17, 2019 at 4:36 PM Kanstantsin Kamkou <kkam...@gmail.com> > wrote: > >> Hi guys! As I understood (I hope I’m wrong) the current design concept of >> the watermarking mechanism is that it tight to the latest watermark and >> there is no way to separate those watermarks by key in keyed stream (I hope >> at some point it’l be mentioned in the documentation as it unfortunately >> misleading). Could you share your thoughts on how to replay historical data >> in event–time manner (i.e. from db to working application)? The solution >> with the processing time is not suitable here as the sessions windows are >> needed. >> >> Thank you! >> > -- Best regards, Kanstantsin Kamkou email: kkam...@gmail.com web: http://2ka.by/ mobile: +49 172 5432334 skype: kkamkou
