Re: Purging Late stream data

2017-07-26 Thread G.S.Vijay Raajaa
Sure, Let me try that out. On the same note, does BoundedOutOfOrdernessTimestampExtractor Serve the purpose too? Regards, Vijay Raajaa GS On Wed, Jul 26, 2017 at 9:22 AM, Kien Truong wrote: > Hi, > > One method you can use is using a ProcessFunction. > > In the process function, you get the ti

Re: Purging Late stream data

2017-07-25 Thread Kien Truong
Hi, One method you can use is using a ProcessFunction. In the process function, you get the timer service through the function context, which can then be used to schedule a task to clean up late data. Check out the docs for ProcessFunction https://ci.apache.org/projects/flink/flink-docs-rel

Purging Late stream data

2017-07-25 Thread G.S.Vijay Raajaa
Hi, I am having 3 streams which is being merged from a union of kafka topics on a given timestamp. The problem I am facing is that, if there is a delay in one of the stream and when the data in that particular stream arrives at a later point in time, the merge happens in a delayed fashion. The wa