Re: Multiple Partitions (Source Functions) -> Event Time -> Watermarks -> Trigger

Thanks Max - I will advance watermarks when no event arrives for a while. But when using Kafka is it a good practice to assign events to partitions randomly instead say device id or region id where the devices are located. What I noticed is if devices sending to one of the partitions stop

Re: Multiple Partitions (Source Functions) -> Event Time -> Watermarks -> Trigger

Hi Sameer, If you use Event Time you should make sure to assign Watermarks and Timestamps at the source. As you already observed, Flink may get stuck otherwise because it waits for Watermarks to progress in time. There is no timeout for windows. However, you can implement that logic in your

Re: Multiple Partitions (Source Functions) -> Event Time -> Watermarks -> Trigger

Sorry for replying to my own messages but this is super confusing and logical at the same time to me :-). If I have Kafka Topic with 10 partitions. If I partition by device id when I write to the Topic, and use Event Time, my pipeline freezes (if fewer than 10 devices are active initially).

Re: Multiple Partitions (Source Functions) -> Event Time -> Watermarks -> Trigger

And this is happening in my local environment. As soon as I set the parallelism to 1 it all works fine. Sameer On Wed, Aug 10, 2016 at 3:11 PM, Sameer W wrote: > Hi, > > I am noticing this behavior with Event Time processing- > > I have a Kafka topic with 10 partitions.

Multiple Partitions (Source Functions) -> Event Time -> Watermarks -> Trigger

Hi, I am noticing this behavior with Event Time processing- I have a Kafka topic with 10 partitions. Each Event Source sends data to any one of the partitions. Say I have only 1 event source active at this moment, which means only one partition is receiving data. None of my windows will fire