Re: Event time window questions

2018-01-24 Thread Sendoh
setAutoWatermarkInterval configures how often the watermark is produced. so if watermark is not proceeding, if you set shorter interval, you would see t1, t1, t1, t1, t1 more often. But what you would like to see is t1, t2, t3, t4 If you want to see count 0 when there is no incoming events,0

Re: Event time window questions

2018-01-23 Thread Navneeth Krishnan
Thanks Sendoh. Is there a way to advance watermark even when there are no incoming events. What exactly does setAutoWatermarkInterval do? Also I don't see the watermark displayed in flink dashboard. Will the watermark advance only when there is data from all consuming kafka topic and partitions?

Re: Event time window questions

2018-01-23 Thread Sendoh
Hi, you can write your own trigger and window, and implement whatever logic there. There are some examples https://github.com/apache/flink/blob/1875cac03042dad4a4c47b0de8364f02fbe457c6/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers/ If you don't see any

Event time window questions

2018-01-21 Thread Navneeth Krishnan
Hi, I am having few issues with event time windowing. Here is my scenario, data is ingested from a kafka consumer and then keyed by user followed by a Tumbling event window for 10 seconds. The max lag tolerance limit is 1 second. I have the BoundedOutOfOrdernessGenerator that extends