Re: Reset kafka offets to latest on restart

2018-11-24 Thread Vishal Santoshi
I think I can set . a new uuid but it seems `allowNonRestoreState` is a CLI hint. I need the "automatic" restart on failure to use the new uuid. Our use case has no use of data on Kafka that is not current. On Thu, Nov 22, 2018 at 11:16 PM Tony Wei wrote: > Hi Vishal, > > AFAIK, the current

Counter Implementation in Flink

2018-11-24 Thread Anil
I'm using Flink 1.4.2 and deploying job on Yarn Cluster. I have a streaming job, which flattens the data and outputs it. It basically takes a input record and produces n output record. I'm using Table Function for this. The logic to flatten the data is implemented in a UDF. The UDF has a counter

Joining more than 2 streams

2018-11-24 Thread Gagan Agrawal
Hi, I want to do window join on multiple Kafka streams (say a, b, c) on common field in all 3 streams and apply some custom function on joined stream. As I understand we can join only 2 streams at a time via DataStream api. So may be I need to join a and b first and then join first joined stream