Re: How to rewind Kafka cursors into a Flink job ?

2017-03-30 Thread Tzu-Li (Gordon) Tai
Hi Dominique, What your plan A is suggesting is that a downstream operator can provide signals to upstream operators and alter their behaviour. In general, this isn’t possible, as in a distributed streaming environment it’s hard to guarantee what records exactly will be altered by the

How to rewind Kafka cursors into a Flink job ?

2017-03-27 Thread Dominique De Vito
Hi, Is there a way to rewind Kafka cursors (while using here Kafka as a consumer) from (inside) a Flink job ? Use case [plan A] * The Flink job would listen 1 main "data" topic + 1 secondary "event" topic * In case of a given event, the Flink job would rewind all Kafka cursors of the "data"