Re: Kafka control source in addition to Kafka data source

2017-07-20 Thread Gabriele Di Bernardo
Hi Konstantin, Thank you so much for your answer. Yes, I think this is exactly what I need. Thank you. Best, Gabriele > On 18 Jul 2017, at 21:27, Konstantin Knauf > wrote: > > Hi Gabriele, > > I think this is actually a quite common pattern. Generally, you can > `join` the two streams a

Re: Kafka control source in addition to Kafka data source

2017-07-18 Thread Konstantin Knauf
Hi Gabriele, I think this is actually a quite common pattern. Generally, you can `join` the two streams and then use a `CoFlatMapFunction`. A `CoFlatMapFunction` allows you to keep shared (checkpointed) state between two streams. It has two callbacks `flatMap1` and `flatMap2` which are called when

Kafka control source in addition to Kafka data source

2017-07-18 Thread Gabriele Di Bernardo
Hello everyone, I am a Flink newcomer and I would like to implement a Flink application with two Kafka sources: one for the data stream to be processed and the other one for control purposes. The application should be able to read from the control stream and then apply the control operation to