Re: Superstep-like synchronization of streaming iteration

2018-10-01 Thread Paris Carbone
Hi Christian, It is great to see use iterative use cases, thanks for sharing your problem! Superstep iterative BSP synchronization for streams is a problem we have been looking into extensively, however this functionality is still not standardised yet on Flink. I think your use case is fully co

Superstep-like synchronization of streaming iteration

2018-09-29 Thread Christian Lehner
Hi all, if you don't want to read the wall of text below, in short, I want to know if it is possible to get a superstep-based iteration on a possibly unbounded DataStream in Flink in an efficient way and what general concept(s) of synchronization you would suggest for that. I would like to