As far as I learned from folks with better understanding than myself , barrier alignment might be only path to get deterministic output.
Any state or outcome between barrier alignments requires second thought(like UDP packages from network). Currently, alignment is used only do heavyweight checkpointing. If folks decided to improve algorithm and use in other ways like auto scaling or secondary task shadowing is still TBD. Chen > On Jul 24, 2018, at 18:57, vino yang <yanghua1...@gmail.com> wrote: > > Hi jiaxl, > > The paper you mentioned was published at 2017. I think it doesn't have much > reference value now. > Over time, both frameworks are constantly evolving. > At the end of May this year, Flink has supported the major feature of local > recovery in the latest release of version 1.5. > This greatly improves the speed of recovery. > Flink has not stopped the improvement of state recovery and fault > tolerance. > I think you can verify it yourself. > > Thanks, vino. > > > 2018-07-24 23:15 GMT+08:00 jiaxl <jthunder1...@hotmail.com>: > >> From conclusion of this paper https://dl.acm.org/citation.cfm?id=3132750 >> <https://dl.acm.org/citation.cfm?id=3132750http://> , Flink's recovery >> speed is slower than that of Spark Streaming, which will be a problem in >> large scale deployment where fault happens frequently. >> I'd like to know whether this is still a problem or not. Any advices are >> appreciated. >> >> >> >> -- >> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ >>
