No. This is the thread that answers my question - http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Does-FlinkKafkaConsumer010-care-about-consumer-group-td14323.html
Moiz — sent from phone On 19-Jul-2017, at 10:04 PM, Ted Yu <yuzhih...@gmail.com> wrote: This was the other thread, right ? http://search-hadoop.com/m/Flink/VkLeQ0dXIf1SkHpY?subj=Re+Does+job+restart+resume+from+last+known+internal+checkpoint+ > On Wed, Jul 19, 2017 at 9:02 AM, Moiz Jinia <moiz.ji...@gmail.com> wrote: > Yup! Thanks. > > Moiz > > — > sent from phone > > On 19-Jul-2017, at 9:21 PM, Aljoscha Krettek [via Apache Flink User Mailing > List archive.] <[hidden email]> wrote: > > This was now answered in your other Thread, right? > > Best, > Aljoscha > >>> On 18. Jul 2017, at 11:37, Moiz Jinia <[hidden email]> wrote: >>> >>> Aljoscha Krettek wrote >>> Hi, >>> zero-downtime updates are currently not supported. What is supported in >>> Flink right now is a savepoint-shutdown-restore cycle. With this, you >>> first >>> draw a savepoint (which is essentially a checkpoint with some meta data), >>> then you cancel your job, then you do whatever you need to do (update >>> machines, update Flink, update Job) and restore from the savepoint. >>> >>> A possible solution for zero-downtime update would be to do a savepoint, >>> then start a second Flink job from that savepoint, then shutdown the first >>> job. With this, your data sinks would need to be able to handle being >>> written to by 2 jobs at the same time, i.e. writes should probably be >>> idempotent. >>> >>> This is the link to the savepoint doc: >>> https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/savepoints.html >>> >>> Does that help? >>> >>> Cheers, >>> Aljoscha >>> >>> On Fri, 16 Dec 2016 at 18:16 Andrew Hoblitzell < >> >>> ahoblitzell@ >> >>> > >>> wrote: >>> >>>> Hi. Does Apache Flink currently have support for zero down time or the = >>>> ability to do rolling upgrades? >>>> >>>> If so, what are concerns to watch for and what best practices might = >>>> exist? Are there version management and data inconsistency issues to = >>>> watch for?= >>>> >> >> When a second job instance is started in parallel from a savepoint, my >> incoming kafka messages would get sharded between the 2 running instances of >> the job (since they both would belong to the same consumer group). So when I >> stop the older version of the job, i stand to lose data (inspite of the fact >> that my downstream consumer is idempotent) >> >> If I used a different consumer group for the new job version (and start it >> from a savepoint), will the savepoint ensure that the second job instance >> starts from the correct offset? Do I need to do anything extra to make this >> work? (example set the uid on the source of the job). >> >> Thanks! >> Moiz >> >> >> >> -- >> View this message in context: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-rolling-upgrade-support-tp10674p14313.html >> Sent from the Apache Flink User Mailing List archive. mailing list archive >> at Nabble.com. > > > > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-rolling-upgrade-support-tp10674p14337.html > To unsubscribe from Flink rolling upgrade support, click here. > NAML > > View this message in context: Re: Flink rolling upgrade support > > Sent from the Apache Flink User Mailing List archive. mailing list archive at > Nabble.com.
