Hi, AFAIK, restoring a Flink 1.0 savepoint should not be possible on Flink 1.2. Only restoring from Flink 1.1 savepoints is supported.
@Gordon If the consumer group stays the same the new Flink job should pick up where the old one stopped, right? Best, Aljoscha > On 18. Apr 2017, at 16:19, Gwenhael Pasquiers > <gwenhael.pasqui...@ericsson.com> wrote: > > Thanks for your answer. > Does that means that flink does not rely on the offset in written to > zookeeper anymore, but relies on the snapshots data, implying that it’s > crucial to keep the same snapshot folder before and after the migration to > Flink 1.2 ? > We were also wondering if the flink consumer was able to restore it’s offset > from Zookeeper. > Another question : is there an expiration to the snapshots ? We’ve been > having issues with an app that we forgot to restart. We did it after a couple > of days, but it looks like it did not restore correctly the offset and it > started consuming from the oldest offset, creating duplicated data (the kafka > queue has over a week of buffer). > B.R. > > From: Tzu-Li (Gordon) Tai [mailto:tzuli...@apache.org] > Sent: lundi 17 avril 2017 07:40 > To: user@flink.apache.org > Subject: Re: Kafka offset commits > > Hi, > > The FlinkKafkaConsumer in 1.2 is able to restore from older version state > snapshots and bridge the migration, so there should be no problem in reading > the offsets from older state. The smallest or highest offsets will only be > used if the offset no longer exists due to Kafka data retention settings. > > Besides this, there was once fix related to the Kafka 0.8 offsets for Flink > 1.2.0 [1]. Shortly put, before the fix, the committed offsets to ZK was off > by 1 (wrt to how Kafka itself defines the committed offsets). > However, this should not affect the behavior of restoring from offsets in > savepoints, so it should be fine. > > Cheers, > Gordon > > [1] https://issues.apache.org/jira/browse/FLINK-4723 > <https://issues.apache.org/jira/browse/FLINK-4723> > > On 13 April 2017 at 10:55:40 PM, Gwenhael Pasquiers > (gwenhael.pasqui...@ericsson.com <mailto:gwenhael.pasqui...@ericsson.com>) > wrote: > > Hello, > > We’re going to migrate some applications that consume data from a Kafka 0.8 > from Flink 1.0 to Flink 1.2. > > We are wondering if the offset commitment system changed between those two > versions: is there a risk that the Flink 1.2-based application will start > with no offset (thus will use either the smallest or highest one) ? > Or can we assume that the Flink 1.2 app will resume its work at the same > offset than the Flink 1.0 app stopped (if they use the same consumer group > id) ? > > B.R.
