Makes sense. Thanks Jiayi!! Regards, Jayant
On Wed, Nov 28, 2018 at 3:32 PM bupt_ljy <bupt_...@163.com> wrote: > Hi, > > It will only affect the states. Because Flink will serialize these states > when doing snapshot, therefore, we need to make sure that the states can be > deserialized if the program is restarted. As for the stateless operators, > it’s not affected because Flink won’t do a snapshot on them. > > Best, > Jiayi Liao > Original Message > *Sender:* Jayant Ameta<wittyam...@gmail.com> > *Recipient:* bupt_ljy<bupt_...@163.com> > *Cc:* user<user@flink.apache.org> > *Date:* Wednesday, Nov 28, 2018 17:50 > *Subject:* Re: Flink operator UUID and serialVersionUID > > Thanks, I'll look into the bravo project. > > Will it just impact the MapState or all the operators? > If I have a map operator which converts DataStream<MyObject> to > DataStream<Tuple2<String, MyObject>>. Will this fail to recover as well if > a field is added to MyObject? > Jayant Ameta > > > On Wed, Nov 28, 2018 at 3:08 PM bupt_ljy <bupt_...@163.com> wrote: > >> Hi, >> >> It’ll fail because Flink can’t successfully deserialize the data in >> savepoint into the new “MyObject” class. There is no official way to fix >> this problem. However, you can take a look at the bravo project >> https://github.com/king/bravo, which can help to reconstruct the >> savepoint, but only with the RocksDBStateBackend now. >> >> Best, >> Jiayi Liao >> >> Original Message >> *Sender:* Jayant Ameta<wittyam...@gmail.com> >> *Recipient:* bupt_ljy<bupt_...@163.com> >> *Cc:* user<user@flink.apache.org> >> *Date:* Wednesday, Nov 28, 2018 17:14 >> *Subject:* Re: Flink operator UUID and serialVersionUID >> >> If I upgrade my flink job, and add a field in "MyObject" class. Will the >> restore fail? >> If so, how to handle such scenarios? Should I convert the "MyObject" >> instance in json and store the string? >> >> Jayant Ameta >> >> >> On Wed, Nov 28, 2018 at 1:26 PM bupt_ljy <bupt_...@163.com> wrote: >> >>> Hi Jayant, >>> >>> If you change the “MyObject” class and influence the serialized >>> value, then the “MyObject” instance can’t be deserialized when restoring, >>> which causes a failure of restoring. You can just use the default >>> serialVersionUID instead defining it explicitly(it makes no difference if >>> you do it). >>> >>> >>> Best, >>> >>> Jiayi Liao >>> >>> Original Message >>> *Sender:* Jayant Ameta<wittyam...@gmail.com> >>> *Recipient:* bupt_ljy<bupt_...@163.com> >>> *Cc:* user<user@flink.apache.org> >>> *Date:* Wednesday, Nov 28, 2018 15:46 >>> *Subject:* Re: Flink operator UUID and serialVersionUID >>> >>> Thanks for clarifying Jiayi. >>> If there is a change in "MyObject" class, would it help to have a >>> serialVersionUID defined? >>> >>> Thanks, >>> Jayant >>> >>> >>> On Wed, Nov 28, 2018 at 12:52 PM bupt_ljy <bupt_...@163.com> wrote: >>> >>>> Hi, Jayant >>>> >>>> 1. The uuid is an unique identifier for a specific operator, which >>>> means that Flink uses the uuid to recognize the operator when restoring. >>>> >>>> 2. The operator has already implemented the Serializable interface >>>> so you don’t need to do it explicitly. >>>> >>>> 3. The type information of “MyObject” should be defined in the >>>> MapStateDescriptor, so no need to worry about it. >>>> >>>> >>>> Best, >>>> >>>> Jiayi Liao >>>> >>>> Original Message >>>> *Sender:* Jayant Ameta<wittyam...@gmail.com> >>>> *Recipient:* user<user@flink.apache.org> >>>> *Date:* Wednesday, Nov 28, 2018 15:09 >>>> *Subject:* Flink operator UUID and serialVersionUID >>>> >>>> Hi all, I've a few questions regarding serial version: >>>> >>>> 1. The production ready checklist mentions using uuids >>>> <https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/production_ready.html#set-uuids-for-operators> >>>> for >>>> operators. How is it different from setting a serialVersionUID on an >>>> operator? >>>> >>>> 2. Which operators need to have a serialVersionUID present (or >>>> implement Serializable interface)? >>>> >>>> 3. If I have a MapState<String, MyObject>, does MyObject need to have a >>>> serialVersionUID and does it need to implement Serializable interface? >>>> >>>> Thanks, >>>> Jayant >>>> >>>
