You shouldn't rely on being able to restart from a checkpoint after changing code, regardless of whether the change was explicitly related to serialization.
If you are relying on checkpoints to hold state, specifically which offsets have been processed, that state will be lost if you can't recover from the checkpoint. After restart the stream will start receiving messages based on the auto.offset.reset setting, either the beginning or the end of the kafka retention. To avoid this, save state in your own data store. On Sat, Jul 4, 2015 at 9:01 PM, Vinoth Chandar <[email protected]> wrote: > Hi, > > Just looking for some clarity on the below 1.4 documentation. > > >>And restarting from earlier checkpoint information of pre-upgrade code > cannot be done. The checkpoint information essentially contains serialized > Scala/Java/Python objects and trying to deserialize objects with new, > modified classes may lead to errors. > > Does this mean, new code cannot be deployed over the same checkpoints even > if there are not any serialization related changes? (in other words, if the > new code does not break previous checkpoint code w.r.t serialization, would > new deploys work?) > > > >>In this case, either start the upgraded app with a different checkpoint > directory, or delete the previous checkpoint directory. > > Assuming this applies to metadata & data checkpointing, does it mean that > effectively all the computed 'state' is gone? If I am reading from Kafka, > does the new code start receiving messages from where it left off? > > Thanks > Vinoth >
