Thanks for the clarification, Cody! On Mon, Jul 6, 2015 at 6:44 AM, Cody Koeninger <[email protected]> wrote:
> You shouldn't rely on being able to restart from a checkpoint after > changing code, regardless of whether the change was explicitly related to > serialization. > > If you are relying on checkpoints to hold state, specifically which > offsets have been processed, that state will be lost if you can't recover > from the checkpoint. After restart the stream will start receiving > messages based on the auto.offset.reset setting, either the beginning or > the end of the kafka retention. > > To avoid this, save state in your own data store. > > On Sat, Jul 4, 2015 at 9:01 PM, Vinoth Chandar <[email protected]> wrote: > >> Hi, >> >> Just looking for some clarity on the below 1.4 documentation. >> >> >>And restarting from earlier checkpoint information of pre-upgrade code >> cannot be done. The checkpoint information essentially contains serialized >> Scala/Java/Python objects and trying to deserialize objects with new, >> modified classes may lead to errors. >> >> Does this mean, new code cannot be deployed over the same checkpoints >> even if there are not any serialization related changes? (in other words, >> if the new code does not break previous checkpoint code w.r.t >> serialization, would new deploys work?) >> >> >> >>In this case, either start the upgraded app with a different checkpoint >> directory, or delete the previous checkpoint directory. >> >> Assuming this applies to metadata & data checkpointing, does it mean that >> effectively all the computed 'state' is gone? If I am reading from Kafka, >> does the new code start receiving messages from where it left off? >> >> Thanks >> Vinoth >> > >
