Thanks for the clarification, Cody!

On Mon, Jul 6, 2015 at 6:44 AM, Cody Koeninger <[email protected]> wrote:

> You shouldn't rely on being able to restart from a checkpoint after
> changing code, regardless of whether the change was explicitly related to
> serialization.
>
> If you are relying on checkpoints to hold state, specifically which
> offsets have been processed, that state will be lost if you can't recover
> from the checkpoint.  After restart the stream will start receiving
> messages based on the auto.offset.reset setting, either the beginning or
> the end of the kafka retention.
>
> To avoid this, save state in your own data store.
>
> On Sat, Jul 4, 2015 at 9:01 PM, Vinoth Chandar <[email protected]> wrote:
>
>> Hi,
>>
>> Just looking for some clarity on the below 1.4 documentation.
>>
>> >>And restarting from earlier checkpoint information of pre-upgrade code
>> cannot be done. The checkpoint information essentially contains serialized
>> Scala/Java/Python objects and trying to deserialize objects with new,
>> modified classes may lead to errors.
>>
>> Does this mean, new code cannot be deployed over the same checkpoints
>> even if there are not any serialization related changes? (in other words,
>> if the new code does not break previous checkpoint code w.r.t
>> serialization, would new deploys work?)
>>
>>
>> >>In this case, either start the upgraded app with a different checkpoint
>> directory, or delete the previous checkpoint directory.
>>
>> Assuming this applies to metadata & data checkpointing, does it mean that
>> effectively all the computed 'state' is gone? If I am reading from Kafka,
>> does the new code start receiving messages from where it left off?
>>
>> Thanks
>> Vinoth
>>
>
>

Reply via email to