[
https://issues.apache.org/jira/browse/FLINK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983153#comment-15983153
]
Aljoscha Krettek commented on FLINK-6383:
-----------------------------------------
I had a quick offline chat with [~plucas] about this and it seemed like now
would be a good time to add this. We could also introduce the whole scheme the
first time that we want to break the scheme, it will work because of the magic
number.
[~StephanEwen], [~srichter] what do you think?
> Add versioning for AbstractStreamOperator state
> -----------------------------------------------
>
> Key: FLINK-6383
> URL: https://issues.apache.org/jira/browse/FLINK-6383
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
> Affects Versions: 1.3.0
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
> Priority: Blocker
> Fix For: 1.3.0
>
>
> Right now, {{AbstractStreamOperator}} checkpoints key-group state in
> {{AbstractStreamOperator.snapshotState(StateSnapshotContext context)}}.
> Currently, this is the timers for each key group. We might want to change the
> format of this or what we store in the future. In order to facilitate this we
> should write a version to the beginning of the stream so that we can deal
> with schema updates.
> For dealing with the fact that we don't currently write a version number we
> have to at a magic number add the beginning (think {{0xCAFEBABE}}). For
> dealing with state that was written prior to versioning we read the first 8
> bytes, buffer them, if they match the magic number we read the version and
> from then on jump to the logic based on the version. If the magic number
> doesn't match we know we have state from a pre-version version and read the
> stream using the old code, including the 8 buffered bytes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
