[
https://issues.apache.org/jira/browse/FLINK-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695941#comment-14695941
]
Sheetal Parade commented on FLINK-2314:
---------------------------------------
I added split information in the checkpoint state.
{code}
if(splitNumber == checkpointedSplit){
if(currRecord < checkpointedRecord) {
currRecord++;
continue;
}
}
{code}
restore state and snapshot checkpoint change accordingly.
{code}
@Override
public String snapshotState(long checkpointId, long checkpointTimestamp) throws
Exception {
return currRecord+":"+ splitNumber;
}
@Override
public void restoreState(String state){
String[] res = state.split(":");
checkpointedRecord = Long.valueOf(res[0]);
checkpointedSplit = Integer.valueOf(res[1]);
}
{code}
> Make Streaming File Sources Persistent
> --------------------------------------
>
> Key: FLINK-2314
> URL: https://issues.apache.org/jira/browse/FLINK-2314
> Project: Flink
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 0.9
> Reporter: Stephan Ewen
> Assignee: Sheetal Parade
> Labels: easyfix, starter
>
> Streaming File sources should participate in the checkpointing. They should
> track the bytes they read from the file and checkpoint it.
> One can look at the sequence generating source function for an example of a
> checkpointed source.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
