[
https://issues.apache.org/jira/browse/FLINK-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954207#comment-16954207
]
lichong commented on FLINK-12574:
---------------------------------
[~yitz589] I think it's all right for the flink design about this problem, but
I want to know how do you resolve this when you resume your application or
reprocess old data from MQ such as kafka. In this case we need to consume the
data in kafka from a give offset instead of the offset in the checkpoint or
savepoint.
Any reply will be appreciated.
Thx.
> using sink StreamingFileSink files are overwritten when resuming application
> causing data loss
> ----------------------------------------------------------------------------------------------
>
> Key: FLINK-12574
> URL: https://issues.apache.org/jira/browse/FLINK-12574
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Affects Versions: 1.8.0
> Reporter: yitzchak lieberman
> Priority: Critical
>
> when part files are saved to s3 bucket (with bucket assigner) with simple
> names such as:
> part-0-0 and part-1-2
> restarting or resuming application causes checkpoint id to start from 0 and
> old files will be replaced by new part files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
