[
https://issues.apache.org/jira/browse/FLINK-19799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221298#comment-17221298
]
Jingsong Lee commented on FLINK-19799:
--------------------------------------
> the checkpoint state is in general a combination of the enumerator's state
>and the assigner's state
I think yes, it can solve the issue, now for FileSource, it is
"{{Collection<FileSourceSplit> splits"}} and "{{Collection<Path>
alreadyProcessedPaths"}}
> Make FileSource extensible
> --------------------------
>
> Key: FLINK-19799
> URL: https://issues.apache.org/jira/browse/FLINK-19799
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Priority: Major
> Fix For: 1.12.0
>
>
> The File System Source currently assumes all formats can represent their work
> units as {{FileSourceSplit}}. If that is not the case, the formats cannot be
> implemented using the {{FileSource}}.
> We need to support extending the splits to carry additional information in
> the splits, and to use that information when creating bulk readers and
> handling split state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
