[
https://issues.apache.org/jira/browse/FLINK-19799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221291#comment-17221291
]
Stephan Ewen commented on FLINK-19799:
--------------------------------------
Thanks, Steven. [~lzljs3620320] I think you mentioned something similar in a
separate discussion.
Is it fair to say that on the FileSourceEnumerator, the checkpoint state is in
general a combination of the enumerator's state and the assigner's state? So if
we generify it like that, would that solve issues?
> Make FileSource extensible
> --------------------------
>
> Key: FLINK-19799
> URL: https://issues.apache.org/jira/browse/FLINK-19799
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Priority: Major
> Fix For: 1.12.0
>
>
> The File System Source currently assumes all formats can represent their work
> units as {{FileSourceSplit}}. If that is not the case, the formats cannot be
> implemented using the {{FileSource}}.
> We need to support extending the splits to carry additional information in
> the splits, and to use that information when creating bulk readers and
> handling split state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
