[
https://issues.apache.org/jira/browse/FLINK-21364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501576#comment-17501576
]
Steven Zhen Wu commented on FLINK-21364:
----------------------------------------
actually, I think we should look into FLIP-182 FLINK-18450 first and see if
Iceberg source can use the same mechanism to watermark alignment.
> piggyback finishedSplitIds in RequestSplitEvent
> -----------------------------------------------
>
> Key: FLINK-21364
> URL: https://issues.apache.org/jira/browse/FLINK-21364
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Common
> Affects Versions: 1.12.1
> Reporter: Steven Zhen Wu
> Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available
>
> For some split assignment strategy, the enumerator/assigner needs to track
> the completed splits to advance watermark for event time alignment or rough
> ordering. Right now, `RequestSplitEvent` for FLIP-27 source doesn't support
> pass-along of the `finishedSplitIds` info and hence we have to create our own
> custom source event type for Iceberg source.
> Here is the proposal of add such optional info to `RequestSplitEvent`.
> {code}
> public RequestSplitEvent(
> @Nullable String hostName,
> @Nullable Collection<String> finishedSplitIds)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
