[
https://issues.apache.org/jira/browse/FLINK-24207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474629#comment-17474629
]
Roman Khachatryan commented on FLINK-24207:
-------------------------------------------
Thanks for the clarification [~zlzhang0122] .
To me, the approach you described looks more like write-ahead-log rather than
2PC. In TwoPhaseCommitSinkFunction terms, external system is a participant;
while in WAL it's the sink responsibility. So maybe you should take a look at
GenericWriteAheadSink instead?
Another option to consider is the [new sink
API|https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API],
which allows to distribute committables IIUC.
> Add support of KeyedState in TwoPhaseCommitSinkFunction
> -------------------------------------------------------
>
> Key: FLINK-24207
> URL: https://issues.apache.org/jira/browse/FLINK-24207
> Project: Flink
> Issue Type: New Feature
> Components: API / DataStream
> Affects Versions: 1.12.2, 1.13.1
> Reporter: zlzhang0122
> Priority: Major
>
> Now, the implementation of TwoPhaseCommitSinkFunction is based on operator
> state, but operator state will do a deep copy when taking checkpoint, so
> large operator state may produce a OOM error. Add support of KeyedState in
> TwoPhaseCommitSinkFunction maybe a good choice to avoid the OOM error and
> give users more convenience.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
