[
https://issues.apache.org/jira/browse/FLINK-33479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783741#comment-17783741
]
Kevin Tseng commented on FLINK-33479:
-------------------------------------
can someone assign this to me. we have idea on how this implementation can work
and can take it forward if community accept the proposal
> Add Rerouting support to KafkaSink
> ----------------------------------
>
> Key: FLINK-33479
> URL: https://issues.apache.org/jira/browse/FLINK-33479
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
> Affects Versions: 1.18.0
> Reporter: Kevin Tseng
> Priority: Minor
>
> KafkaSink currently operates in a fail / success model. If it fails it will
> throw FlinkRuntimeException, any record that may cause failure has to be
> filtered by the operators preceeding KafkaSink. If there's unexpected failure
> that was caught, this will create continuous failure until a hotfix has been
> provided and issue addressed.
> Relying on checkpoint to restore the process also create duplicated records
> produced within the interval, unless EXACTLY_ONCE is utilized.
>
> Since Flink is meant to be a real-time streaming process, any downtime can
> create detrimental effect to the downstream consumer of the data.
> I'm proposing to add an error handling injection/intercept to KafkaSink (user
> can define an error handler or route the error causing record to another
> downstream operator) to keep the process running. Only throwing
> RuntimeException when the interceptor was not provided.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
