[
https://issues.apache.org/jira/browse/FLINK-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Ng updated FLINK-9295:
----------------------------------
Description:
{{FlinkKafkaProducer011}} can throw {{ProducerFencedExceptions}} if multiple
sinks are used within the same sub-task. This can happen when
operator-chaining results in two different sinks in the same topology being
chained into a task, and thus into each of its sub-tasks.
The problem is that {{TransactionIdsGenerator}} only takes into account the
task name, the subtask index, the number of subtasks, and a couple of other
things. All the attributes are the same between different
{{FlinkKafkaProducer011s}} within the same sub-task, so they get the same
transaction ids and one of them ends up failing.
was:
{{FlinkKafkaProducer011}} can throw {{ProducerFencedExceptions}} if multiple
sinks are used within the same sub-task. This can happen when
operator-chaining results in two different sinks in the same topology being
chained into a single sub-task.
The problem is that {{TransactionIdsGenerator}} only takes into account the
task name, the subtask index, the number of subtasks, and a couple of other
things. All the attributes are the same between different
{{FlinkKafkaProducer011s}} within the same sub-task, so they get the same
transaction ids and one of them ends up failing.
> FlinkKafkaProducer011 sometimes throws ProducerFencedExceptions when in
> EXACTLY_ONCE mode
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-9295
> URL: https://issues.apache.org/jira/browse/FLINK-9295
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Affects Versions: 1.4.2
> Reporter: Christopher Ng
> Priority: Major
>
> {{FlinkKafkaProducer011}} can throw {{ProducerFencedExceptions}} if multiple
> sinks are used within the same sub-task. This can happen when
> operator-chaining results in two different sinks in the same topology being
> chained into a task, and thus into each of its sub-tasks.
> The problem is that {{TransactionIdsGenerator}} only takes into account the
> task name, the subtask index, the number of subtasks, and a couple of other
> things. All the attributes are the same between different
> {{FlinkKafkaProducer011s}} within the same sub-task, so they get the same
> transaction ids and one of them ends up failing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
