[
https://issues.apache.org/jira/browse/FLINK-23839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410670#comment-17410670
]
Arvid Heise commented on FLINK-23839:
-------------------------------------
Actually the description of the error in the PR made it very clear, so thank
you for updating it!
> Unclear severity of Kafka transaction recommit warning in logs
> --------------------------------------------------------------
>
> Key: FLINK-23839
> URL: https://issues.apache.org/jira/browse/FLINK-23839
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Kafka
> Affects Versions: 1.11.4, 1.12.5, 1.13.2
> Reporter: Nico Kruber
> Assignee: David Morávek
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> In a transactional Kafka sink, after recovery, all transactions from the
> recovered checkpoint are recommitted even though they may have already been
> committed before because this is not part of the checkpoint.
> This second commit can lead to a number of WARN entries in the logs coming
> from
> [KafkaCommitter|https://github.com/apache/flink/blob/6c9818323b41a84137c52822d2993df788dbc9bb/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/sink/KafkaCommitter.java#L66]
> or
> [FlinkKafkaProducer|https://github.com/apache/flink/blob/6c9818323b41a84137c52822d2993df788dbc9bb/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer.java#L1057].
> Examples:
> {code}
> WARN [org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer] ...
> Encountered error org.apache.kafka.common.errors.InvalidTxnStateException:
> The producer attempted a transactional operation in an invalid state. while
> recovering transaction KafkaTransactionState [transactionalId=...,
> producerId=12345, epoch=123]. Presumably this transaction has been already
> committed before.
> WARN [org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer] ...
> Encountered error org.apache.kafka.common.errors.ProducerFencedException:
> Producer attempted an operation with an old epoch. Either there is a newer
> producer with the same transactionalId, or the producer's transaction has
> been expired by the broker. while recovering transaction
> KafkaTransactionState [transactionalId=..., producerId=12345, epoch=12345].
> Presumably this transaction has been already committed before
> {code}
> It sounds to me like the second exception is useful and indicates that the
> transaction timeout is too short. The first exception, however, seems
> superfluous and rather alerts the user more than it helps. Or what would you
> do with it?
> Can we instead filter out superfluous exceptions and at least put these onto
> DEBUG logs instead?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
