[
https://issues.apache.org/jira/browse/KAFKA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson resolved KAFKA-14055.
-------------------------------------
Resolution: Fixed
> Transaction markers may be lost during cleaning if data keys conflict with
> marker keys
> --------------------------------------------------------------------------------------
>
> Key: KAFKA-14055
> URL: https://issues.apache.org/jira/browse/KAFKA-14055
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Priority: Critical
> Fix For: 3.3.0, 3.0.2, 3.1.2, 3.2.1
>
>
> We have been seeing recently hanging transactions occur on streams changelog
> topics quite frequently. After investigation, we found that the keys used in
> the changelog topic conflict with the keys used in the transaction markers
> (the schema used in control records is 4 bytes, which happens to be the same
> for the changelog topics that we investigated). When we build the offset map
> prior to cleaning, we do properly exclude the transaction marker keys, but
> the bug is the fact that we do not exclude them during the cleaning phase.
> This can result in the marker being removed from the cleaned log before the
> corresponding data is removed when there is a user record with a conflicting
> key at a higher offset. A side effect of this is a so-called "hanging"
> transaction, but the bigger problem is that we lose the atomicity of the
> transaction.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
