[
https://issues.apache.org/jira/browse/KAFKA-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrian C. updated KAFKA-19126:
------------------------------
Description:
We have observed during normal operations that messages/batches can be written
to disk with the same offset or baseOffset. This happens once in a while in
multiple of our clusters (hosted in Azure and AWS) which blocks follower
replication (error in attachment), causing ISR to shrink to the leader. In
version 3.7 and above it also blocks consumers clients from reading past the
offset.
As mitigation, we deciced not to upgrade from 3.6.1 and we're temporarily
setting min.isr.replicas to 1 until the record expires from the topic.
Our clusters are using ext4 in AWS and xfs in Azure
was:
We have observed during normal operations that messages/batches can be written
to disk with the same offset or baseOffset. This happens once in a while in
multiple of our clusters (hosted in Azure and AWS) which blocks follower
replication (error in attachment), causing ISR to shrink to the leader. In
version 3.7 and above it also blocks consumers clients from reading past the
offset.
As mitigation, we deciced not to upgrade from 3.6.1 and we're temporarily
setting min.isr.replicas to 1 until the record expires from the topic.
> Duplicate offsets written to log for different messages
> -------------------------------------------------------
>
> Key: KAFKA-19126
> URL: https://issues.apache.org/jira/browse/KAFKA-19126
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 3.6.1, 3.8.0, 3.7.1
> Reporter: Adrian C.
> Priority: Major
> Attachments: follower-error.log, log-dump-sample.txt
>
>
> We have observed during normal operations that messages/batches can be
> written to disk with the same offset or baseOffset. This happens once in a
> while in multiple of our clusters (hosted in Azure and AWS) which blocks
> follower replication (error in attachment), causing ISR to shrink to the
> leader. In version 3.7 and above it also blocks consumers clients from
> reading past the offset.
> As mitigation, we deciced not to upgrade from 3.6.1 and we're temporarily
> setting min.isr.replicas to 1 until the record expires from the topic.
> Our clusters are using ext4 in AWS and xfs in Azure
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
