Hello Community,

I noticed Kafka allows the replication factor of a topic and
"min.insync.replicas" to be set such that "min.insync.replicas" > the
topic's replication factor.

As a consequences, the JMX beans
kafka.cluster:type=Partition,name=UnderReplicated and
kafka.cluster:type=Partition,name=UnderMinIsr
can report diverging views on the replication for a topic. The former can
report no under replicated partition, while the second will report under
in-sync replicas.

Even worse, consumption of topics which exhibit this behaviour seems to
fail, the Kafka broker throwing a NotEnoughReplicasException.

[2019-07-22 10:44:29,913] ERROR [ReplicaManager broker=0] Error processing
> append operation on partition __consumer_offsets-0
> (kafka.server.ReplicaManager)
> org.apache.kafka.common.errors.NotEnoughReplicasException: The size of the
> current ISR Set(0) is insufficient to satisfy the min.isr requirement of 2
> for partition __consumer_offsets-0


In order to avoid this scenario, one possibility would be to check the
values of "min.insync.replicas" and "default.replication.factor" when the
broker starts, and "min.insync.replicas" and the replication factor given
to a topic at creation time, and refuses to create the topic if those are
inconsistently set.

This was reproduced with Kafka 2.1.0, 2.2.0 and 2.3.0.

What do you think?

Alexandre

Reply via email to