[
https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086984#comment-14086984
]
Jay Kreps commented on KAFKA-1555:
----------------------------------
Hey [~gwenshap], to answer your question, the behavior if you specify acks=5
with only 3 replicas would be an immediate error about insufficient replicas
(same as what we do today if you do this).
Just for completeness, another variation instead of specifying the min.isr in
the request would be to return to the client the actual ISR size in the
response. The min.isr is in some sense a threshold applied to the isr size on
the server so we might as well just have the server say what happened and let
the client interpret this appropriately.
Gwen/Jun here are the pros and cons I see for doing a topic config:
Pros:
1. A bit simpler to implement.
2. Since the min isr is bounded by the replication factor if this is a topic
config we can make it impossible to specify an invalid min isr and we can
automatically lower the min isr if you change the replication factor to be
lower than the min isr.
Cons:
1. I think it is a little odd that acks would be specified at the request level
and min.isr at the topic level since both are about when a write is considered
"safe".
2. Operationally, I think it is more convenient for the client to be able to
control this in their code rather than having to do it globally.
3. You can imagine rare cases where you have two writers which actually have
different durability requirements on the same topic.
Another question? What happens if the ISR is less than min.isr? Do we do the
write but return an error or do we refuse to do the write? Of course we can't
be sure to refuse the write because we can do the write on the leader and then
have a replica fail and the ISR shrink at the same time. But, when possible, do
we refuse to write on the leader if the isr is less than min.isr?
> provide strong consistency with reasonable availability
> -------------------------------------------------------
>
> Key: KAFKA-1555
> URL: https://issues.apache.org/jira/browse/KAFKA-1555
> Project: Kafka
> Issue Type: Improvement
> Components: controller
> Affects Versions: 0.8.1.1
> Reporter: Jiang Wu
> Assignee: Neha Narkhede
>
> In a mission critical application, we expect a kafka cluster with 3 brokers
> can satisfy two requirements:
> 1. When 1 broker is down, no message loss or service blocking happens.
> 2. In worse cases such as two brokers are down, service can be blocked, but
> no message loss happens.
> We found that current kafka versoin (0.8.1.1) cannot achieve the requirements
> due to its three behaviors:
> 1. when choosing a new leader from 2 followers in ISR, the one with less
> messages may be chosen as the leader.
> 2. even when replica.lag.max.messages=0, a follower can stay in ISR when it
> has less messages than the leader.
> 3. ISR can contains only 1 broker, therefore acknowledged messages may be
> stored in only 1 broker.
> The following is an analytical proof.
> We consider a cluster with 3 brokers and a topic with 3 replicas, and assume
> that at the beginning, all 3 replicas, leader A, followers B and C, are in
> sync, i.e., they have the same messages and are all in ISR.
> According to the value of request.required.acks (acks for short), there are
> the following cases.
> 1. acks=0, 1, 3. Obviously these settings do not satisfy the requirement.
> 2. acks=2. Producer sends a message m. It's acknowledged by A and B. At this
> time, although C hasn't received m, C is still in ISR. If A is killed, C can
> be elected as the new leader, and consumers will miss m.
> 3. acks=-1. B and C restart and are removed from ISR. Producer sends a
> message m to A, and receives an acknowledgement. Disk failure happens in A
> before B and C replicate m. Message m is lost.
> In summary, any existing configuration cannot satisfy the requirements.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
