Hi Peter, Okay. Then, whatever the producer acks configuration is, Leader replica waits for *all* other follower replicas to persist the message before advancing HW(without considering the *current ISR* set)?
Thanks On Tue, 8 Oct 2019 at 14:54, Peter Bukowinski <pmb...@gmail.com> wrote: > The behavior of the high water mark is independent of producer ack > settings. It is a property of topic partitions only. (Remember that > multiple producers with different configurations can write to the same > topic.) The high water mark advances to the latest offset that exists in > all topic partition replicas. If your topic has a replication factor of 3, > then a partition’s HW will only advance toward the LEO after both replicas > are in sync with the leader. > > min.insync.replicas controls how many replicas must be in sync when your > producer in using acks=all. This does not affect consumers. > > HW does affect consumers, since data between the HW and LEO cannot be > consumed yet. > > -- > Peter > > > On Oct 8, 2019, at 12:35 AM, Isuru Boyagane < > isuruboyagane...@cse.mrt.ac.lk> wrote: > > > > Hi, > > > > 4 > > > > I read about the Kafka replication protocol. I found that Kafka maintains > > LEO and HW. As I understood, > > > > LEO: Offset of latest message a replica has seen. > > > > HW: Offset of the latest message which is guaranteed that each replica > has > > persisted into their local log. > > > > We can set the following acknowledgement methods in Kafka producers > > configurations. > > > > > > 1. acks = 0 (Leader replica sends an acknowledgement to the producer > > once it has seen the message) > > 2. acks = 1 (Leader replica sends an acknowledgement to the producer > > once it has persisted the message to its local log) > > 3. acks = all (Leader replica sends an acknowledgement to the > > producer once every in-sync replica has persisted the message to > > its local > > log) > > > > So my question is how the leader advances the HW depending on the > > acknowledgement method message producer uses. > > > > What I think is, > > > > 1. > > > > for acks = 0, Leader advances the HW when it sees a new message. > > 2. > > > > for acks = 1, Leader advances the HW when it has written the new > message > > to its local log. > > 3. > > > > for acks = all, Leader advances HW when each an every follower sent ack > > that they persisted the message. > > > > Is this correct? Can anyone clarify this? > > > > Thanks > -- isuru
