The behavior of the high water mark is independent of producer ack settings. It is a property of topic partitions only. (Remember that multiple producers with different configurations can write to the same topic.) The high water mark advances to the latest offset that exists in all topic partition replicas. If your topic has a replication factor of 3, then a partition’s HW will only advance toward the LEO after both replicas are in sync with the leader.
min.insync.replicas controls how many replicas must be in sync when your producer in using acks=all. This does not affect consumers. HW does affect consumers, since data between the HW and LEO cannot be consumed yet. -- Peter > On Oct 8, 2019, at 12:35 AM, Isuru Boyagane <isuruboyagane...@cse.mrt.ac.lk> > wrote: > > Hi, > > 4 > > I read about the Kafka replication protocol. I found that Kafka maintains > LEO and HW. As I understood, > > LEO: Offset of latest message a replica has seen. > > HW: Offset of the latest message which is guaranteed that each replica has > persisted into their local log. > > We can set the following acknowledgement methods in Kafka producers > configurations. > > > 1. acks = 0 (Leader replica sends an acknowledgement to the producer > once it has seen the message) > 2. acks = 1 (Leader replica sends an acknowledgement to the producer > once it has persisted the message to its local log) > 3. acks = all (Leader replica sends an acknowledgement to the > producer once every in-sync replica has persisted the message to > its local > log) > > So my question is how the leader advances the HW depending on the > acknowledgement method message producer uses. > > What I think is, > > 1. > > for acks = 0, Leader advances the HW when it sees a new message. > 2. > > for acks = 1, Leader advances the HW when it has written the new message > to its local log. > 3. > > for acks = all, Leader advances HW when each an every follower sent ack > that they persisted the message. > > Is this correct? Can anyone clarify this? > > Thanks
