Hi All, We are implementing a use case that needs tight at least once delivery. Even in the case of failures of nodes, no messages must be lost.
I am trying to find out the least restrictive configurations that can give us at least once delivery. What I found was, - By setting the number of acknowledgments (acks) in Kafka producer configuration <https://kafka.apache.org/documentation/#producerconfigs>, we can configure Kafka for our need for data safety and availability. - If I use acks = 0, we can't guarantee that messages will not be lost in a leader failover. - If I use acks=1, we can't guarantee that messages will not be lost in a leader failover. - If I use acks= all, we will have a good data safety but in case of a leader failover where follower which was not in the ISR becomes the next leader, may lead the Kafka to message loss. As per my understanding, setting acks=all in Kafka producer configuration and setting unclean.leader.election.enable = false (to avoid followers which are not in ISR gets elected as the leader) in Kafka topic configurations will give us data safety so that no message will be lost (sacrificing the availability of the system). Is this the best option to get at least once ( zero message loss) even in the presence of failures? Thanks
