Hi all, Following my last question on soft (no failure to cluster..) exactly once semantics during a rolling upgrade for a consumer group on kubernetes.
Maybe the required behavior is already guaranteed.., because with a grace period configured, Kubernetes will send a SIGTERM signal to the consumer for graceful shutdown, Next the consumer will send a leave group request to the group coordinator, where subsequently as part of the rebalancing protocol ConsumerRebalanceListener#onPartitionRevoked (where the lastest processed offset is committed), which will gracefully revoke the current partitions out of the current consumer, and assign them to different consumer.... so already soft (no cluster failure) excatly once is possibly guaranteed? Can someone confirm this behavior and/or provide a more accurate description of the behavior of a graceful shutdown of a consumer within a consumer group ? Thank you so much. ________________________________ From: Mazen Ezzeddine <mazen.ezzedd...@etu.univ-cotedazur.fr> Sent: Thursday, July 22, 2021 10:10 AM To: users@kafka.apache.org <users@kafka.apache.org> Subject: soft excatly once rolling upgrade. Dear all, I am interested in achieving zero down time upgrade of my kafka consumer group running on Kubernetes with say soft exactly once semantics (i.e., assume no failure and/or error is going to happen to my cluster). I configured my consumer group with rolling update and terminationGracePeriodSeconds, I also added a shutdown hook during which I am committing the last processed record by the consumer to be killed… would that guarantee soft exactly once rolling upgrade (i.e., under the assumption of no cluster failure…) Thank you.
