@huxihx There are currently three cases where MM does offset commit. These are 1) regular offset commit, 2) onPartitionsRevoked() and 3) MM shutdown.
It seems that user probably also want MM to try move if offset commit timeout during MM shutdown, since otherwise it just means data duplication which is not good. And it may be reasonable asking user to explicitly kill -9 MM if user decides that offset commit won't happen and they would want to tolerate message duplication. The motivation for infinite offset commit retries for regularly offset commit is less strong because if a regular offset commit timed out, it does not directly mean message duplication. On the other hand it does increase the impact of message duplication if user finally decides the offset commit can not pass due to constant backlog in the broker side. I am inclined to also keep retries in this case. The cons is that MM may stuck on offset commit forever if broker's request queue time is constantly large. But the resulting degraded performance seems reasonable in the event that broker's performance has degraded. On the other hand message duplication is about correctness and it may cause problem for downstream application if MM accidentally duplicates hours worth of data just due to e.g. misconfiguration or broker performance issue. In this case correctness seems more important than performance. [ Full content available at: https://github.com/apache/kafka/pull/5492 ] This message was relayed via gitbox.apache.org for [email protected]
