[GitHub] [kafka] aiquestion commented on a diff in pull request #12349: KAFKA-14024: Consumer keeps Commit offset in onJoinPrepare in Cooperative rebalance

[GitBox]

aiquestion commented on code in PR #12349:
URL: https://github.com/apache/kafka/pull/12349#discussion_r908395237


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java:
##########
@@ -745,6 +745,10 @@ protected boolean onJoinPrepare(int generation, String 
memberId) {
         boolean onJoinPrepareAsyncCommitCompleted = false;
         // async commit offsets prior to rebalance if auto-commit enabled
         RequestFuture<Void> future = maybeAutoCommitOffsetsAsync();
+        // wait for commit offset response if future exist
+        if (future != null) {
+            client.poll(future, 
time.timer(rebalanceConfig.rebalanceTimeoutMs));
+        }

Review Comment:
   @RivenSun2  Ah Yes, you are right.
   i think in previous issue https://issues.apache.org/jira/browse/KAFKA-13310, 
the problem is that `maybeAutoCommitOffsetsSync` falls into an infinite loop if 
meet `UNKNOWN_TOPIC_OR_PARTITION` error.
   But i think we need to wait for the commitOffset response and try to commit 
offset successfully before joinGroup, to  make 'no duplicate message' if every 
thing works well.
   
   Your suggestion can work, but it will cause KafkaConsuemr.poll() to be in a 
busy loop between the time when commitOffsetRequest is in flight.
   
   Do you think we can do:
   * still wait for the response here
   * limit the max retry time of commitOffset in onJoinPrepare function
   
   Thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org