vamossagar12 commented on code in PR #12561:
URL: https://github.com/apache/kafka/pull/12561#discussion_r961591300
##########
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java:
##########
@@ -307,9 +309,31 @@ ClusterAssignment performTaskAssignment(
existing.tasks().addAll(assignment.tasks());
}
);
- canRevoke = toExplicitlyRevoke.size() == 0;
+
+ // If this round and the previous round involved revocation, we
will do an exponential
+ // backoff delay to prevent rebalance storms.
+ if (revokedInPrevious && !toExplicitlyRevoke.isEmpty()) {
+ numSuccessiveRevokingRebalances++;
+ delay = (int)
consecutiveRevokingRebalancesBackoff.backoff(numSuccessiveRevokingRebalances);
+ log.debug("Consecutive revoking rebalances observed. Need to
wait for {} ms", delay);
Review Comment:
@C0urante , the log line is definitely incorrect. Thanks for pointing it
out. I had a question on what you proposed as the better approach.
With the approach I tried to use here, I am allowing `this` revoking
rebalance to happen and set a delay so that if another worker joins within that
delay, it would need to wait for that much of time before we can perform any
revocations. This way the approach is slightly optimistic in the sense atleast
the first time around we would have a balanced allocation but further
allocation-balancing revocations won't allow a barrage of rebalances in
succession.
With what you proposed, it seems to me that even the first such worker
joining would be made to wait so that in my mind seems slightly pessimistic.
IMO since we are already setting the delay for future consecutive
allocation-balancing revocations, it might be better to allow the first one go
through. WDYT? Have I understood the context correctly or is there anything
that I am missing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
