shanthoosh commented on a change in pull request #1484:
URL: https://github.com/apache/samza/pull/1484#discussion_r606419141
##########
File path: samza-core/src/main/java/org/apache/samza/zk/ZkJobCoordinator.java
##########
@@ -274,6 +283,20 @@ void doOnProcessorChange() {
LOG.info("Generating new JobModel with processors: {}.",
currentProcessorIds);
JobModel newJobModel = generateNewJobModel(processorNodes);
+ /*
+ * Leader skips the rebalance even if there are changes in the quorum as
long as the work assignment remains the same
+ * across all the processors. The optimization is useful in the following
scenarios
+ * 1. The processor in the quorum restarts within the debounce window.
Originally, this would trigger rebalance
+ * across the processors stopping and starting their work assignment
which is detrimental to availability of
+ * the system. e.g. common scenario during rolling upgrades
+ * 2. Processors in the quorum which don't have work assignment and
their failures/restarts don't impact the
+ * quorum.
+ */
+ if (newJobModel.equals(activeJobModel)) {
Review comment:
This check would include both the config and work-assignments for
verifying equality between two different JobModels.
Standalone allows users to launch two processors with different configs in a
quorum. Would be better to narrow down this check to work-assignment alone to
achieve the desired effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
