mynameborat commented on a change in pull request #1450:
URL: https://github.com/apache/samza/pull/1450#discussion_r538943207
##########
File path:
samza-core/src/main/java/org/apache/samza/clustermanager/ClusterBasedJobCoordinator.java
##########
@@ -260,8 +267,9 @@ public void run() {
MetadataResourceUtil metadataResourceUtil = new
MetadataResourceUtil(jobModel, this.metrics, config);
metadataResourceUtil.createResources();
- // fan out the startpoints if startpoints is enabled
- if (new JobConfig(config).getStartpointEnabled()) {
+ // fan out the startpoints if startpoints is enabled and if the metadata
changed across attempts.
+ // the metadata changed should be false and only get evaluated if job
coordinator high availability is enabled.
+ if (new JobConfig(config).getStartpointEnabled() &&
!metadataChangedAcrossAttempts) {
Review comment:
yeah. We will not fanout startpoint since there may be containers that
are already running in which case the newly instantiated containers will act on
startpoint but the old ones wont. As a result, we skip fanout and applications
will force job restart through new deployment like how they do it today.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
