lakshmi-manasa-g commented on a change in pull request #1450:
URL: https://github.com/apache/samza/pull/1450#discussion_r539442040
##########
File path:
samza-core/src/main/java/org/apache/samza/clustermanager/ClusterBasedJobCoordinator.java
##########
@@ -267,9 +267,12 @@ public void run() {
MetadataResourceUtil metadataResourceUtil = new
MetadataResourceUtil(jobModel, this.metrics, config);
metadataResourceUtil.createResources();
- // fan out the startpoints if startpoints is enabled and if the metadata
changed across attempts.
- // the metadata changed should be false and only get evaluated if job
coordinator high availability is enabled.
- if (new JobConfig(config).getStartpointEnabled() &&
!metadataChangedAcrossAttempts) {
+ /*
+ * We fan out startpoint if and only if
+ * 1. Startpoint is enabled in configuration
+ * 2. If AM HA is enabled, fan out only if metadata changed
Review comment:
so if metadata has changed then JC will stop all running processors
right. additionally if there are startpoints to fanout then they are assigned
to respective tasks. I am inferring that since the fanouts for task are fetched
before processor entering runloop, the prev-running-now-restarted processors
will pick them up. Am i correct? just trying to ensure we dont lose the fanouts
for tasks that are part of already running processors.
minor: if AM HA is enabled: fanout ONLY if `startpoint enabled && metadata
changed`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
