alirezazamani commented on a change in pull request #1508:
URL: https://github.com/apache/helix/pull/1508#discussion_r516829905
##########
File path:
helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -116,33 +116,6 @@ public synchronized boolean refresh(HelixDataAccessor
accessor,
}
}
- // The following 3 blocks is for finding a list of workflows whose JobDAGs
have been changed
- // because their RuntimeJobDags would need to be re-built
- // newly added jobs
- for (String jobName : newJobConfigs.keySet()) {
- if (!_jobConfigMap.containsKey(jobName) &&
newJobConfigs.get(jobName).getWorkflow() != null) {
- workflowsUpdated.add(newJobConfigs.get(jobName).getWorkflow());
- }
-
- // Only for JobQueues when a new job is enqueued, there exists a race
condition where only
- // JobConfig is updated and the RuntimeJobDag does not get updated
because when the client
- // (TaskDriver) submits, it creates JobConfig ZNode first and modifies
its parent JobDag next.
- // To ensure that they are both properly updated, check that workflow's
DAG and existing
- // JobConfigs are consistent for JobQueues
- JobConfig jobConfig = newJobConfigs.get(jobName);
- if (_workflowConfigMap.containsKey(jobConfig.getWorkflow())) {
- WorkflowConfig workflowConfig =
_workflowConfigMap.get(jobConfig.getWorkflow());
- // Check that the job's parent workflow's DAG contains this job
- if ((workflowConfig.isJobQueue() || !workflowConfig.isTerminable()) &&
!_runtimeJobDagMap
-
.get(workflowConfig.getWorkflowId()).getAllNodes().contains(jobName)) {
- // Inconsistency between JobConfigs and DAGs found. Add the workflow
to workflowsUpdated
- // to rebuild the RuntimeJobDag
- workflowsUpdated.add(jobConfig.getWorkflow());
- }
- }
- }
-
- // Removed jobs
// This block makes sure that the workflow config has been changed.
// This avoid the race condition where job config has been purged but job
has not been deleted
Review comment:
Maybe this comment needs to be changed. It was added for the purge race
condition. Now it is serving as general-purpose workflow config modification.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
