[
https://issues.apache.org/jira/browse/HELIX-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613456#comment-14613456
]
ASF GitHub Bot commented on HELIX-601:
--------------------------------------
Github user kanakb commented on the pull request:
https://github.com/apache/helix/pull/30#issuecomment-118425221
1. Let's say we have a target resource with 2 partitions and 0 replicas,
with one partition assigned to node A, and one partition assigned to node B.
Job 0 runs on nodes A and B, it finishes on node A, and then Job 1 starts on
node A. Then imagine node B fails, and both partitions are now on node A. Job 1
is running on node A, but Job 0 did not finish for the partition that was
reassigned to node A. We have a dependency inversion, and that's why this is
unsafe.
2. If the job does not have a target resource, this change doesn't make
sense. An exception should be thrown if you attempt to submit an untargeted
workflow that has parallelism set.
> Allow work flow to schedule dependency jobs in parallel
> -------------------------------------------------------
>
> Key: HELIX-601
> URL: https://issues.apache.org/jira/browse/HELIX-601
> Project: Apache Helix
> Issue Type: New Feature
> Reporter: Congrui Ji
>
> Currently, Helix won't schedule dependency jobs in a same work flow. For
> example, if Job2 depends on Job1, Job2 won't be scheduled until every
> partition of Job1 is completed.
> However, if some participant is very slow, then all dependency jobs is
> waiting for that single participant.
> Helix should be able to schedule multiple jobs according to a parameter.
> A.C.
> 1. Introduce parallel count parameter in work flow and job queue.
> 2. Dependency jobs can be scheduled according to the parameter (Now the
> parameter is always 1, so no parallel)
> 3. If Job2 depends on Job1, Job1 is scheduled before Job2.
> 4. No parallel jobs on the same instance. If a instance is running Job1, it
> won't run Job2 until Job1 is finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
