Hi Hunter,
I am currently evaluating the Helix Job functionality. The number of live
instances varies between 1 and 3. I set the numConcurrentTasksPerInstance to 1
since I don’t expect the workers to do more than one task concurrently. Right
now I am only testing it with 1 task per job, but I expect to have tens or
hundreds of tasks when fully functional.
ZKHelixManager.ALLOW_PARTICIPANT_AUTO_JOIN = true ;
JobConfig.Builder jobBuilder = new
JobConfig.Builder().addTaskConfigs(helixTaskConfigs)
.setInstanceGroupTag(instanceTag)
.setNumConcurrentTasksPerInstance(numConcurrentTasksPerInstance)
.setIgnoreDependentJobFailure(true)
.setMaxAttemptsPerTask(10)
.setFailureThreshold(Integer.MAX_VALUE)
;
Thanks,
Digant
From: Hunter Lee [mailto:[email protected]]
Sent: Monday, October 22, 2018 2:20 PM
To: [email protected]
Subject: Re: Helix 0.8.2 Task Reassignment
Digant,
I might have a clue - but before we jump to a conclusion, do you mind sharing
your workflow/job configs? Also, the number of jobs and tasks per job, and the
number of live instances.
More specifically, what are you setting NumConcurrentTasksPerInstance in the
jobs' configs?
Thanks,
Hunter
Sent from
BlueMail<https://clicktime.symantec.com/a/1/Aox4D8ZPi6QeSixwvmmcRl7KP66IhKEG9zr_FPlhc5Q=?d=fo8FV_9pq_NjX4B52Yb9IQYC9iXsFs9tyjfKDyiAoa8BxTcK1JJm05YzzI0q86U_-O09Yj7_Pq72APcLoIeYwzohUBT7yzao85v6r1LO39Ym4pljTzW9Afiy8ng8FvQjEK5y8g3nZVBuSt_ICHDRI_Gi5z6VhJmVdk1mi7dNnhdWWU_4Ie4IiCv4CnVBHNRv5DpBEMlkrmdGpfOSpPbPxrqckWTOngpoRqUAVwIhucE48Ki6z27Kb4zHJlzra4Jsz1YEVUj7EeFPj_i3lFypFErVZQL_bg6qFbaPYVhztYM6Zcvrm0teF3heNBvDXCJNvnzMGnq1muLQYhQjQdUQaoN7GPB1uEQ5pb2jrL_i5o2hZmEF5n7TAhYDtE8NewgSrjfHxh_p0nDWUY7Qfo7dGDa1P9AQTgXCR8PiziXvP7gXZg%3D%3D&u=http%3A%2F%2Fwww.bluemail.me%2Fr%3Fb%3D13688>
On Oct 22, 2018, at 11:08, "Modha, Digant"
<[email protected]<mailto:[email protected]>> wrote:
Hi,
When an instance a task is assigned to dies after task is assigned but before
task is completed, task does not seem to be reassigned to another instance.
Even if the original instance comes back up, it does not seem to process the
task.
I am using helix 0.8.2 and submitting task to a job queue.
Should the job context partition state be set to TASK_ERROR in
AssignableInstanceManager when the instance a task is assigned to is no longer
a LiveInstance?
jobContext.setPartitionState(taskIndex, TaskPartitionState.TASK_ERROR);
Digant
TD Securities disclaims any liability or losses either direct or consequential
caused by the use of this information. This communication is for informational
purposes only and is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation of any
transaction. TD Securities is neither making any investment recommendation nor
providing any professional or advisory services relating to the activities
described herein. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change without
notice Any products described herein are (i) not insured by the FDIC, (ii) not
a deposit or other obligation of, or guaranteed by, an insured depository
institution and (iii) subject to investment risks, including possible loss of
the principal amount invested. The information shall not be further distributed
or duplicated in whole or in part by any means without the prior written
consent of TD Securities. TD Securities is a trademark of The Toronto-Dominion
Bank and represents TD Securities (USA) LLC and certain investment banking
activities of The Toronto-Dominion Bank and its subsidiaries.
TD Securities disclaims any liability or losses either direct or consequential
caused by the use of this information. This communication is for informational
purposes only and is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation of any
transaction. TD Securities is neither making any investment recommendation nor
providing any professional or advisory services relating to the activities
described herein. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change without
notice Any products described herein are (i) not insured by the FDIC, (ii) not
a deposit or other obligation of, or guaranteed by, an insured depository
institution and (iii) subject to investment risks, including possible loss of
the principal amount invested. The information shall not be further distributed
or duplicated in whole or in part by any means without the prior written
consent of TD Securities. TD Securities is a trademark of The Toronto-Dominion
Bank and represents TD Securities (USA) LLC and certain investment banking
activities of The Toronto-Dominion Bank and its subsidiaries.
