Hi, I can’t find anything w/ session id after worker is restarted.
http://localhost:8100/admin/v2/clusters/taskTestCluster/instances/BOB-2/1669dbeb1fa0003 I get error 404 http://localhost:8100/admin/v2/clusters/taskTestCluster/instances/BOB-2/1669dbeb1fa0003/Task_REST-GET_1540245606312 also gets error 404 http://localhost:8100/admin/v2/clusters/taskTestCluster/instances/BOB-2 liveInstance id "BOB-2" simpleFields HELIX_VERSION "0.8.2" LIVE_INSTANCE "35436@XXXXXXXX" SESSION_ID "1669dbeb1fa0003" mapFields {} listFields {} id "BOB-2" config id "BOB-2" simpleFields HELIX_ENABLED "true" HELIX_ENABLED_TIMESTAMP "1539205545920" HELIX_HOST "BOB-2" HELIX_PORT "" mapFields {} listFields TAG_LIST 0 "minion_untagged" Digant From: Hunter Lee [mailto:[email protected]] Sent: Monday, October 22, 2018 5:20 PM To: [email protected] Subject: Re: Helix 0.8.2 Task Reassignment Hi Digant, Helix Task Framework has automatic task-reassign built in. However, I've recently discovered an edge case where a task could get stuck in case there is a Participant connection issue/Zookeeper session reset. I am currently working on the fix, but just to verify this, could you tell me what you see under currentStates for the instance? The currentState of the task should be located in /<YOUR_CLUSTER_NAME>/INSTANCES/<YOUR_INSTANCE>/<session string>/<YOUR_JOB_NAME> and it should contain fields like ASSIGNED_PARTICIPANT, INFO, STATE, etc. Thanks, Hunter On Mon, Oct 22, 2018 at 11:36 AM Modha, Digant <[email protected]<mailto:[email protected]>> wrote: Hi Hunter, I am currently evaluating the Helix Job functionality. The number of live instances varies between 1 and 3. I set the numConcurrentTasksPerInstance to 1 since I don’t expect the workers to do more than one task concurrently. Right now I am only testing it with 1 task per job, but I expect to have tens or hundreds of tasks when fully functional. ZKHelixManager.ALLOW_PARTICIPANT_AUTO_JOIN = true ; JobConfig.Builder jobBuilder = new JobConfig.Builder().addTaskConfigs(helixTaskConfigs) .setInstanceGroupTag(instanceTag) .setNumConcurrentTasksPerInstance(numConcurrentTasksPerInstance) .setIgnoreDependentJobFailure(true) .setMaxAttemptsPerTask(10) .setFailureThreshold(Integer.MAX_VALUE) ; Thanks, Digant From: Hunter Lee [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, October 22, 2018 2:20 PM To: [email protected]<mailto:[email protected]> Subject: Re: Helix 0.8.2 Task Reassignment Digant, I might have a clue - but before we jump to a conclusion, do you mind sharing your workflow/job configs? Also, the number of jobs and tasks per job, and the number of live instances. More specifically, what are you setting NumConcurrentTasksPerInstance in the jobs' configs? Thanks, Hunter Sent from BlueMail<https://clicktime.symantec.com/a/1/Aox4D8ZPi6QeSixwvmmcRl7KP66IhKEG9zr_FPlhc5Q=?d=fo8FV_9pq_NjX4B52Yb9IQYC9iXsFs9tyjfKDyiAoa8BxTcK1JJm05YzzI0q86U_-O09Yj7_Pq72APcLoIeYwzohUBT7yzao85v6r1LO39Ym4pljTzW9Afiy8ng8FvQjEK5y8g3nZVBuSt_ICHDRI_Gi5z6VhJmVdk1mi7dNnhdWWU_4Ie4IiCv4CnVBHNRv5DpBEMlkrmdGpfOSpPbPxrqckWTOngpoRqUAVwIhucE48Ki6z27Kb4zHJlzra4Jsz1YEVUj7EeFPj_i3lFypFErVZQL_bg6qFbaPYVhztYM6Zcvrm0teF3heNBvDXCJNvnzMGnq1muLQYhQjQdUQaoN7GPB1uEQ5pb2jrL_i5o2hZmEF5n7TAhYDtE8NewgSrjfHxh_p0nDWUY7Qfo7dGDa1P9AQTgXCR8PiziXvP7gXZg%3D%3D&u=http%3A%2F%2Fwww.bluemail.me%2Fr%3Fb%3D13688> On Oct 22, 2018, at 11:08, "Modha, Digant" <[email protected]<mailto:[email protected]>> wrote: Hi, When an instance a task is assigned to dies after task is assigned but before task is completed, task does not seem to be reassigned to another instance. Even if the original instance comes back up, it does not seem to process the task. I am using helix 0.8.2 and submitting task to a job queue. Should the job context partition state be set to TASK_ERROR in AssignableInstanceManager when the instance a task is assigned to is no longer a LiveInstance? jobContext.setPartitionState(taskIndex, TaskPartitionState.TASK_ERROR); Digant TD Securities disclaims any liability or losses either direct or consequential caused by the use of this information. This communication is for informational purposes only and is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. TD Securities is neither making any investment recommendation nor providing any professional or advisory services relating to the activities described herein. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice Any products described herein are (i) not insured by the FDIC, (ii) not a deposit or other obligation of, or guaranteed by, an insured depository institution and (iii) subject to investment risks, including possible loss of the principal amount invested. The information shall not be further distributed or duplicated in whole or in part by any means without the prior written consent of TD Securities. TD Securities is a trademark of The Toronto-Dominion Bank and represents TD Securities (USA) LLC and certain investment banking activities of The Toronto-Dominion Bank and its subsidiaries. TD Securities disclaims any liability or losses either direct or consequential caused by the use of this information. This communication is for informational purposes only and is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. TD Securities is neither making any investment recommendation nor providing any professional or advisory services relating to the activities described herein. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice Any products described herein are (i) not insured by the FDIC, (ii) not a deposit or other obligation of, or guaranteed by, an insured depository institution and (iii) subject to investment risks, including possible loss of the principal amount invested. The information shall not be further distributed or duplicated in whole or in part by any means without the prior written consent of TD Securities. TD Securities is a trademark of The Toronto-Dominion Bank and represents TD Securities (USA) LLC and certain investment banking activities of The Toronto-Dominion Bank and its subsidiaries. TD Securities disclaims any liability or losses either direct or consequential caused by the use of this information. This communication is for informational purposes only and is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. TD Securities is neither making any investment recommendation nor providing any professional or advisory services relating to the activities described herein. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice Any products described herein are (i) not insured by the FDIC, (ii) not a deposit or other obligation of, or guaranteed by, an insured depository institution and (iii) subject to investment risks, including possible loss of the principal amount invested. The information shall not be further distributed or duplicated in whole or in part by any means without the prior written consent of TD Securities. TD Securities is a trademark of The Toronto-Dominion Bank and represents TD Securities (USA) LLC and certain investment banking activities of The Toronto-Dominion Bank and its subsidiaries.
