Digant, Sorry I was being unclear on this. Are you able to look at the ZNodes via Zooinspector? I don't think Helix REST api lets you take a look at the currentstates.
Hunter On Mon, Oct 22, 2018 at 3:07 PM Modha, Digant <[email protected]> wrote: > Hi, > > > > I can’t find anything w/ session id after worker is restarted. > > > > > http://localhost:8100/admin/v2/clusters/taskTestCluster/instances/BOB-2/1669dbeb1fa0003 > I get error 404 > > http://localhost:8100/admin/v2/clusters/taskTestCluster/instances/BOB-2/1669dbeb1fa0003/Task_REST-GET_1540245606312 > also gets error 404 > > > > http://localhost:8100/admin/v2/clusters/taskTestCluster/instances/BOB-2 > > > > liveInstance > > id > > "BOB-2" > > simpleFields > > HELIX_VERSION > > "0.8.2" > > LIVE_INSTANCE > > "35436@XXXXXXXX" > > SESSION_ID > > "1669dbeb1fa0003" > > mapFields > > {} > > listFields > > {} > > id > > "BOB-2" > > config > > id > > "BOB-2" > > simpleFields > > HELIX_ENABLED > > "true" > > HELIX_ENABLED_TIMESTAMP > > "1539205545920" > > HELIX_HOST > > "BOB-2" > > HELIX_PORT > > "" > > mapFields > > {} > > listFields > > TAG_LIST > > 0 > > "minion_untagged" > > > > > > Digant > > > > *From:* Hunter Lee [mailto:[email protected]] > *Sent:* Monday, October 22, 2018 5:20 PM > *To:* [email protected] > *Subject:* Re: Helix 0.8.2 Task Reassignment > > > > Hi Digant, > > > > Helix Task Framework has automatic task-reassign built in. However, I've > recently discovered an edge case where a task could get stuck in case there > is a Participant connection issue/Zookeeper session reset. I am currently > working on the fix, but just to verify this, could you tell me what you see > under currentStates for the instance? > > > > The currentState of the task should be located in > /<YOUR_CLUSTER_NAME>/INSTANCES/<YOUR_INSTANCE>/<session > string>/<YOUR_JOB_NAME> > > and it should contain fields like ASSIGNED_PARTICIPANT, INFO, STATE, etc. > > > > Thanks, > > Hunter > > > > On Mon, Oct 22, 2018 at 11:36 AM Modha, Digant < > [email protected]> wrote: > > Hi Hunter, > > > > I am currently evaluating the Helix Job functionality. The number of > live instances varies between 1 and 3. I set the > numConcurrentTasksPerInstance to 1 since I don’t expect the workers to do > more than one task concurrently. Right now I am only testing it with 1 > task per job, but I expect to have tens or hundreds of tasks when fully > functional. > > > > > > ZKHelixManager.*ALLOW_PARTICIPANT_AUTO_JOIN **= true *; > > > > > > JobConfig.Builder jobBuilder = *new * > JobConfig.Builder().addTaskConfigs(helixTaskConfigs) > .setInstanceGroupTag(instanceTag) > .setNumConcurrentTasksPerInstance(numConcurrentTasksPerInstance) > .setIgnoreDependentJobFailure(*true*) > .setMaxAttemptsPerTask(10) > .setFailureThreshold(Integer.*MAX_VALUE*) > ; > > Thanks, > > > > Digant > > > > *From:* Hunter Lee [mailto:[email protected]] > *Sent:* Monday, October 22, 2018 2:20 PM > *To:* [email protected] > *Subject:* Re: Helix 0.8.2 Task Reassignment > > > > Digant, > > I might have a clue - but before we jump to a conclusion, do you mind > sharing your workflow/job configs? Also, the number of jobs and tasks per > job, and the number of live instances. > > More specifically, what are you setting NumConcurrentTasksPerInstance in > the jobs' configs? > > Thanks, > > Hunter > > Sent from BlueMail > <https://clicktime.symantec.com/a/1/Aox4D8ZPi6QeSixwvmmcRl7KP66IhKEG9zr_FPlhc5Q=?d=fo8FV_9pq_NjX4B52Yb9IQYC9iXsFs9tyjfKDyiAoa8BxTcK1JJm05YzzI0q86U_-O09Yj7_Pq72APcLoIeYwzohUBT7yzao85v6r1LO39Ym4pljTzW9Afiy8ng8FvQjEK5y8g3nZVBuSt_ICHDRI_Gi5z6VhJmVdk1mi7dNnhdWWU_4Ie4IiCv4CnVBHNRv5DpBEMlkrmdGpfOSpPbPxrqckWTOngpoRqUAVwIhucE48Ki6z27Kb4zHJlzra4Jsz1YEVUj7EeFPj_i3lFypFErVZQL_bg6qFbaPYVhztYM6Zcvrm0teF3heNBvDXCJNvnzMGnq1muLQYhQjQdUQaoN7GPB1uEQ5pb2jrL_i5o2hZmEF5n7TAhYDtE8NewgSrjfHxh_p0nDWUY7Qfo7dGDa1P9AQTgXCR8PiziXvP7gXZg%3D%3D&u=http%3A%2F%2Fwww.bluemail.me%2Fr%3Fb%3D13688> > > On Oct 22, 2018, at 11:08, "Modha, Digant" <[email protected]> > wrote: > > Hi, > > > > When an instance a task is assigned to dies after task is assigned but > before task is completed, task does not seem to be reassigned to another > instance. Even if the original instance comes back up, it does not seem to > process the task. > > > > > > I am using helix 0.8.2 and submitting task to a job queue. > > > > Should the job context partition state be set to TASK_ERROR in > AssignableInstanceManager when the instance a task is assigned to is no > longer a LiveInstance? > > > > jobContext.setPartitionState(taskIndex, TaskPartitionState.*TASK_ERROR*); > > > > Digant > > > > > > TD Securities disclaims any liability or losses either direct or > consequential caused by the use of this information. This communication is > for informational purposes only and is not intended as an offer or > solicitation for the purchase or sale of any financial instrument or as an > official confirmation of any transaction. TD Securities is neither making > any investment recommendation nor providing any professional or advisory > services relating to the activities described herein. All market prices, > data and other information are not warranted as to completeness or accuracy > and are subject to change without notice Any products described herein are > (i) not insured by the FDIC, (ii) not a deposit or other obligation of, or > guaranteed by, an insured depository institution and (iii) subject to > investment risks, including possible loss of the principal amount invested. > The information shall not be further distributed or duplicated in whole or > in part by any means without the prior written consent of TD Securities. TD > Securities is a trademark of The Toronto-Dominion Bank and represents TD > Securities (USA) LLC and certain investment banking activities of The > Toronto-Dominion Bank and its subsidiaries. > > > > TD Securities disclaims any liability or losses either direct or > consequential caused by the use of this information. This communication is > for informational purposes only and is not intended as an offer or > solicitation for the purchase or sale of any financial instrument or as an > official confirmation of any transaction. TD Securities is neither making > any investment recommendation nor providing any professional or advisory > services relating to the activities described herein. All market prices, > data and other information are not warranted as to completeness or accuracy > and are subject to change without notice Any products described herein are > (i) not insured by the FDIC, (ii) not a deposit or other obligation of, or > guaranteed by, an insured depository institution and (iii) subject to > investment risks, including possible loss of the principal amount invested. > The information shall not be further distributed or duplicated in whole or > in part by any means without the prior written consent of TD Securities. TD > Securities is a trademark of The Toronto-Dominion Bank and represents TD > Securities (USA) LLC and certain investment banking activities of The > Toronto-Dominion Bank and its subsidiaries. > > > > TD Securities disclaims any liability or losses either direct or > consequential caused by the use of this information. This communication is > for informational purposes only and is not intended as an offer or > solicitation for the purchase or sale of any financial instrument or as an > official confirmation of any transaction. TD Securities is neither making > any investment recommendation nor providing any professional or advisory > services relating to the activities described herein. All market prices, > data and other information are not warranted as to completeness or accuracy > and are subject to change without notice Any products described herein are > (i) not insured by the FDIC, (ii) not a deposit or other obligation of, or > guaranteed by, an insured depository institution and (iii) subject to > investment risks, including possible loss of the principal amount invested. > The information shall not be further distributed or duplicated in whole or > in part by any means without the prior written consent of TD Securities. TD > Securities is a trademark of The Toronto-Dominion Bank and represents TD > Securities (USA) LLC and certain investment banking activities of The > Toronto-Dominion Bank and its subsidiaries. >
