[
https://issues.apache.org/jira/browse/YARN-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196102#comment-16196102
]
Arun Suresh commented on YARN-7275:
-----------------------------------
Thanks for the patch [~kartheek]
Couple of comments:
* We can also avoid explicitly having to write the container version key. Since
the container token also has the version. Thus, during recovery, you should
also obtain the version for the token set the 'rcs.version' when you see a
container update token.
* Once the containers are recovered, we also need to update the
ContainerScheduler data structures. Create a {{recoverActiveContainer}}
container method in the ContainerScheduler which takes a recovered container
and places it in the queue (if it is queued or paused) or puts it in the
runningContainers it is running, then call that method in
{{ContainerManagerImpl::recoverActiveContainer}} right after you add the
container to the context.
* Unfortunately, we do need to worry a bit about rollback. Essentially, if
2.8.x is upgraded to 2.9.x or 3.0.0 and then rolled back. To this extent, I
guess all you need to do is: If it is a resource change, in addition to storing
the container update token, use the old resource update key and store the
changed resource also. During recovery, if you see a resource change key, we
just ignore it.
> NM Statestore cleanup for Container updates
> -------------------------------------------
>
> Key: YARN-7275
> URL: https://issues.apache.org/jira/browse/YARN-7275
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun Suresh
> Assignee: kartheek muthyala
> Priority: Blocker
> Attachments: YARN-7275.001.patch, YARN-7275.002.patch
>
>
> Currently, only resource updates are recorded in the NM state store, we need
> to add ExecutionType updates as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
