[
https://issues.apache.org/jira/browse/YARN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434851#comment-15434851
]
Arun Suresh commented on YARN-1503:
-----------------------------------
Thanks for driving this [~jianhe]
I was wondering how this would tie into the NM Container state machine.
Currently, a Container is in the LOCALIZING state while resources are being
localized and then it starts. What state would the container be in now if
re-Localization is allowed asynchronously ? What happens if localization failed
for that specific resource ? Should the Container continue running ?
Also looks like this overlaps a bit with what was being attempted in YARN-4876.
In that JIRA, we do a couple of things:
# it explicitly exposes an INIT state where the Container is LOCALIZED but not
started yet.
# it allows running containers to be re-initialized (and re-started) with new
resources.
The current prototype stops the container process, re-initializes and the
re-starts it. It would be pretty straight forward to have a policy to not stop
the container (keep the process running). Since we have explicit hooks in the
container life-cycle, it would be possible to version the resources and provide
support for roll-back etc.
> Continuous resource-localization for YARN containers
> ----------------------------------------------------
>
> Key: YARN-1503
> URL: https://issues.apache.org/jira/browse/YARN-1503
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Jian He
> Attachments: Continuous-resource-localization.pdf
>
>
> We have a use case, where additional resources (jars, libraries etc) need to
> be made available to an already running container. Ideally, we'd like this to
> be done via YARN (instead of having potentially multiple containers per node
> download resources on their own).
> The goal is to enable NodeManagers to localize resources while container is
> running. Today, resource-localization is always the first step before
> starting a container. It will be useful if YARN can localize the resources
> continuously even while container is running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
