[
https://issues.apache.org/jira/browse/YARN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436378#comment-15436378
]
Jian He commented on YARN-1503:
-------------------------------
Thanks for the feedback, Arun, Varun, Junping.
bq. I was wondering how this would tie into the NM Container state machine.
The re-localization process should not tie to container state-machine,
regardless whether the localization fails or succeed. Container continues to
run. This echoes the requirement for Tez relocalization. The AM also gets
notification whether the localization process succeeded or failed.
bq. I haven't found our solution details for some failed over cases, like: AM
or NM restart:
For AM restart, it simply queries NM for the localization status. For NM
restart, it needs to persist the symlink mapping. I had thought to add symlink
into the LocalResource object itself so that it gets persisted automatically.
Anyway, these are advanced stuff and not conflicting with core change. I'll
open separate jira and talk about how to implement it when it comes.
> Support making additional 'LocalResources' available to running containers
> --------------------------------------------------------------------------
>
> Key: YARN-1503
> URL: https://issues.apache.org/jira/browse/YARN-1503
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Jian He
> Attachments: Continuous-resource-localization.pdf
>
>
> We have a use case, where additional resources (jars, libraries etc) need to
> be made available to an already running container. Ideally, we'd like this to
> be done via YARN (instead of having potentially multiple containers per node
> download resources on their own).
> Proposal:
> NM to support an additional API where a list of resources can be specified.
> Something like "localiceResource(ContainerId, Map<String, LocalResource>)
> NM would also require an additional API to get state for these resources -
> "getLocalizationState(ContainerId)" - which returns the current state of all
> local resources for the specified container(s).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
