[
https://issues.apache.org/jira/browse/YARN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435976#comment-15435976
]
Junping Du commented on YARN-1503:
----------------------------------
Thanks [~jianhe] for the document.
The use cases and proposal make sense to me. However, I haven't found our
solution details for some failed over cases, like: AM or NM restart:
- In case of AM get failed and restart, does these localization effort need to
redo again? Do we have some way to track re-localization effort on running
containers by communication with NMs or shared some context across different
attempts?
- For NM, it could get just restart when receiving/replying additional
localization request from AM. We should consider to persistent it to NM state
store before replying back to AM.
May be we should add these scenarios into detail design?
> Support making additional 'LocalResources' available to running containers
> --------------------------------------------------------------------------
>
> Key: YARN-1503
> URL: https://issues.apache.org/jira/browse/YARN-1503
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Jian He
> Attachments: Continuous-resource-localization.pdf
>
>
> We have a use case, where additional resources (jars, libraries etc) need to
> be made available to an already running container. Ideally, we'd like this to
> be done via YARN (instead of having potentially multiple containers per node
> download resources on their own).
> Proposal:
> NM to support an additional API where a list of resources can be specified.
> Something like "localiceResource(ContainerId, Map<String, LocalResource>)
> NM would also require an additional API to get state for these resources -
> "getLocalizationState(ContainerId)" - which returns the current state of all
> local resources for the specified container(s).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
