[
https://issues.apache.org/jira/browse/YARN-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721452#comment-15721452
]
Varun Saxena commented on YARN-5955:
------------------------------------
Sorry was away so could not reply to either of the comments.
Agree that Zookeeper is not necessarily meant to be a data store. There has
been work going on in the community viz. YARN-5123 to support alternate state
store implementations.
The attempt at our end was to improve performance of RM recovery while using
Zookeeper backed state store. And with the improvements done, it should serve
use cases of most.
Gradually, one more option community is moving towards is that we will use
ATSv2 as a data store for completed apps and use RM state store only for
running apps (which Zookeeper should be able to handle).
ATSv2 though is still in alpha stage and requires a HBase cluster as well
currently.
So in the meanwhile we thought it will good to improve existing stable ZK based
state store performance and allow it to store more data. And support use cases
where having an HBase cluster (once ATSv2 stabilizes) is not a viable option.
Async loading, using multiple threads and YARN-2962(i.e. store apps
differently) are all an attempt in that direction.
> Use threadpool or multiple thread to recover app
> ------------------------------------------------
>
> Key: YARN-5955
> URL: https://issues.apache.org/jira/browse/YARN-5955
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Affects Versions: 2.7.1
> Reporter: Zhaofei Meng
> Assignee: Ajith S
> Fix For: 2.7.1
>
>
> current app recovery is one by one,use thead pool can make recovery faster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
