[
https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14537264#comment-14537264
]
Varun Saxena commented on YARN-2962:
------------------------------------
[~vinodkv] / [~asuresh], supporting 2 separate hierarchies will increase
complexity. Let us consider option 1 i.e. having {{RM_APP_ROOT/hierarchies}}.
Here, we also need to consider the case where split index can be changed from
say 2 to 3. To handle this case we can have multiple folders under hierarchies
to indicate split index. But, this would mean that for an app we may have to
look under upto 5 locations till we succeed.
Option 2 can also be done. Here we can check whether data exists under a znode
or not to determine whether we found the app or not. Here also we may have to
look up multiple times before finding an app though.
We can also do as under :
1. As Vinod suggested, write a tool or utility like "yarn resourcemanager
-format-state-store" to migrate apps from the current scheme to the newly
configured scheme. Can also allow giving the app index from command line. Not
sure though how much time migrating 10000 apps(default value of max number of
apps in store) in state store will take.
2. Current code will continue as it is. We can abort running of RM if we find
mismatch in the scheme used for storing of apps. We can then warn the admin to
run the tool above before he tries to restart RM.
> ZKRMStateStore: Limit the number of znodes under a znode
> --------------------------------------------------------
>
> Key: YARN-2962
> URL: https://issues.apache.org/jira/browse/YARN-2962
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Affects Versions: 2.6.0
> Reporter: Karthik Kambatla
> Assignee: Varun Saxena
> Priority: Critical
> Attachments: YARN-2962.01.patch, YARN-2962.2.patch, YARN-2962.3.patch
>
>
> We ran into this issue where we were hitting the default ZK server message
> size configs, primarily because the message had too many znodes even though
> they individually they were all small.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
