[
https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200154#comment-15200154
]
Varun Saxena commented on YARN-2962:
------------------------------------
Thanks Daniel for the review.
bq. One optimization I might consider is to only add the splits that actually
exist to the rmAppRootHierarchies since I would assume that the common case
will be to not use splits.
That is done in {{loadRMAppState}}. Only those hierarchies which have apps are
considered. We can go with not creating the directories which are not
configured ever, at all. But I went with this approach.
bq. what happens if an app happens to exist in more than one split?
Should not happen until and unless somebody does manual operations in state
store. And in that case, a lot can go wrong with state store. So we do make
some assumptions, even in previous code.
However, I am not sure whether we need to fence certain operations or not as I
highlighted in an earlier comment. But could not come up with a scenario
because I felt state store operations are synchronized, inconsistency should
not occur.
> ZKRMStateStore: Limit the number of znodes under a znode
> --------------------------------------------------------
>
> Key: YARN-2962
> URL: https://issues.apache.org/jira/browse/YARN-2962
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Affects Versions: 2.6.0
> Reporter: Karthik Kambatla
> Assignee: Varun Saxena
> Priority: Critical
> Attachments: YARN-2962.01.patch, YARN-2962.04.patch,
> YARN-2962.2.patch, YARN-2962.3.patch
>
>
> We ran into this issue where we were hitting the default ZK server message
> size configs, primarily because the message had too many znodes even though
> they individually they were all small.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
