[
https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526130#comment-14526130
]
Vinod Kumar Vavilapalli commented on YARN-2962:
-----------------------------------------------
bq. it will indeed be backwards incompatible. We should get this into trunk. To
include this in branch-2, we should probably guard this with a config, and
users will have to format the store to start using this. Otherwise, an RM
restart will fail.
Apologies for coming in late. I don't think we should give up on compatibility
that easily. You can easily provide compat by doing one of the following
- Forcing the new structure into its own parent directly. For e.g. instead of
directly under RM_APP_ROOT, put it under something like
RM_APP_ROOT/hierarchies, that way RM can read the old and new styles. /cc
[~jianhe].
- If not that, we can simplify it without special hierarchies by having RM
recursively read nodes all the time. As we already specifically look for
"application_" prefix to read app-date, having application_1234_0456 right next
to a 00/ directory is simply going to work without much complexity.
That way, we should also have a more reasonable default instead of the current
no-split default config value.
> ZKRMStateStore: Limit the number of znodes under a znode
> --------------------------------------------------------
>
> Key: YARN-2962
> URL: https://issues.apache.org/jira/browse/YARN-2962
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Affects Versions: 2.6.0
> Reporter: Karthik Kambatla
> Assignee: Varun Saxena
> Priority: Critical
> Attachments: YARN-2962.01.patch, YARN-2962.2.patch, YARN-2962.3.patch
>
>
> We ran into this issue where we were hitting the default ZK server message
> size configs, primarily because the message had too many znodes even though
> they individually they were all small.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
