Vinod Kumar Vavilapalli commented on YARN-2962:

bq. it will indeed be backwards incompatible. We should get this into trunk. To 
include this in branch-2, we should probably guard this with a config, and 
users will have to format the store to start using this. Otherwise, an RM 
restart will fail.

Apologies for coming in late. I don't think we should give up on compatibility 
that easily. You can easily provide compat by doing one of the following

 - Forcing the new structure into its own parent directly. For e.g. instead of 
directly under RM_APP_ROOT, put it under something like 
RM_APP_ROOT/hierarchies, that way RM can read the old and new styles. /cc 
 - If not that, we can simplify it without special hierarchies by having RM 
recursively read nodes all the time. As we already specifically look for 
"application_" prefix to read app-date, having application_1234_0456 right next 
to a 00/ directory is simply going to work without much complexity.

That way, we should also have a more reasonable default instead of the current 
no-split default config value.

> ZKRMStateStore: Limit the number of znodes under a znode
> --------------------------------------------------------
>                 Key: YARN-2962
>                 URL: https://issues.apache.org/jira/browse/YARN-2962
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.6.0
>            Reporter: Karthik Kambatla
>            Assignee: Varun Saxena
>            Priority: Critical
>         Attachments: YARN-2962.01.patch, YARN-2962.2.patch, YARN-2962.3.patch
> We ran into this issue where we were hitting the default ZK server message 
> size configs, primarily because the message had too many znodes even though 
> they individually they were all small.

This message was sent by Atlassian JIRA

Reply via email to