[
https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804088#comment-13804088
]
Tsuyoshi OZAWA commented on YARN-1307:
--------------------------------------
{quote}
When the data size exceeds the size of a batch then a new znode is created.
{quote}
Yes, we can take this apporach. However, let's marshal merits and demerits
between batching data and znode with encoded sequential numbers before doing
this.
1. Batching data approach
* Merits:
** Number of znodes is reduced.
** No need to encode sequential numbers when storing
* Demerits:
** remove*() costs get much higher. We need to scan and deseliarize objects
from upto 500KB byte array for each znode. After doing this, we need to write
updated data again.
** store*() costs gets a bit higher, because we need to read upto 500KB data
and write it back again.
2. Znodes with encoded sequential numbers approach:
* Merits:
** We can delete keys in O(1) when names of key are given.
** We can avoid reading data in znodes when storing/removing DelegationKeys or
DelegatoinTokens.
* Demerits:
** If secret key for encoding is changed by user, stored secret keys gets
invalid and need to delete all data.
3. No change between them
* Memory overhead, because no or subtle difference between data and znode.
IMO, znodes approach is better, because this approach can reduce ZooKeeper's
server-side loads compare to the other. One demerit is we need secret key to
encode if we cannot encode sequence numbers directly as I described.
If some points are wrong, please let me know.
> Rethink znode structure for RM HA
> ---------------------------------
>
> Key: YARN-1307
> URL: https://issues.apache.org/jira/browse/YARN-1307
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Tsuyoshi OZAWA
> Assignee: Tsuyoshi OZAWA
> Attachments: YARN-1307.1.patch, YARN-1307.2.patch
>
>
> Rethink for znode structure for RM HA is proposed in some JIRAs(YARN-659,
> YARN-1222). The motivation of this JIRA is quoted from Bikas' comment in
> YARN-1222:
> {quote}
> We should move to creating a node hierarchy for apps such that all znodes for
> an app are stored under an app znode instead of the app root znode. This will
> help in removeApplication and also in scaling better on ZK. The earlier code
> was written this way to ensure create/delete happens under a root znode for
> fencing. But given that we have moved to multi-operations globally, this isnt
> required anymore.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
