[
https://issues.apache.org/jira/browse/YARN-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated YARN-7262:
--------------------------------
Attachment: YARN-7262.003.patch
{quote}I kinda want to suggest that you also use the HIERARCHIES directory just
for consistency.{quote}
My main concern is that the znode path is already pretty long and this is going
to make it even longer (and is also not necessary). If I could, I'd get rid of
the other one, but it's already been in a release so that would make things
_really_ complicated. I'm not sure we should repeat a decision with downsides
and no clear upsides just for (internal) consistency; if it was external to the
user, maybe.
The 003 patch:
- Added Javadoc for the new property in {{YarnConfiguration}}
- All new and updates assert statements now have messages
- Cleaned up {{loadRMDelegationTokenState}}
-- Renamed incorrectly named variable
-- Misc simplification
-- Unknown node check now only allows for "1", "2", "3", and "4" under the
token root node instead of anywhere
- Made {{TestZKRMStateStore#getDelegationTokenNode}} and other methods in
{{TestZKRMStateStore}} private or package private where possible
- Removed unnecessary boxing for {{renewDate}}
{quote}And if most of the other properties jumped off a bridge, would you do
it, too? {quote}
I'd definitely consider it - there's probably a good reason why they're all
jumping off a bridge. Maybe the bridge is collapsing or something?
> Add a hierarchy into the ZKRMStateStore for delegation token znodes to
> prevent jute buffer overflow
> ---------------------------------------------------------------------------------------------------
>
> Key: YARN-7262
> URL: https://issues.apache.org/jira/browse/YARN-7262
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 2.6.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: YARN-7262.001.patch, YARN-7262.002.patch,
> YARN-7262.003.patch
>
>
> We've seen users who are running into a problem where the RM is storing so
> many delegation tokens in the {{ZKRMStateStore}} that the _listing_ of those
> znodes is higher than the jute buffer. This is fine during operations, but
> becomes a problem on a fail over because the RM will try to read in all of
> the token znodes (i.e. call {{getChildren}} on the parent znode). This is
> particularly bad because everything appears to be okay, but then if a
> failover occurs you end up with no active RMs.
> There was a similar problem with the Yarn application data that was fixed in
> YARN-2962 by adding a (configurable) hierarchy of znodes so the RM could pull
> subchildren without overflowing the jute buffer (though it's off by default).
> We should add a hierarchy similar to that of YARN-2962, but for the
> delegation token znodes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
