[
https://issues.apache.org/jira/browse/YARN-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488333#comment-15488333
]
Jason Lowe commented on YARN-5547:
----------------------------------
Yes, having the ability to recover unknown keys that cannot be ignored by
failing their corresponding containers would be a nice addition and work well
with the approach of only storing those keys when absolutely necessary. Bonus
points if there's the ability to indicate that even failing the container isn't
sufficient to properly a particular unknown key. e.g.: the NM has to
unregister with a service as part of the container failure or cleanup some
other local space, etc. that the old software doesn't know how to do.
We'd still need to address policies for keys encountered outside of the
container key space, e.g.: application keys, deletion service keys, keys for
entirely new top-level systems in the state store, etc.
> NMLeveldbStateStore should be more tolerant of unknown keys
> -----------------------------------------------------------
>
> Key: YARN-5547
> URL: https://issues.apache.org/jira/browse/YARN-5547
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.6.0
> Reporter: Jason Lowe
> Assignee: Ajith S
> Attachments: YARN-5547.01.patch
>
>
> Whenever new keys are added to the NM state store it will break rolling
> downgrades because the code will throw if it encounters an unrecognized key.
> If instead it skipped unrecognized keys it could be simpler to continue
> supporting rolling downgrades. We need to define the semantics of
> unrecognized keys when containers and apps are cleaned up, e.g.: we may want
> to delete all keys underneath an app or container directory when it is being
> removed from the state store to prevent leaking unrecognized keys.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
