[
https://issues.apache.org/jira/browse/MAPREDUCE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13773214#comment-13773214
]
Daryn Sharp commented on MAPREDUCE-5332:
----------------------------------------
+JHSDelegationTokenSecretManager+
# I'm not sure {{updateStoredToken}} should remove and then "recreate" the
token. The impact of removing the token will cause tokens to be lost if the
store can't save another. In the case of the FS, a renewal will delete the
persisted token file, and may then fail to write the token with the updated
renewal time. Since you're creating a tmp file and then renaming it, the
delete can be implicitly handled by the rename. This also reduces an rpc call.
+HistoryServerFileSystemStateStoreService+
# Closing the fs isn't advised since it is using the global fs cache instance...
# NUM_TOKENS_PER_BUCKET isn't "per" anymore
# Closing streams in a finally block should be via {{IOUtils.closeStream}}
# {{loadTokenState}} calls {{loadTokensFromBucket}} without checking if the
bucket is really a directory. {{loadTokensFromBucket}} calls {{listStatus}}
which returns the file if it's a file, so it will just warn about the path.
Later when {{storeToken}} is invoked, it calls {{createDir}} which is happy
that the path exists even though it's a file, then when {{storeToken}} tries to
write the new token file it will blow up with a very confusing error and
probably take down the server when the first job submission wants a token.
# You tried to reduce the RPC overhead when writing/renaming tokens, but
{{storeToken}} is going to stat the bucket directory every time. Perhaps
initialization should create any missing bucket dirs.
# {{createFile}} doesn't remove the tmp file if the write or rename fails.
+JobHistoryServer+
# Ideally, but just a suggestion, {{createJHSSecretManager}} should take the
{{stateStore}} as an arg instead of implicitly relying on it already being
defined.
# The DTSM has the {{stateStore}} so its recovery method could load the state -
instead of the caller loading the state from the {{stateStore}} and passing it
in. The code may become a bit easier to follow, but just a suggestion.
+HistoryServerMemStateStoreService+
# Should {{startStorage}} instantiate the {{state}} instance rather than
{{initStorage}}? Otherwise if you close it, the state gets nulled, starting it
again does nothing, then the code will later NPE. The semantics are a bit
unclear.
> Support token-preserving restart of history server
> --------------------------------------------------
>
> Key: MAPREDUCE-5332
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5332
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobhistoryserver
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-5332-2.patch, MAPREDUCE-5332-3.patch,
> MAPREDUCE-5332-4.patch, MAPREDUCE-5332-5.patch, MAPREDUCE-5332-5.patch,
> MAPREDUCE-5332-6.patch, MAPREDUCE-5332.patch
>
>
> To better support rolling upgrades through a cluster, the history server
> needs the ability to restart without losing track of delegation tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
