[
https://issues.apache.org/jira/browse/YARN-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234801#comment-14234801
]
Jian He commented on YARN-2837:
-------------------------------
Thanks Zhijie and Li, some comments on my side:
- Logic in {{Version loadedVersion = loadVersion();}} Consider this scenario:
CURRENT_VERSION_INFO = 2.0; there’s no version info currently saved in
store-store. {{loadVersion}} returns 1.0; It’ll throw inCompatible exception,
even though it should not.
- we can probably use protobuf to incorporate both tokenIdentifer and the
renewDate to support better compatibility. e.g. RMDelegationTokenIdentifierData
{code}
renewDate = in.readLong();
tokenId.readFields(in);
{code}
- {{LeveldbTimelineStateStore#updateToken}}, it’s always adding new token, the
old token still remain, we should remove the old token
- {{AbstractDelegationTokenSecretManager#delegationTokenSequenceNumber}} is not
updated on recovery; the implementation seems using sequenceNumber as the key,
we need to keep track of the latest sequenceNumber so that it can be recovered.
- the following looks ok , a simpler way might be to just concatenate two
strings.
{code}
.add(TOKEN_MASTER_KEY_ENTRY_PREFIX).add(Integer.toString(keyId)){code}
- the following log is present in both
{{TimelineDelegationTokenSecretManager#storeNewMasterKey}} and the underlying
state-store implementation; only printing in one place is enough. similar to
other operations.
{code}
if (LOG.isDebugEnabled()) {
LOG.debug("Storing master key " + key.getKeyId());
}
{code}
- FILENAME->DB_NAME; leveldb-state-store.ldb -> timeline-state-store.ldb
- default path for state store is the same as time-line store for application
data. If apps posts massive data in store, will that also affect system data
seek performance ? If so, we should have a different store path from the one
for apps. Or we could force user to configure the path properly and throw
exception otherwise.
> Timeline server needs to recover the timeline DT when restarting
> ----------------------------------------------------------------
>
> Key: YARN-2837
> URL: https://issues.apache.org/jira/browse/YARN-2837
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: timelineserver
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
> Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-2837.1.patch, YARN-2837.2.patch, YARN-2837.3.patch
>
>
> Timeline server needs to recover the stateful information when restarting as
> RM/NM/JHS does now. So far the stateful information only includes the
> timeline DT. Without recovery, the timeline DT of the existing YARN apps is
> not long valid, and cannot be renewed any more after the timeline server is
> restarted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
