Jian He commented on YARN-2837:

Thanks Zhijie and Li, some comments on my side:

- Logic in {{Version loadedVersion = loadVersion();}} Consider this scenario: 
CURRENT_VERSION_INFO = 2.0; there’s no version info currently saved in 
store-store. {{loadVersion}} returns 1.0; It’ll throw inCompatible exception, 
even though it should not. 
- we can probably use protobuf to incorporate both tokenIdentifer and the 
renewDate to support better compatibility. e.g. RMDelegationTokenIdentifierData
      renewDate = in.readLong();
- {{LeveldbTimelineStateStore#updateToken}}, it’s always adding new token, the 
old token still remain, we should remove the old token
- {{AbstractDelegationTokenSecretManager#delegationTokenSequenceNumber}} is not 
updated on recovery; the implementation seems using sequenceNumber as the key, 
we need to keep track of the latest sequenceNumber so that it can be recovered.
- the following looks ok , a simpler way might be to just concatenate two 
- the following log is present in both 
{{TimelineDelegationTokenSecretManager#storeNewMasterKey}} and the underlying 
state-store implementation; only printing  in one place is enough. similar to 
other operations.
      if (LOG.isDebugEnabled()) {
        LOG.debug("Storing master key " + key.getKeyId());
- FILENAME->DB_NAME; leveldb-state-store.ldb -> timeline-state-store.ldb
- default path for state store is the same as time-line store for application 
data. If apps posts massive data in store, will that also affect system data 
seek performance ? If so, we should have a different store path from the one 
for apps.  Or we could force user to configure the path properly and throw 
exception otherwise.

> Timeline server needs to recover the timeline DT when restarting
> ----------------------------------------------------------------
>                 Key: YARN-2837
>                 URL: https://issues.apache.org/jira/browse/YARN-2837
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>            Priority: Blocker
>             Fix For: 2.7.0
>         Attachments: YARN-2837.1.patch, YARN-2837.2.patch, YARN-2837.3.patch
> Timeline server needs to recover the stateful information when restarting as 
> RM/NM/JHS does now. So far the stateful information only includes the 
> timeline DT. Without recovery, the timeline DT of the existing YARN apps is 
> not long valid, and cannot be renewed any more after the timeline server is 
> restarted.

This message was sent by Atlassian JIRA

Reply via email to