[ https://issues.apache.org/jira/browse/YARN-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318415#comment-14318415 ]
Jason Lowe commented on YARN-2079: ---------------------------------- Thanks for the review, Rohith! bq. why can't store delay time in seconds directly instead of taking system time and summing up with delaySeconds. We can't store the delay time directly, as we need an absolute timestamp and not a relative timestamp in order to know how much time is left between the original delete request and when it should be deleted. If we just store the delete delay then we have no choice but to wait another full delete delay after restarting before deleting, as we cannot determine how long it's been since the delete was requested when we restart. As for should we store the value in seconds vs. milliseconds, I don't think it matters much either way. 64-bits is plenty of precision, and timestamps tend to be milliseconds (because of System.currentTimeMillis) and are more precise. IMHO we shouldn't throw away precision without a good reason. However if people feel it's important to make sure this is stored in seconds I can make that change. bq. All the mothods in NMMemoryStateStoreService are being synchronized but not in othere stateStoreService. That's because leveldb is already thread-safe but the data structures used by NMMemoryStateStoreService are not. In other words, NMLeveldbStateStoreService is simply relying on the atomicity of leveldb interactions, as the state store operations don't operate on any member except the leveldb instance itself. NMMemoryStateStoreService needs to synchronize because it is iterating/modifying non-threadsafe members such as maps, sets, etc. > Recover NonAggregatingLogHandler state upon nodemanager restart > --------------------------------------------------------------- > > Key: YARN-2079 > URL: https://issues.apache.org/jira/browse/YARN-2079 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager > Affects Versions: 2.4.0 > Reporter: Jason Lowe > Assignee: Jason Lowe > Attachments: YARN-2079.002.patch, YARN-2079.patch > > > The state of NonAggregatingLogHandler needs to be persisted so logs are > properly deleted across a nodemanager restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)