Jason Lowe commented on YARN-2079:

Thanks for the review, Rohith!

bq. why can't store delay time in seconds directly instead of taking system 
time and summing up with delaySeconds.

We can't store the delay time directly, as we need an absolute timestamp and 
not a relative timestamp in order to know how much time is left between the 
original delete request and when it should be deleted.  If we just store the 
delete delay then we have no choice but to wait another full delete delay after 
restarting before deleting, as we cannot determine how long it's been since the 
delete was requested when we restart.

As for should we store the value in seconds vs. milliseconds, I don't think it 
matters much either way.  64-bits is plenty of precision, and timestamps tend 
to be milliseconds (because of System.currentTimeMillis) and are more precise.  
IMHO we shouldn't throw away precision without a good reason.  However if 
people feel it's important to make sure this is stored in seconds I can make 
that change.

bq. All the mothods in NMMemoryStateStoreService are being synchronized but not 
in othere stateStoreService.

That's because leveldb is already thread-safe but the data structures used by 
NMMemoryStateStoreService are not.  In other words, NMLeveldbStateStoreService 
is simply relying on the atomicity of leveldb interactions, as the state store 
operations don't operate on any member except the leveldb instance itself.  
NMMemoryStateStoreService needs to synchronize because it is 
iterating/modifying non-threadsafe members such as maps, sets, etc.

> Recover NonAggregatingLogHandler state upon nodemanager restart
> ---------------------------------------------------------------
>                 Key: YARN-2079
>                 URL: https://issues.apache.org/jira/browse/YARN-2079
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-2079.002.patch, YARN-2079.patch
> The state of NonAggregatingLogHandler needs to be persisted so logs are 
> properly deleted across a nodemanager restart.

This message was sent by Atlassian JIRA

Reply via email to