[
https://issues.apache.org/jira/browse/YARN-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318415#comment-14318415
]
Jason Lowe commented on YARN-2079:
----------------------------------
Thanks for the review, Rohith!
bq. why can't store delay time in seconds directly instead of taking system
time and summing up with delaySeconds.
We can't store the delay time directly, as we need an absolute timestamp and
not a relative timestamp in order to know how much time is left between the
original delete request and when it should be deleted. If we just store the
delete delay then we have no choice but to wait another full delete delay after
restarting before deleting, as we cannot determine how long it's been since the
delete was requested when we restart.
As for should we store the value in seconds vs. milliseconds, I don't think it
matters much either way. 64-bits is plenty of precision, and timestamps tend
to be milliseconds (because of System.currentTimeMillis) and are more precise.
IMHO we shouldn't throw away precision without a good reason. However if
people feel it's important to make sure this is stored in seconds I can make
that change.
bq. All the mothods in NMMemoryStateStoreService are being synchronized but not
in othere stateStoreService.
That's because leveldb is already thread-safe but the data structures used by
NMMemoryStateStoreService are not. In other words, NMLeveldbStateStoreService
is simply relying on the atomicity of leveldb interactions, as the state store
operations don't operate on any member except the leveldb instance itself.
NMMemoryStateStoreService needs to synchronize because it is
iterating/modifying non-threadsafe members such as maps, sets, etc.
> Recover NonAggregatingLogHandler state upon nodemanager restart
> ---------------------------------------------------------------
>
> Key: YARN-2079
> URL: https://issues.apache.org/jira/browse/YARN-2079
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Affects Versions: 2.4.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: YARN-2079.002.patch, YARN-2079.patch
>
>
> The state of NonAggregatingLogHandler needs to be persisted so logs are
> properly deleted across a nodemanager restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
