[
https://issues.apache.org/jira/browse/YARN-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387498#comment-14387498
]
Li Lu commented on YARN-3334:
-----------------------------
Hi [~sjlee0], I have some quick questions about getTimelineClient(). I agree
you raised a very valid concern that we may initialize more clients than
needed. However, here, I think here we're initializing the client before
putIfAbsent call to avoid other concurrent thread seeing uninitialized clients.
If we initialize the client after we do putIfAbsent, concurrent threads may see
uninitialized clients. For example:
thread 0:
{code}
client = createTimelineClient
old = putIfAbsent(id, client) // old = null, client is not initialized and
started
// thread 0 paused for some reason
{code}
thread 1 with same id:
{code}
old = putIfAbsent(id, client) // and we get client from thread 0
client = old
return client
{code}
Then thread 1 will have a non-started client. Since the uninitialized client is
published, it's even possible to have initializations concurrent to other
client method calls. This may introduce weird semantics under the specification
of Java memory model.
I think we're doing the redundant init and start operations to avoid publishing
uninitialized clients. The atomic putIfAbsent operation published the
initialized data. However, as [~sjlee0] raised, we do need to make sure
unreferenced, redundant clients can be reclaimed by the garbage collector (not
leaking the memory).
> [Event Producers] NM TimelineClient life cycle handling and container metrics
> posting to new timeline service.
> --------------------------------------------------------------------------------------------------------------
>
> Key: YARN-3334
> URL: https://issues.apache.org/jira/browse/YARN-3334
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Affects Versions: YARN-2928
> Reporter: Junping Du
> Assignee: Junping Du
> Attachments: YARN-3334-demo.patch, YARN-3334-v1.patch,
> YARN-3334-v2.patch, YARN-3334-v3.patch
>
>
> After YARN-3039, we have service discovery mechanism to pass app-collector
> service address among collectors, NMs and RM. In this JIRA, we will handle
> service address setting for TimelineClients in NodeManager, and put container
> metrics to the backend storage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
