[
https://issues.apache.org/jira/browse/YARN-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297397#comment-15297397
]
Li Lu commented on YARN-5018:
-----------------------------
The current situation is a little bit nasty... In the original version we
firstly called collector.start, then put it in the collector map, because we do
not want to publish any containers that are not yet started (although "started"
is only a trivial status change). If we start a collector after putting it into
the collector map, then anyone getting collectors may get collectors that are
not yet started.
So the problem is that without a separate collector.postPut, we have the
following dependencies on TimelineCollectorManager#putIfAbsent:
1. Collector start happens before putting the collector into the map
2. When postPut is called, the collector is in the map (I'm not 100% sure about
this assumption)
3. Aggregation should only run after postPut is called.
Now I'm not very sure about 2. Breaking this assumption seems to be the best
bet for us if we don't like the extra postPut (I don't like it, frankly
speaking. ).
> Online aggregation logic should not run immediately after collectors got
> started
> --------------------------------------------------------------------------------
>
> Key: YARN-5018
> URL: https://issues.apache.org/jira/browse/YARN-5018
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Li Lu
> Assignee: Li Lu
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5018-YARN-2928.001.patch,
> YARN-5018-YARN-2928.002.patch, YARN-5018-YARN-2928.003.patch,
> YARN-5018-YARN-2928.004.patch
>
>
> In app level collector, we launch the aggregation logic immediately after the
> collector got started. However, at this time, important context data has yet
> to be published to the container. Also, if the aggregation result is empty,
> we do not need to publish them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
