[
https://issues.apache.org/jira/browse/YARN-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943760#comment-14943760
]
Sangjin Lee commented on YARN-3367:
-----------------------------------
Sorry for my late reply.
{quote}
Timelineclient async calls are only to ensure the client need not wait till the
server response & just return immediately after requesting to post entity or
even in server side we need to ensure some thing ? As currently we are trying
to send the async parameter to the server.
{quote}
I think at minimum the flush should not be done on the server side. If the
client is fine without the server response, it clearly implies flush is not
needed (we had this discussion on another JIRA).
{quote}
Is it important to maintain the order of events which are sent from sync and
async ? i.e. Is it req to ensure all the async events are also pushed along
with the current sync event or is it ok to send only the sync ? (current patch
just ensures async events are in order) .
{quote}
I'm not sure if it is a requirement that the timeline *client* has to ensure
the order of events for both sync and async. First of all, the timestamp should
be set for most of the entities, metrics, events, etc., and the server should
rely on the timestamps to resolve ordering. Also, even if the client ensures a
certain order, there are many situations under which the events will be
received by the server out of order.
{quote}
Whether its req to merge entities of multiple async calls as they belong to
same application ?
{quote}
I'm not really sure if this is something the client needs to do. If anything,
this requirement should fall on the app level timeline collector. I don't see a
whole lot of situations where the timeline client can do this easily and
unambiguously. Thoughts, [~Naganarasimha]?
> Replace starting a separate thread for post entity with event loop in
> TimelineClient
> ------------------------------------------------------------------------------------
>
> Key: YARN-3367
> URL: https://issues.apache.org/jira/browse/YARN-3367
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Junping Du
> Assignee: Naganarasimha G R
> Attachments: YARN-3367.YARN-2928.001.patch
>
>
> Since YARN-3039, we add loop in TimelineClient to wait for
> collectorServiceAddress ready before posting any entity. In consumer of
> TimelineClient (like AM), we are starting a new thread for each call to get
> rid of potential deadlock in main thread. This way has at least 3 major
> defects:
> 1. The consumer need some additional code to wrap a thread before calling
> putEntities() in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting
> operation thread get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side,
> putEntities() only put related entities into a queue of entities and a
> separated thread handle to deliver entities in queue to collector via REST
> call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
