[
https://issues.apache.org/jira/browse/YARN-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944028#comment-14944028
]
Sangjin Lee commented on YARN-3367:
-----------------------------------
bq. Yes i agree with this, but what should be the behavior of Sync calls ? IMO
in the wake of YARN-4061 (Fault tolerant writer for timeline v2), we need not
worry abt it either, Thoughts ?
I added a couple of comments to YARN-4061. I think it remains to be seen what
we will choose as the behavior/implementation at the end. But at least I think
it'd be fair to say that there will be a certain type of calls that will need
to trigger flush (and a synchronous wait for the response). Whether we will do
that on the sync side or not, I think we have some flexibility.
{quote}
Well i can understand if all the events are received and timestamp is filled at
the client side we need not worry abt the order but in the case client goes
down send some events out of order ? like containerFinished event gets
Published but logAggregation does not succeed. And from Client App side not
sure how important is the order to be maintained.
{quote}
I think we might be saying the same thing. What I'm saying is that it would not
be practical to ensure the order of events for sync and async writes. As for
the timestamps, I am also arguing that the timestamps should always be set
explicitly for entities/metrics/events, and that the server should rely on the
explicit timestamps, rather than on time of receipt.
> Replace starting a separate thread for post entity with event loop in
> TimelineClient
> ------------------------------------------------------------------------------------
>
> Key: YARN-3367
> URL: https://issues.apache.org/jira/browse/YARN-3367
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Junping Du
> Assignee: Naganarasimha G R
> Attachments: YARN-3367.YARN-2928.001.patch
>
>
> Since YARN-3039, we add loop in TimelineClient to wait for
> collectorServiceAddress ready before posting any entity. In consumer of
> TimelineClient (like AM), we are starting a new thread for each call to get
> rid of potential deadlock in main thread. This way has at least 3 major
> defects:
> 1. The consumer need some additional code to wrap a thread before calling
> putEntities() in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting
> operation thread get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side,
> putEntities() only put related entities into a queue of entities and a
> separated thread handle to deliver entities in queue to collector via REST
> call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
