Sangjin Lee commented on YARN-4061:

Sorry it took me a while to get to this. Thanks for the proposal [~gtCarrera9]!

One potential area of concern is regarding flush and the associated contract 
with the client. If the client wanted to write critical data synchronously and 
specifically wanted to block until it receives a server (storage) response, how 
would that work in this scheme? The proposal has the following for the flush 
When one log segment reaches a predefined size, or a timeĀ­ trigger, or an 
explicit flush call happens, it is published to a log queue.

Since the actual storage writer (HBase) always acts on this queue 
asynchronously, it seems that the client cannot have a synchronous write 
semantics. Is that a correct reading? If so, how would we implement such a 
synchronous write?

> [Fault tolerance] Fault tolerant writer for timeline v2
> -------------------------------------------------------
>                 Key: YARN-4061
>                 URL: https://issues.apache.org/jira/browse/YARN-4061
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: FaulttolerantwriterforTimelinev2.pdf
> We need to build a timeline writer that can be resistant to backend storage 
> down time and timeline collector failures. 

This message was sent by Atlassian JIRA

Reply via email to