[
https://issues.apache.org/jira/browse/YARN-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943999#comment-14943999
]
Sangjin Lee commented on YARN-4061:
-----------------------------------
Sorry it took me a while to get to this. Thanks for the proposal [~gtCarrera9]!
One potential area of concern is regarding flush and the associated contract
with the client. If the client wanted to write critical data synchronously and
specifically wanted to block until it receives a server (storage) response, how
would that work in this scheme? The proposal has the following for the flush
situation:
{quote}
When one log segment reaches a predefined size, or a time trigger, or an
explicit flush call happens, it is published to a log queue.
{quote}
Since the actual storage writer (HBase) always acts on this queue
asynchronously, it seems that the client cannot have a synchronous write
semantics. Is that a correct reading? If so, how would we implement such a
synchronous write?
> [Fault tolerance] Fault tolerant writer for timeline v2
> -------------------------------------------------------
>
> Key: YARN-4061
> URL: https://issues.apache.org/jira/browse/YARN-4061
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Li Lu
> Assignee: Li Lu
> Attachments: FaulttolerantwriterforTimelinev2.pdf
>
>
> We need to build a timeline writer that can be resistant to backend storage
> down time and timeline collector failures.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
