[ 
https://issues.apache.org/jira/browse/YARN-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944009#comment-14944009
 ] 

Sangjin Lee commented on YARN-4061:
-----------------------------------

Another thing to consider is also the throughput of writing to filesystems 
(local or hdfs). This may or may not be a big problem for app-level timeline 
collector, but it would certainly be something we need to analyze rigorously 
for the RM timeline collector. If we go the route of writing all writes to 
disk, then we should ensure that we can sustain the throughput for the RM 
collector of a very large cluster (> 10,000 nodes, a large number of apps being 
created).

> [Fault tolerance] Fault tolerant writer for timeline v2
> -------------------------------------------------------
>
>                 Key: YARN-4061
>                 URL: https://issues.apache.org/jira/browse/YARN-4061
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: FaulttolerantwriterforTimelinev2.pdf
>
>
> We need to build a timeline writer that can be resistant to backend storage 
> down time and timeline collector failures. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to