[
https://issues.apache.org/jira/browse/YARN-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648057#comment-14648057
]
Jason Lowe commented on YARN-3942:
----------------------------------
The logs are created based on app attempt. It helps avoid the split-brain,
double-writer issue where the previous attempt is still running when the RM
expires it (e.g.: due to network cut) and decides to launch another. The files
are stored and looked up in a directory that is named after the application ID,
and the entity files within that directory are stored based on application
attempt ID. I don't think the latter is crucial to use the app attempt ID and
the reader is not relying on the attempt ID from those files, but it was a
simple way to avoid colliding with previous attempts and having the reader
process the files in attempt order.
> Timeline store to read events from HDFS
> ---------------------------------------
>
> Key: YARN-3942
> URL: https://issues.apache.org/jira/browse/YARN-3942
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: timelineserver
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: YARN-3942.001.patch
>
>
> This adds a new timeline store plugin that is intended as a stop-gap measure
> to mitigate some of the issues we've seen with ATS v1 while waiting for ATS
> v2. The intent of this plugin is to provide a workable solution for running
> the Tez UI against the timeline server on a large-scale clusters running many
> thousands of jobs per day.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
