[
https://issues.apache.org/jira/browse/YARN-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhijie Shen updated YARN-975:
-----------------------------
Attachment: YARN-975.6.patch
Upload new patch of the new implementation of
FileSystemApplicationHistoryStore. It contains the following major new features:
1. All the history data of one application and its attempts and containers is
going to be stored in only one file. The file is opened when applicationStart
is called and is closed when applicationFinish is called. All the history data
that is to be written after the file is closed will result in exception. The
file is named according to the applicationId.
2. Use TFile data model. We write the following key-value pairs into the file:
* <ApplicationId + START_DATA_SUFFIX, ApplicationStartData>
* <ApplicationId + FINISH_DATA_SUFFIX, ApplicationFinishData>
* <ApplicationAttemptId + START_DATA_SUFFIX, ApplicationAttemptStartData>
* <ApplicationAttemptId + FINISH_DATA_SUFFIX, ApplicationAttemptFinishData>
* <ContainerId + START_DATA_SUFFIX, ContainerStartData>
* <ContainerId + FINISH_DATA_SUFFIX, ContainerFinishData>
3. When reading the history data, we locate the particular file, retrieve the
relevant XXXXStartData and XXXXFinishData, and combine them into a
XXXXHistoryData and return.
> Add a file-system implementation for history-storage
> ----------------------------------------------------
>
> Key: YARN-975
> URL: https://issues.apache.org/jira/browse/YARN-975
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
> Attachments: YARN-975.1.patch, YARN-975.2.patch, YARN-975.3.patch,
> YARN-975.4.patch, YARN-975.5.patch, YARN-975.6.patch
>
>
> HDFS implementation should be a standard persistence strategy of history
> storage
--
This message was sent by Atlassian JIRA
(v6.1#6144)
