[
https://issues.apache.org/jira/browse/YARN-934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhijie Shen updated YARN-934:
-----------------------------
Attachment: YARN-934.1.patch
Push my first draft of HistoryWriter out. In general, it is an abstract class,
which have three protected abstract methods. The three abstract methods need to
be implemented to define the actual logic of storing RMApp, RMAppAttempt and
RMContainer's information into memory, HDFS and so on.
There're three other public methods (the APIs), which can be called by RM to
schedule the writing tasks. The APIs are therefore asynchronized.
Multi-threading model is adopted here, because I expect the writing history
tasks will be I/O intensive.
Please have a look give some feedback.
In addition, I'm still thinking about the param for the three protected
abstract class. It seems better to define the classes which contains the exact
fields to be persist. Raw RMApp, RMAppAttempt and RMContainer will confuse the
implementations which records to persist.
Moreover, I'm thinking about how the worker thread throwable should be
processed. The current idea is to collect and process them at the thread of
HistoryWriter.
Finally, I'm not sure we want to generalize the framework enough now in case we
want to store per-framework data later. Thoughts?
> Defining Writing Interface of HistoryStorage
> --------------------------------------------
>
> Key: YARN-934
> URL: https://issues.apache.org/jira/browse/YARN-934
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
> Attachments: YARN-934.1.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
