[
https://issues.apache.org/jira/browse/YARN-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhijie Shen updated YARN-975:
-----------------------------
Attachment: YARN-975.8.patch
[~mayank_bansal], thanks for review. I've uploaded a new patch accordingly.
bq. Yarn-default.xml yarn.ahs.fs-history-store.uri should be defaulted to
${hadoop.log.dir}
Addressed
bq. We need to have meta deta for the applications to be stored on the fs , I
think keeping the way JHS , keeping it on the file name is useful
The meda data of the applications are stored in the history file. If in the
further, we'd like to support filters for getAllApplications(), we can either
append the related fields into the history file name to speed up the filtering,
or write the index fields into a separate meta data file, which can even be
loaded into the memory when FS history store is started.
Anyway, the current API's doesn't need the meta data. We can incrementally
change current implementation whenever we've confirmed the filters we what to
add.
bq. In service Init why we are using FS_RM_STATE_STORE_URI?
Addressed
bq. We also need to change the permissions for the root dir as all the users
are going to write the AHS store, so it should be writable by all the users.
Thanks for pointing out the file permission problem. Instead of allowing all
users to access the history directory and files, we should only give the
writing access for RM user (dir needs the execution access as well), and
reading access for RM user and the users in the same group of RM user.
This is because RM is supposed to be the only one to write the history file,
not all users, while AHS is supposed to be the only one to read the history
file.
bq. Why we are adding historyDataMap.put(appId, null); ? WHy we cant just don't
even add the app id in the history map?
Addressed
> Add a file-system implementation for history-storage
> ----------------------------------------------------
>
> Key: YARN-975
> URL: https://issues.apache.org/jira/browse/YARN-975
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
> Attachments: YARN-975.1.patch, YARN-975.2.patch, YARN-975.3.patch,
> YARN-975.4.patch, YARN-975.5.patch, YARN-975.6.patch, YARN-975.7.patch,
> YARN-975.8.patch
>
>
> HDFS implementation should be a standard persistence strategy of history
> storage
--
This message was sent by Atlassian JIRA
(v6.1#6144)
