[
https://issues.apache.org/jira/browse/MAPREDUCE-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903892#comment-16903892
]
Eric Payne commented on MAPREDUCE-7227:
---------------------------------------
[~luoyuan], thank you for raising this issue and bringing it to our attention.
The fix in patch 004 doesn't look quite right to me. I feel that there should
be a way to solve this within the MRAppMaster. Please give me some time to
investigate.
> Fix job staging directory residual problem in a big yarn cluster composed of
> multiple independent hdfs clusters
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7227
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7227
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mrv2
> Affects Versions: 2.6.0, 2.7.0, 3.1.2
> Reporter: Yuan LUO
> Assignee: Yuan LUO
> Priority: Major
> Attachments: HADOOP-MAPREDUCE-7227.001.patch,
> HADOOP-MAPREDUCE-7227.002.patch, HADOOP-MAPREDUCE-7227.003.patch,
> HADOOP-MAPREDUCE-7227.004.patch
>
>
> Our yarn cluster is made up of some independent hdfs cluster, the
> 'default.FS' in every hdfs cluster is different.when user submit job to yarn
> cluster, if the 'default.FS' between client and nodemanager is
> inconsistent, then the job staging dir can't be cleanup by AppMaster. Because
> it will produce two job staging dirs in our conditions by client and
> appmaster. So we can modify AppMaster through client's ‘default.FS’ to
> create job staging dir.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
