[
https://issues.apache.org/jira/browse/MAPREDUCE-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997431#comment-13997431
]
Steve Loughran commented on MAPREDUCE-5887:
-------------------------------------------
This is good, especially when the client is something like a laptop trying to
submit to in-cloud deployments.
One test to try there is what happens when the blocksize is reported as very,
very small (you can configure this in swiftfs). in the client this will cause
the submitting process to OOM and fail. Presumably the same outcome in the AM
is the simplest to implement -we just need to make sure that YARN recognises
this as a failure and only tries a couple of times
> Move split creation from submission client to MRAppMaster
> ---------------------------------------------------------
>
> Key: MAPREDUCE-5887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5887
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster, client
> Affects Versions: 2.4.0
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Attachments: MAPREDUCE-5887.v01.patch
>
>
> This JIRA is filed to improve scalability of job submission, specifically
> when there is a significant latency between the submission client and the
> cluster nodes RM and NN, e.g. in a multi-datacenter environment.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
