[
https://issues.apache.org/jira/browse/MAPREDUCE-6703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated MAPREDUCE-6703:
-----------------------------------
Attachment: MAPREDUCE-6703.001.patch
Uploading initial patch
* Changes to allow the MRAppMaster to request a mix of *OPPORTUNISTIC* and
*GUARANTEED* containers for it first attempt map tasks (reduce tasks and maps
to be retried are always GUARANTEED)
* User specifies number of *OPPORTUNISTIC* Containers via the
*mapreduce.job.num-opportunistic-maps-per-100* job conf : Default value is 0,
which implies all maps will be guaranteed. A value of 100 means all maps will
be requested as opportunistic. For any other value say _x_, the *FIRST* _x_
maps requested by the AM will be opportunistic. If the total number of maps for
the job < _x_, then *ALL* maps will be opportunistic
* The patch also contains a test case {{TestMROpportunisticMaps}} that uses the
*MiniYARNCluster* to run a simple external sorting MR job using the above flag.
*PS* : This feature currently requires Distributed Scheduling (YARN-2885), NM
Queuing (YARN-2883) and the AMRMProxyService (YARN-2884) to be turned on in the
yarn cluster.
> Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-6703
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6703
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Arun Suresh
> Assignee: Arun Suresh
> Attachments: MAPREDUCE-6703.001.patch
>
>
> YARN-2882 and YARN-4335 introduces the concept of container ExecutionTypes
> and specifically OPPORTUNISTIC containers.
> The default ExecutionType is GUARANTEED. This JIRA proposes to allow users to
> provide hints via config to the MR framework as to the number of containers
> it would like to schedule as OPPORTUNISTIC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
