[
https://issues.apache.org/jira/browse/MAPREDUCE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated MAPREDUCE-5583:
----------------------------------------
Release Note:
<!-- markdown -->
This introduces two new MR2 job configs, mentioned below, which allow users to
control the maximum simultaneously-running tasks of the submitted job, across
the cluster:
* mapreduce.job.running.map.limit (default: 0, for no limit)
* mapreduce.job.running.reduce.limit (default: 0, for no limit)
This is controllable at a per-job level.
was:
This introduces two new MR2 job configs, mentioned below, which allow users to
control the maximum simultaneously-running tasks of the submitted job, across
the cluster:
* mapreduce.job.running.map.limit (default: 0, for no limit)
* mapreduce.job.running.reduce.limit (default: 0, for no limit)
This is controllable at a per-job level.
> Ability to limit running map and reduce tasks
> ---------------------------------------------
>
> Key: MAPREDUCE-5583
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5583
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mr-am, mrv2
> Affects Versions: 0.23.9, 2.1.1-beta
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 2.7.0
>
> Attachments: MAPREDUCE-5583-branch2.4.1.patch,
> MAPREDUCE-5583v1.patch, MAPREDUCE-5583v2.patch, MAPREDUCE-5583v3.patch,
> MAPREDUCE-5583v4.patch
>
>
> It would be nice if users could specify a limit to the number of map or
> reduce tasks that are running simultaneously. Occasionally users are
> performing operations in tasks that can lead to DDoS scenarios if too many
> tasks run simultaneously (e.g.: accessing a database, web service, etc.).
> Having the ability to throttle the number of tasks simultaneously running
> would provide users a way to mitigate issues with too many tasks on a large
> cluster attempting to access a serivce at any one time.
> This is similar to the functionality requested by MAPREDUCE-224 and
> implemented by HADOOP-3412 but was dropped in mrv2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
