[jira] [Comment Edited] (MAPREDUCE-7274) Enable to set running task limit when mapreduce job is running
[ https://issues.apache.org/jira/browse/MAPREDUCE-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084662#comment-17084662 ] Chengwei Wang edited comment on MAPREDUCE-7274 at 4/16/20, 3:01 PM: Upload patch for hadoop 3.2.0. It enable us to set task running limit when job is running by mapred client command: {code:bash} mapred job -set-running-task-limit JOB_ID TASK_TYPE LIMIT e.g. mapred job -set-running-task-limit job_1583809537551_21297 MAP 100 mapred job -set-running-task-limit job_1583809537551_21297 REDUCE 100 {code} it send a rpc resquest to AM, and AM would update the max running limit of the specified type task as specified limit count. was (Author: smarthan): Upload patch for hadoop 3.2.0. It enable us to set task running limit when job is running by mapred client command: {code:bash} mapred job -set-running-task-limit JOB_ID TASK_TYPE LIMIT e.g. mapred job -set-running-task-limit job_1583809537551_21297 MAP 100 mapred job -set-running-task-limit job_1583809537551_21297 REDUCE 100 {code} it send a rpc resquest to AM, and AM would update the max running limit of the specified type task as specified limit count. > Enable to set running task limit when mapreduce job is running > -- > > Key: MAPREDUCE-7274 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7274 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: mr-am, mrv2 >Reporter: Chengwei Wang >Priority: Major > Attachments: MAPREDUCE-7274-branch-3.2.0.patch > > > MRv2 enabled users to control the number of map or reduce tasks running > simultaneously by configuration *_mapreduce.job.running.map.limit_* or > _*mapreduce.job.running.reduce.limit*._ But users can only set limit number > before submitting the job to rm. So, it's meaningful to enable users to set > the limit of running map or reduce tasks when job is running, which can help > users to restrict resource usage of job and give resources to high-priority > job. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-7274) Enable to set running task limit when mapreduce job is running
[ https://issues.apache.org/jira/browse/MAPREDUCE-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084662#comment-17084662 ] Chengwei Wang edited comment on MAPREDUCE-7274 at 4/16/20, 11:15 AM: - Upload patch for hadoop 3.2.0. It enable us to set task running limit when job is running by mapred client command: {code:bash} mapred job -set-running-task-limit JOB_ID TASK_TYPE LIMIT e.g. mapred job -set-running-task-limit job_1583809537551_21297 MAP 100 mapred job -set-running-task-limit job_1583809537551_21297 REDUCE 100 {code} it send a rpc resquest to AM, and AM would update the max running limit of the specified type task as specified limit count. was (Author: smarthan): Upload patch for hadoop 3.2.0. It enable us to set task running limit when job is running by mapred client command: {code:bash} mapred job -set-running-task-limit JOB_ID TASK_TYPE LIMIT e.g. mapred job -set-running-task-limit job_1583809537551_21297 MAP 100 mapred job -set-running-task-limit job_1583809537551_21297 REDUCE 100 {code} This command would send a rpc resquest to AM, and AM would update the max running limit of the specified type task as specified limit count. > Enable to set running task limit when mapreduce job is running > -- > > Key: MAPREDUCE-7274 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7274 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: mr-am, mrv2 >Reporter: Chengwei Wang >Priority: Major > Attachments: MAPREDUCE-7274-branch-3.2.0.patch > > > MRv2 enabled users to control the number of map or reduce tasks running > simultaneously by configuration _mapreduce.job.running.map.limit_ or > _mapreduce.job.running.reduce.limit._ But users can only set limit number > before submitting the job to rm. So, it's meaningful to enable users to set > the limit of running map or reduce tasks when job is running, which can help > users to restrict resource usage of job and give resources to high-priority > job. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-7274) Enable to set running task limit when mapreduce job is running
[ https://issues.apache.org/jira/browse/MAPREDUCE-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084199#comment-17084199 ] Chengwei Wang edited comment on MAPREDUCE-7274 at 4/16/20, 8:04 AM: I would upload a patch for hadoop 2.x or 3.2.0 if needed. was (Author: smarthan): I would upload a patch for hadoop 2.x or 3.2.0 if needed. > Enable to set running task limit when mapreduce job is running > -- > > Key: MAPREDUCE-7274 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7274 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Chengwei Wang >Priority: Major > Attachments: MAPREDUCE-7274-branch-3.2.0.patch > > > MRv2 enabled users to control the number of map or reduce tasks running > simultaneously by configuration _mapreduce.job.running.map.limit_ or > _mapreduce.job.running.reduce.limit._ But users can only set limit number > before submitting the job to rm. So, it's meaningful to enable users to set > the limit of running map or reduce tasks when job is running, which can help > users to restrict resource usage of job and give resources to high-priority > job. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-7274) Enable to set running task limit when mapreduce job is running
[ https://issues.apache.org/jira/browse/MAPREDUCE-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084199#comment-17084199 ] Chengwei Wang edited comment on MAPREDUCE-7274 at 4/16/20, 8:03 AM: I would upload a patch for hadoop 2.x or 3.2.0 if needed. was (Author: smarthan): I would upload a patch based on hadoop 2.x if needed. > Enable to set running task limit when mapreduce job is running > -- > > Key: MAPREDUCE-7274 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7274 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Chengwei Wang >Priority: Major > Attachments: MAPREDUCE-7274-branch-3.2.0.patch > > > MRv2 enabled users to control the number of map or reduce tasks running > simultaneously by configuration _mapreduce.job.running.map.limit_ or > _mapreduce.job.running.reduce.limit._ But users can only set limit number > before submitting the job to rm. So, it's meaningful to enable users to set > the limit of running map or reduce tasks when job is running, which can help > users to restrict resource usage of job and give resources to high-priority > job. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org