[
https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287387#comment-15287387
]
zhihai xu commented on MAPREDUCE-6696:
--------------------------------------
All these test failures are not related to my changes:
TestMRCJCFileOutputCommitter is passed in my local build also the failure for
TestMRCJCFileOutputCommitter is due to test environment problem:
2016-05-17 17:29:33,792 WARN [main] util.NativeCodeLoader
(NativeCodeLoader.java:<clinit>(60)) - Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable.
TestMiniMRChildTask and TestMiniMRChildTask.testTaskOldEnv failure happened at
launchContainer which already pass job submission phase after my code change.
2016-05-17 17:45:48,781 WARN [ContainersLauncher #1]
nodemanager.DefaultContainerExecutor
(DefaultContainerExecutor.java:launchContainer(245)) - Exception from
container-launch with container ID: container_1463507138005_0001_01_000002 and
exit code: 127
ExitCodeException exitCode=127: nice: bash: No such file or directory
at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
at org.apache.hadoop.util.Shell.run(Shell.java:850)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
> Add a configuration to limit the number of map tasks allowed per job.
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-6696
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: job submission
> Affects Versions: 2.8.0
> Reporter: zhihai xu
> Assignee: zhihai xu
> Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks
> allowed per job. It will be useful for Hadoop admin to save Hadoop cluster
> resource by preventing users from submitting big mapreduce jobs. A mapredeuce
> job with too many mappers may fail with OOM after running for long time. It
> will be a big waste.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
