[
https://issues.apache.org/jira/browse/YARN-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13795706#comment-13795706
]
Hitesh Shah commented on YARN-1289:
-----------------------------------
bq. There already are some mapreduce related configurations and even with
default values there in yarn-default.xml.
Could you file jiras for the the ones that you have come across so that such
issues can be addressed and fixed.
bq. I don't see any problem there as default value is easily get overridden. Am
I missing something here?
Also, overriding a default value implies an incompatible change.
I agree that everyone has to configure the shuffle service but it is common
knowledge ( atleast I hope it is given that YARN has been around for quite some
time ) that it needs to be configured when using MR with YARN. The
mapreduce.shuffle issue was a recent change and has probably caused a few
problems to quite a few users. But it was a one-off issue and introducing a
default as a result does not seem the right solution. The fact that everything
is packaged/released together ( as of today ) does not imply we should enforce
more tighter dependencies. MR is already moving towards becoming a user-land
library ( MAPREDUCE-4421 ) although the aux-service question has not been
addressed yet.
> Configuration "yarn.nodemanager.aux-services" should have default value for
> mapreduce_shuffle.
> ----------------------------------------------------------------------------------------------
>
> Key: YARN-1289
> URL: https://issues.apache.org/jira/browse/YARN-1289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Reporter: wenwupeng
> Assignee: Junping Du
> Attachments: YARN-1289.patch
>
>
> Failed to run benchmark when not configure yarn.nodemanager.aux-services
> value in yarn-site.xml', it is better to configure default value.
> 13/10/09 22:19:23 INFO mapreduce.Job: Task Id :
> attempt_1381371516570_0001_m_000000_1, Status : FAILED
> Container launch failed for container_1381371516570_0001_01_000005 :
> org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The
> auxService:mapreduce_shuffle does not exist
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:152)
> at
> org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
