[
https://issues.apache.org/jira/browse/YARN-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15181608#comment-15181608
]
Sunil G commented on YARN-4764:
-------------------------------
Thanks [~jianhe] for the clarification. Yes, APP_REJECTED exception will be
thrown from {{CapacityScheduler#addApplication}} in this case.
If ACLs are enabled as per current code, then for "queue-not-exist" scenario we
will not be creating an RMApp and directly AccessControlException will be
thrown from {{createAndPopulateRMApp}}. But with ACLs disabled, RMApp will be
created and then APP_REJECTED will be thrown back from scheduler
(reservation/normal case).
So there is a behavioral change exists as per now and this is added in
YARN-4522.
Quoting the "queue-not-exist" error in non-acl scenario
{noformat}
java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to
submit application_1457161239294_0002 to YARN : Application
application_1457161239294_0002 submitted by user root to unknown queue: a1
{noformat}
I think we can make the behavior same in both cases. Also by seeing below code
snippet in {{createAndPopulateRMApp}},
{code}
UserGroupInformation userUgi = UserGroupInformation.createRemoteUser(user);
// Since FairScheduler queue mapping is done inside scheduler,
// if FairScheduler is used and the queue doesn't exist, we should not
// fail here because queue will be created inside FS. Ideally, FS queue
// mapping should be done outside scheduler too like CS.
// For now, exclude FS for the acl check.
if (!isRecovery && YarnConfiguration.isAclEnabled(conf)
{code}
its better we do NOT do a "non-existent-queue" check here. So I think we can
have the current check done in the patch inside below if condition.
{code}
if (!isRecovery && YarnConfiguration.isAclEnabled(conf)
&& scheduler instanceof CapacityScheduler) {
{code}
And if queue doesnt exist, then we can create a clear message like "Application
application_1457161239294_0002 submitted by user root to unknown queue: a1" and
throw YarnException directly. Thoughts?
> Application submission fails when submitted queue is not available in
> scheduler xml
> -----------------------------------------------------------------------------------
>
> Key: YARN-4764
> URL: https://issues.apache.org/jira/browse/YARN-4764
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4764.patch
>
>
> Available queues in capacity scheduler
> -root
> --queue1
> --queue2
> Submit application with queue3
> {noformat}
> 16/03/04 16:40:08 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1457077554812_1901
> 16/03/04 16:40:08 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN,
> Service: ha-hdfs:hacluster, Ident: (HDFS_DELEGATION_TOKEN token 3938 for
> mapred with renewer yarn)
> 16/03/04 16:40:08 WARN retry.RetryInvocationHandler: Exception while invoking
> class
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication
> over rm2. Not retrying because try once and fail.
> java.lang.NullPointerException: java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:366)
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:289)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:618)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:252)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:483)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2305)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2301)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2301)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication(ApplicationClientProtocolPBClientImpl.java:272)
> {noformat}
> Should be queue doesnt exist
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
