[
https://issues.apache.org/jira/browse/FLINK-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959371#comment-16959371
]
Yang Wang commented on FLINK-14511:
-----------------------------------
So we are trying to get the conclusion.
For capacity scheduler, `YarnClusterDescriptor#checkYarnQueues` works well.
For fair scheduler, if we use absolute queue path `-yqu root.a.product`, it
also works well. The only case is using short queue name `-yqu product`, the
warning log will show. I suggest to use absolute queue path in fair scheduler,
because there is no guarantee that short queue name is unique and flink
application could be submitted successfully.
No matter what is the scheduler, we could submit a flink application
successfully when the queue does not exist. It is based on the policy of yarn
scheduler. So i think the warning log is reasonable if "Available queues: xxx,
yyy" could be found at the same time. [~Dillon.] could you please help to check
the warning log in fair scheduler?
[~trohrmann] How do you think?
> Checking YARN queues should add "root" prefix
> ---------------------------------------------
>
> Key: FLINK-14511
> URL: https://issues.apache.org/jira/browse/FLINK-14511
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Reporter: Zhanchun Zhang
> Assignee: Zhanchun Zhang
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> As we all know, all queues in the YARN cluster are children of the "root"
> queue. While submitting an application to "root.product" queue with -qu
> product parameter, the client logs that "The specified queue 'product' does
> not exist. Available queues....". But this queue is exist and we can still
> submit application to YARN cluster, which is confusing for users. So I think
> that when checking queues should add "root." prefix to the queue name.
> {code:java}
> List<QueueInfo> queues = yarnClient.getAllQueues();
> if (queues.size() > 0 && this.yarnQueue != null) { // check only if there are
> queues configured in yarn and for this session.
> boolean queueFound = false;
> for (QueueInfo queue : queues) {
> if (queue.getQueueName().equals(this.yarnQueue) {
> queueFound = true;
> break;
> }
> }
> if (!queueFound) {
> String queueNames = "";
> for (QueueInfo queue : queues) {
> queueNames += queue.getQueueName() + ", ";
> }
> LOG.warn("The specified queue '" + this.yarnQueue + "' does not
> exist. " +
> "Available queues: " + queueNames);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
