[
https://issues.apache.org/jira/browse/YARN-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364693#comment-14364693
]
zhihai xu commented on YARN-3241:
---------------------------------
I uploaded a patch YARN-3241.000.patch for review. Without the patch, the test
will fail with MetricsException: Metrics source QueueMetrics,q0=root,q1=A
already exists!
{code}
------------------------------------------------------
T E S T S
-------------------------------------------------------
Running
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
Tests run: 87, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.887 sec <<<
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
testQueueNameWithTrailingSpace(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler)
Time elapsed: 0.036 sec <<< ERROR!
org.apache.hadoop.metrics2.MetricsException: Metrics source
QueueMetrics,q0=root,q1=A already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:143)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:120)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:111)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.<init>(FSQueue.java:63)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.<init>(FSLeafQueue.java:75)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.createQueue(QueueManager.java:185)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getQueue(QueueManager.java:131)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getLeafQueue(QueueManager.java:88)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.assignToQueue(FairScheduler.java:699)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:601)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1182)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueNameWithTrailingSpace(TestFairScheduler.java:4466)
Results :
Tests in error:
TestFairScheduler.testQueueNameWithTrailingSpace:4466 ยป Metrics Metrics
source...
Tests run: 87, Failures: 0, Errors: 1, Skipped: 0
{code}
> Leading space, trailing space and empty sub queue name may cause
> MetricsException for fair scheduler
> ----------------------------------------------------------------------------------------------------
>
> Key: YARN-3241
> URL: https://issues.apache.org/jira/browse/YARN-3241
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Reporter: zhihai xu
> Assignee: zhihai xu
> Attachments: YARN-3241.000.patch
>
>
> Leading space, trailing space and empty sub queue name may cause
> MetricsException(Metrics source XXX already exists! ) when add application to
> FairScheduler.
> The reason is because QueueMetrics parse the queue name different from the
> QueueManager.
> QueueMetrics use Q_SPLITTER to parse queue name, it will remove Leading space
> and trailing space in the sub queue name, It will also remove empty sub queue
> name.
> {code}
> static final Splitter Q_SPLITTER =
> Splitter.on('.').omitEmptyStrings().trimResults();
> {code}
> But QueueManager won't remove Leading space, trailing space and empty sub
> queue name.
> This will cause out of sync between FSQueue and FSQueueMetrics.
> QueueManager will think two queue names are different so it will try to
> create a new queue.
> But FSQueueMetrics will treat these two queue names as same queue which will
> create "Metrics source XXX already exists!" MetricsException.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
