[jira] [Commented] (YARN-10456) RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry

2021-07-15 Thread Qi Zhu (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381238#comment-17381238
 ] 

Qi Zhu commented on YARN-10456:
---

Thanks [~epayne] for this work.

The patch LGTM +1.

> RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics 
> registry
> -
>
> Key: YARN-10456
> URL: https://issues.apache.org/jira/browse/YARN-10456
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: resourcemanager
>Affects Versions: 3.3.0, 3.2.1, 3.1.4, 2.10.1
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
> Attachments: YARN-10456.001.patch
>
>
> Several queue metrics (such as AppsRunning, PendingContainers, etc.) stopped 
> working after we upgraded to 2.10.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10456) RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry

2021-07-14 Thread Jim Brennan (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380864#comment-17380864
 ] 

Jim Brennan commented on YARN-10456:


Thanks [~epayne]!   The patch looks good and it matches the change we have been 
running with internally.

I am +1 on this and I will commit tomorrow if there are no objections.

> RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics 
> registry
> -
>
> Key: YARN-10456
> URL: https://issues.apache.org/jira/browse/YARN-10456
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: resourcemanager
>Affects Versions: 3.3.0, 3.2.1, 3.1.4, 2.10.1
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
> Attachments: YARN-10456.001.patch
>
>
> Several queue metrics (such as AppsRunning, PendingContainers, etc.) stopped 
> working after we upgraded to 2.10.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10456) RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry

2021-07-14 Thread Eric Payne (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380826#comment-17380826
 ] 

Eric Payne commented on YARN-10456:
---

[~Jim_Brennan], [~ebadger], [~edfi202], [~prabhujoseph], [~BilwaST], [~snemeth] 
:
Would someone be willing to review this? Thanks!

> RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics 
> registry
> -
>
> Key: YARN-10456
> URL: https://issues.apache.org/jira/browse/YARN-10456
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: resourcemanager
>Affects Versions: 3.3.0, 3.2.1, 3.1.4, 2.10.1
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
> Attachments: YARN-10456.001.patch
>
>
> Several queue metrics (such as AppsRunning, PendingContainers, etc.) stopped 
> working after we upgraded to 2.10.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10456) RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry

2021-07-14 Thread Hadoop QA (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380507#comment-17380507
 ] 

Hadoop QA commented on YARN-10456:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 18m  
0s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green}{color} | {color:green} No case conflicting files 
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green}{color} | {color:green} The patch does not contain any 
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green}  0m  0s{color} 
| {color:green}test4tests{color} | {color:green} The patch appears to include 1 
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
10s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green}{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
50s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
59s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m 58s{color} | {color:green}{color} | {color:green} branch has no errors when 
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green}{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
40s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 18m 
14s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are 
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green}  1m 
49s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
53s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green}{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
56s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green}{color} | {color:green} the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
48s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green}{color} | {color:green} The patch has no whitespace 
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m 44s{color} | {color:green}{color} | {color:green} patch has no errors when 
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green}{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green}{color} | {color:green} the patch passed

[jira] [Commented] (YARN-10456) RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry

2020-10-16 Thread Eric Payne (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215416#comment-17215416
 ] 

Eric Payne commented on YARN-10456:
---

Both {{CSQueueMetrics}} and {{FSQueueMetrics}} are also children of 
{{QueueMetrics}}, and when those classes output {{QueueMetrics}} records, their 
RECORDNAME is {{QueueMetrics}}. These two child classes are mutually exclusive 
since Capacity Scheduler and Fair Scheduler will never be running at the same 
time. However, what makes {{PartitionQueueMetrics}} different is that it will 
be writing out records intermingled with {{QueueMetrics}}. If they both have 
the same RECORDNAME, it confuses the Simon reader.

> RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics 
> registry
> -
>
> Key: YARN-10456
> URL: https://issues.apache.org/jira/browse/YARN-10456
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: resourcemanager
>Affects Versions: 3.3.0, 3.2.1, 3.1.4, 2.10.1
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
>
> Several queue metrics (such as AppsRunning, PendingContainers, etc.) stopped 
> working after we upgraded to 2.10.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10456) RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry

2020-10-09 Thread Eric Payne (Jira) 


[ 
https://issues.apache.org/jira/browse/YARN-10456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211439#comment-17211439
 ] 

Eric Payne commented on YARN-10456:
---

We use hadoop-metrics2.properties to set up Simon metrics aggregation.

The format of the output of the RM aggregation metrics begins with:
{noformat}
EPOCH CONTEXT.RECORDNAME ...
{noformat}
Here, {{CONTEXT=yarn}} and {{RECORDNAME=QueueMetrics}} for both 
{{QueueMetrics}} and {{PartitionQueueMetrics}}. This is incorrect and is 
confusing the Simon aggregator and causing the numbers for several metrics to 
be incorrect.

The {{RECORDNAME}} is coming from the {{MetricsInfo}} object in the 
{{MetricsRegistry}} in each {{*Metrics}} class. In this case, 
{{PartitionQueueMetrics}} is a child of the {{QueueMetrics}} class, and when 
{{PartitionQueueMetrics}} is constructed, the {{MetricsInfo}} name for 
{{PartitionQueueMetrics}} is assigned "{{QueueMetrics}}" instead of 
"{{PartitionQueueMetrics}}".

> RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics 
> registry
> -
>
> Key: YARN-10456
> URL: https://issues.apache.org/jira/browse/YARN-10456
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: resourcemanager
>Affects Versions: 3.3.0, 3.2.1, 3.1.4, 2.10.1
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
>
> Several queue metrics (such as AppsRunning, PendingContainers, etc.) stopped 
> working after we upgraded to 2.10.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org