[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[ https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16910998#comment-16910998 ] Bilwa S T commented on HADOOP-16391: Thank you [~xkrogen] for review and committing > Duplicate values in rpcDetailedMetrics > -- > > Key: HADOOP-16391 > URL: https://issues.apache.org/jira/browse/HADOOP-16391 > Project: Hadoop Common > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16391-001.patch, HADOOP-16391-002.patch, > HADOOP-16391-003.patch, image-2019-06-25-20-30-15-395.png, screenshot-1.png, > screenshot-2.png > > > In RpcDetailedMetrics init is called two times . Once for deferredRpcrates > and other one rates metrics which causes duplicate values in RM and NM > metrics. > !image-2019-06-25-20-30-15-395.png! -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[ https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909404#comment-16909404 ] Hudson commented on HADOOP-16391: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17137 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17137/]) HADOOP-16391 Add a prefix to the metric names for (xkrogen: rev e356e4f4b70b756667f76c2f9d0d47eb8fceeb9d) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RpcDetailedMetrics.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRatesWithAggregation.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java > Duplicate values in rpcDetailedMetrics > -- > > Key: HADOOP-16391 > URL: https://issues.apache.org/jira/browse/HADOOP-16391 > Project: Hadoop Common > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16391-001.patch, HADOOP-16391-002.patch, > HADOOP-16391-003.patch, image-2019-06-25-20-30-15-395.png, screenshot-1.png, > screenshot-2.png > > > In RpcDetailedMetrics init is called two times . Once for deferredRpcrates > and other one rates metrics which causes duplicate values in RM and NM > metrics. > !image-2019-06-25-20-30-15-395.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908982#comment-16908982
]
Hadoop QA commented on HADOOP-16391:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 45s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 37s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
47s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 93m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16391 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977766/HADOOP-16391-003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d886d4146c6c 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2216ec5 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16486/artifact/out/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16486/testReport/ |
| Max. process+thread count | 1382 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16486/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
T
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[ https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908840#comment-16908840 ] Bilwa S T commented on HADOOP-16391: [~xkrogen] i agree with your points on MutableRate. I have updated patch. Please review > Duplicate values in rpcDetailedMetrics > -- > > Key: HADOOP-16391 > URL: https://issues.apache.org/jira/browse/HADOOP-16391 > Project: Hadoop Common > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: HADOOP-16391-001.patch, HADOOP-16391-002.patch, > image-2019-06-25-20-30-15-395.png, screenshot-1.png, screenshot-2.png > > > In RpcDetailedMetrics init is called two times . Once for deferredRpcrates > and other one rates metrics which causes duplicate values in RM and NM > metrics. > !image-2019-06-25-20-30-15-395.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907568#comment-16907568
]
Erik Krogen commented on HADOOP-16391:
--
Hi [~BilwaST], thanks for continuing to work on this :) The checkstlye and
whitespace issues seem legitimate.
Besides those, I had one thought regarding the changes to {{MutableRate}}. The
new constructor basically makes it no longer a {{MutableRate}}, just a
{{MutableStat}} – the only difference between the two is that {{MutableRate}}
enforces the "Ops"/"Time" names and this constructor removes that convention.
Instead, would it make more sense to leave {{MutableRate}} unchanged and then
do this:
{code:java|title=MutableRatesWithAggregation}
metric = new MutableRate(name + typePrefix, name + typePrefix, false);
{code}
The output will look basically the same, with names like
"GetLongDeferredNumOps" instead of "GetLongNumDeferredOps", and seems a change
more in line with the intent of {{MutableRate}}. What do you think?
Also, the new test looks great, but can we also add an assertion that there is
a call for the deferred method name as well? You also have a typo: "Deferrred"
(three r's)
> Duplicate values in rpcDetailedMetrics
> --
>
> Key: HADOOP-16391
> URL: https://issues.apache.org/jira/browse/HADOOP-16391
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: HADOOP-16391-001.patch, HADOOP-16391-002.patch,
> image-2019-06-25-20-30-15-395.png, screenshot-1.png, screenshot-2.png
>
>
> In RpcDetailedMetrics init is called two times . Once for deferredRpcrates
> and other one rates metrics which causes duplicate values in RM and NM
> metrics.
> !image-2019-06-25-20-30-15-395.png!
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907036#comment-16907036
]
Hadoop QA commented on HADOOP-16391:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 35m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
10s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 42s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 2 new + 17 unchanged - 0 fixed = 19 total (was 17) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 28s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
50s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16391 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977557/HADOOP-16391-002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 795de952d7b5 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 846848a |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16478/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16478/artifact/out/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16478/testReport/ |
| Max. process+thread count | 1480 (vs. ulimit of 5500) |
|
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[ https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906927#comment-16906927 ] Bilwa S T commented on HADOOP-16391: Thanks [~vinayakumarb] [~xkrogen] for reviewing patch. I have updated patch. Please review > Duplicate values in rpcDetailedMetrics > -- > > Key: HADOOP-16391 > URL: https://issues.apache.org/jira/browse/HADOOP-16391 > Project: Hadoop Common > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: HADOOP-16391-001.patch, HADOOP-16391-002.patch, > image-2019-06-25-20-30-15-395.png, screenshot-1.png, screenshot-2.png > > > In RpcDetailedMetrics init is called two times . Once for deferredRpcrates > and other one rates metrics which causes duplicate values in RM and NM > metrics. > !image-2019-06-25-20-30-15-395.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880731#comment-16880731
]
Erik Krogen commented on HADOOP-16391:
--
[~BilwaST] the new changes also look good to me. I agree that it could use a
test, and can we also add Javadoc to {{typePrefix}} and the new constructor for
{{MutableRate}} (explain what sampleName / valueName are)?
> Duplicate values in rpcDetailedMetrics
> --
>
> Key: HADOOP-16391
> URL: https://issues.apache.org/jira/browse/HADOOP-16391
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: HADOOP-16391-001.patch,
> image-2019-06-25-20-30-15-395.png, screenshot-1.png, screenshot-2.png
>
>
> In RpcDetailedMetrics init is called two times . Once for deferredRpcrates
> and other one rates metrics which causes duplicate values in RM and NM
> metrics.
> !image-2019-06-25-20-30-15-395.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16879163#comment-16879163
]
Vinayakumar B commented on HADOOP-16391:
{quote}As I understand it, currently the deferred RPC metrics are basically
unusable because they exactly match the metric names of the non-deferred calls,
making them indistinguishable. I think we can safely change this name
{quote}
I think this is fine.
[~BilwaST] Changes looks good. Can you also add a test to identify to different
metrics are getting generated.?
> Duplicate values in rpcDetailedMetrics
> --
>
> Key: HADOOP-16391
> URL: https://issues.apache.org/jira/browse/HADOOP-16391
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: HADOOP-16391-001.patch,
> image-2019-06-25-20-30-15-395.png, screenshot-1.png, screenshot-2.png
>
>
> In RpcDetailedMetrics init is called two times . Once for deferredRpcrates
> and other one rates metrics which causes duplicate values in RM and NM
> metrics.
> !image-2019-06-25-20-30-15-395.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878396#comment-16878396
]
Hadoop QA commented on HADOOP-16391:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
6s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 14s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
45s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 43s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 40s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
14s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}113m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.09.5 Server=18.09.5 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16391 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12973632/HADOOP-16391-001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6183292eec1d 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / acd2d52 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16369/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16369/artifact/out/whitespace-eol.txt
|
| Test Results |
https
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16874262#comment-16874262
]
Erik Krogen commented on HADOOP-16391:
--
[~vinayakumarb], I see what you mean, that's a good find. Basically the
{{init()}} is setting up metrics for declared methods on the interface, so even
if you don't use the deferred response feature, it will emit a duplicate
metric. Removing the method from the interface just makes the initialization
functionality not work properly, and also I'm hesitant to tamper with any
protocol definitions. I think the right solution here is to make {{init()}}
accept a {{String prefix}} parameter which indicates that the method names on
the protocol should be prefixed. We can reuse the same prefix here as well as
when calling {{add()}} to make the names match.
{quote}
This will be the correct fix. But I am afraid, whether this will be considered
as incompatible change?
Or should we say, user might be picking wrong metric all along because of
duplicate metrics, this will be fixed now and not an 'incompatible' change?
{quote}
As I understand it, currently the deferred RPC metrics are basically unusable
because they exactly match the metric names of the non-deferred calls, making
them indistinguishable. I think we can safely change this name.
> Duplicate values in rpcDetailedMetrics
> --
>
> Key: HADOOP-16391
> URL: https://issues.apache.org/jira/browse/HADOOP-16391
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: image-2019-06-25-20-30-15-395.png, screenshot-1.png,
> screenshot-2.png
>
>
> In RpcDetailedMetrics init is called two times . Once for deferredRpcrates
> and other one rates metrics which causes duplicate values in RM and NM
> metrics.
> !image-2019-06-25-20-30-15-395.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[ https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873835#comment-16873835 ] Bilwa S T commented on HADOOP-16391: Thank you [~vinayakumarb] [~xkrogen] for ur suggestions. I have a doubt. why does this happen even in case of HAServiceProtocol#getServiceStatus. !screenshot-1.png! There is no explicit method declared in HAServiceProtocolPB !screenshot-2.png! > Duplicate values in rpcDetailedMetrics > -- > > Key: HADOOP-16391 > URL: https://issues.apache.org/jira/browse/HADOOP-16391 > Project: Hadoop Common > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: image-2019-06-25-20-30-15-395.png, screenshot-1.png, > screenshot-2.png > > > In RpcDetailedMetrics init is called two times . Once for deferredRpcrates > and other one rates metrics which causes duplicate values in RM and NM > metrics. > !image-2019-06-25-20-30-15-395.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873827#comment-16873827
]
Vinayakumar B commented on HADOOP-16391:
Thanks for hint [~xkrogen].
Yes, duplicate metrics is because of the exact metric names.
But, catch here is that, _deferred_ response approach is never used in any of
the Hadoop Code, yet. Still there are duplicate entries in the metrics.
This issue is not common for all protocols. It comes only for protocols which
have methods declared in them.
Example: 'ContainerManagementProtocolPB' have a method declared explicitly,
even though same method is declared in Protobuf generated _BlockingInterface_.
In this case, explicitly declared method will be added as a metric for
{{deferredRpcRates.init()}}.
{code:java}
public interface ContainerManagementProtocolPB extends
ContainerManagementProtocolService.BlockingInterface {
SignalContainerResponseProto signalToContainer(RpcController arg0,
SignalContainerRequestProto proto) throws ServiceException;
}{code}
This explicit declaration of method in interface is not-necessary IMO. And as a
workaround fix, this will solve the current duplicate metrics.
{quote}We probably need to adjust addDeferredProcessingTime to add a prefix or
suffix to all of the metric names.
{quote}
This will be the correct fix. But I am afraid, whether this will be considered
as incompatible change?
Or should we say, user might be picking wrong metric all along because of
duplicate metrics, this will be fixed now and not an 'incompatible' change?
> Duplicate values in rpcDetailedMetrics
> --
>
> Key: HADOOP-16391
> URL: https://issues.apache.org/jira/browse/HADOOP-16391
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: image-2019-06-25-20-30-15-395.png
>
>
> In RpcDetailedMetrics init is called two times . Once for deferredRpcrates
> and other one rates metrics which causes duplicate values in RM and NM
> metrics.
> !image-2019-06-25-20-30-15-395.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[
https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873444#comment-16873444
]
Erik Krogen commented on HADOOP-16391:
--
Hi [~BilwaST], thanks for reporting this. I took a quick look and as far as I
can tell, this was introduced by HADOOP-11552 which initially introduced the
concept of the deferred metrics. I actually disagree with you that the problem
is calling {{init}} twice; that method simply initializes the local data
structures within {{RpcDetailedMetricsWithAggregation}}. I believe the problem
is that both {{rates}} and {{deferredRpcRates}} use the exact same method
names, thus the metric names overlap. We probably need to adjust
{{addDeferredProcessingTime}} to add a prefix or suffix to all of the metric
names.
> Duplicate values in rpcDetailedMetrics
> --
>
> Key: HADOOP-16391
> URL: https://issues.apache.org/jira/browse/HADOOP-16391
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: image-2019-06-25-20-30-15-395.png
>
>
> In RpcDetailedMetrics init is called two times . Once for deferredRpcrates
> and other one rates metrics which causes duplicate values in RM and NM
> metrics.
> !image-2019-06-25-20-30-15-395.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16391) Duplicate values in rpcDetailedMetrics
[ https://issues.apache.org/jira/browse/HADOOP-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872913#comment-16872913 ] Bilwa S T commented on HADOOP-16391: cc [~sseth] [~cgregori] [~xkrogen] > Duplicate values in rpcDetailedMetrics > -- > > Key: HADOOP-16391 > URL: https://issues.apache.org/jira/browse/HADOOP-16391 > Project: Hadoop Common > Issue Type: Bug >Reporter: Bilwa S T >Priority: Major > Attachments: image-2019-06-25-20-30-15-395.png > > > In RpcDetailedMetrics init is called two times . Once for deferredRpcrates > and other one rates metrics which causes duplicate values in RM and NM > metrics. > !image-2019-06-25-20-30-15-395.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
