[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16382368#comment-16382368
]
Hadoop QA commented on HBASE-18409:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-18409 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889896/0004-HBASE-18409-MetricsConnection-client-metrics-migration.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11746/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Migrate Client Metrics from codahale to hbase-metrics
> -
>
> Key: HBASE-18409
> URL: https://issues.apache.org/jira/browse/HBASE-18409
> Project: HBase
> Issue Type: Improvement
> Components: Client, java, metrics
>Affects Versions: 3.0.0
>Reporter: Ronald Macmaster
>Priority: Major
> Labels: newbie
> Fix For: 2.0.0
>
> Attachments:
> 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch,
> 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch,
> 0003-HBASE-18409-MetricsConnection-client-metrics-migration.patch,
> 0004-HBASE-18409-MetricsConnection-client-metrics-migration.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Currently, the metrics for hbase-client are tailored for reporting via a
> client-side JMX server.
> The MetricsConnection handles the metrics management and reporting via the
> metrics platform from codahale.
> This approach worked well for hbase-1.3.1 when the metrics platform was still
> relatively young, but it could be improved by using the new
> hbase-metrics-api.
> Now that we have an actual hbase-metrics-api that master, regionserver,
> zookeeper, and other daemons use, it would be good to also allow the client
> to leverage the metrics-api.
> Then, the client could also report its metrics via Hadoop's metrics2 if
> desired or through another platform that utilizes the hbase-metrics-api.
> If left alone, client metrics will continue to be only barely visible through
> a client-side JMX server.
> The migration to the new metrics-api could be done by simply changing the
> Metrics data types from codahale types to hbase-metrics types without
> changing the metrics signatures of MetricsConnection unless completely
> necessary.
> The codahale MetricsRegistry would also have to be exchanged for a
> hbase-metrics MetricsRegistry.
> I found this to be a necessary change after attempting to implement my own
> Reporter to use within the MetricsConnection class.
> I was attempting to create a HadoopMetrics2Reporter that extends the codahale
> ScheduledReporter and reports the MetricsConnection metrics to Hadoop's
> metrics2 system.
> The already existing infrastructure in the hbase-metrics and
> hbase-metrics-api projects could be easily leveraged for a cleaner solution.
> If completed successfully, users could instead access their client-side
> metrics through the hbase-metrics-api.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190728#comment-16190728 ] Josh Elser commented on HBASE-18409: I wouldn't worry about it ;) > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch, > 0003-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0004-HBASE-18409-MetricsConnection-client-metrics-migration.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190633#comment-16190633 ] Ronald Macmaster commented on HBASE-18409: -- No problem [~elserj]. I appreciate the help, and I'm sorry for the delay. Would you like me to submit another patch with the whitespace fix? I was hesitant to pollute the thread with too many Hudson reports xD > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch, > 0003-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0004-HBASE-18409-MetricsConnection-client-metrics-migration.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188390#comment-16188390 ] Josh Elser commented on HBASE-18409: Thanks [~qwerty2.0]! Rebased patch looks like it did the trick * You have one whitespace nit to clean up (we can probably fix that on commit, honestly). * I don't think the test-failures are your fault. I think the unit-tests timed out which caused these to get reported as "failures" when it's more that they simply didn't get run. I have a note to follow up on the timed out tests on HBASE-18891. I'll try to circle back around here when I get to the bottom of that. > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch, > 0003-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0004-HBASE-18409-MetricsConnection-client-metrics-migration.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187597#comment-16187597
]
Hadoop QA commented on HBASE-18409:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
23s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 70m 53s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.client.TestAsyncReplicationAdminApi |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| |
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187527#comment-16187527 ] Josh Elser commented on HBASE-18409: Hey [~qwerty2.0], thanks for making the time to turn around the fixes! I'm trying to piece together what your third patch is. It looks like this is a delta on top of your 2nd patch to me, but I'm not certain. Assuming that is the case, could you rebase and squash your changes on top of the current master branch and put up a new patch? If you're busy, I'll try to do it Monday (assuming I can figure out the inevitable conflicts ;)). > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch, > 0003-HBASE-18409-MetricsConnection-client-metrics-migration.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187325#comment-16187325
]
Hadoop QA commented on HBASE-18409:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-18409 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889881/0003-HBASE-18409-MetricsConnection-client-metrics-migration.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8887/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Migrate Client Metrics from codahale to hbase-metrics
> -
>
> Key: HBASE-18409
> URL: https://issues.apache.org/jira/browse/HBASE-18409
> Project: HBase
> Issue Type: Improvement
> Components: Client, java, metrics
>Affects Versions: 3.0.0
>Reporter: Ronald Macmaster
> Labels: newbie
> Fix For: 3.0.0
>
> Attachments:
> 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch,
> 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch,
> 0003-HBASE-18409-MetricsConnection-client-metrics-migration.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Currently, the metrics for hbase-client are tailored for reporting via a
> client-side JMX server.
> The MetricsConnection handles the metrics management and reporting via the
> metrics platform from codahale.
> This approach worked well for hbase-1.3.1 when the metrics platform was still
> relatively young, but it could be improved by using the new
> hbase-metrics-api.
> Now that we have an actual hbase-metrics-api that master, regionserver,
> zookeeper, and other daemons use, it would be good to also allow the client
> to leverage the metrics-api.
> Then, the client could also report its metrics via Hadoop's metrics2 if
> desired or through another platform that utilizes the hbase-metrics-api.
> If left alone, client metrics will continue to be only barely visible through
> a client-side JMX server.
> The migration to the new metrics-api could be done by simply changing the
> Metrics data types from codahale types to hbase-metrics types without
> changing the metrics signatures of MetricsConnection unless completely
> necessary.
> The codahale MetricsRegistry would also have to be exchanged for a
> hbase-metrics MetricsRegistry.
> I found this to be a necessary change after attempting to implement my own
> Reporter to use within the MetricsConnection class.
> I was attempting to create a HadoopMetrics2Reporter that extends the codahale
> ScheduledReporter and reports the MetricsConnection metrics to Hadoop's
> metrics2 system.
> The already existing infrastructure in the hbase-metrics and
> hbase-metrics-api projects could be easily leveraged for a cleaner solution.
> If completed successfully, users could instead access their client-side
> metrics through the hbase-metrics-api.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186720#comment-16186720 ] Josh Elser commented on HBASE-18409: No worries if you can't get to it. Our primary concern is to not stomp on your toes if you'd like to do this :) We all understand how life gets busy. > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186302#comment-16186302 ] Ronald Macmaster commented on HBASE-18409: -- Yep, I've got to work out a few cases with the unit test failures. Also, the metaPool metrics were kept such that projects already using the old HBase metrics could still rely on them. I will see if I can push out another patch this weekend. Been pretty busy with school, so I'm sorry for the delay. > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186253#comment-16186253 ] Josh Elser commented on HBASE-18409: bq. What would it take to get this over the finish line, do you think? I think just addressing some of the minor stuff I called out in the above comments is all that's needed :) Thanks for bringing it up. I totally let this slip off my radar. If [~qwerty2.0] doesn't have the time to clean it up, I think you/I/anyone could easily do the same and push it in. > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186244#comment-16186244
]
Hadoop QA commented on HBASE-18409:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s{color}
| {color:red} HBASE-18409 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879446/0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8868/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Migrate Client Metrics from codahale to hbase-metrics
> -
>
> Key: HBASE-18409
> URL: https://issues.apache.org/jira/browse/HBASE-18409
> Project: HBase
> Issue Type: Improvement
> Components: Client, java, metrics
>Affects Versions: 3.0.0
>Reporter: Ronald Macmaster
> Labels: newbie
> Fix For: 3.0.0
>
> Attachments:
> 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch,
> 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Currently, the metrics for hbase-client are tailored for reporting via a
> client-side JMX server.
> The MetricsConnection handles the metrics management and reporting via the
> metrics platform from codahale.
> This approach worked well for hbase-1.3.1 when the metrics platform was still
> relatively young, but it could be improved by using the new
> hbase-metrics-api.
> Now that we have an actual hbase-metrics-api that master, regionserver,
> zookeeper, and other daemons use, it would be good to also allow the client
> to leverage the metrics-api.
> Then, the client could also report its metrics via Hadoop's metrics2 if
> desired or through another platform that utilizes the hbase-metrics-api.
> If left alone, client metrics will continue to be only barely visible through
> a client-side JMX server.
> The migration to the new metrics-api could be done by simply changing the
> Metrics data types from codahale types to hbase-metrics types without
> changing the metrics signatures of MetricsConnection unless completely
> necessary.
> The codahale MetricsRegistry would also have to be exchanged for a
> hbase-metrics MetricsRegistry.
> I found this to be a necessary change after attempting to implement my own
> Reporter to use within the MetricsConnection class.
> I was attempting to create a HadoopMetrics2Reporter that extends the codahale
> ScheduledReporter and reports the MetricsConnection metrics to Hadoop's
> metrics2 system.
> The already existing infrastructure in the hbase-metrics and
> hbase-metrics-api projects could be easily leveraged for a cleaner solution.
> If completed successfully, users could instead access their client-side
> metrics through the hbase-metrics-api.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186240#comment-16186240 ] Andrew Purtell commented on HBASE-18409: bq. A little bit of cleanup, and I think this would be good to go! [~elserj] What would it take to get this over the finish line, do you think? > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129888#comment-16129888
]
Josh Elser commented on HBASE-18409:
Deployed this locally and played around a bit with PE -- I'm seeing the
expected metrics client side. A little bit of cleanup, and I think this would
be good to go!
One more:
{code}
-RatioGauge executorMetrics = (RatioGauge) METRICS.getMetricRegistry()
-.getMetrics().get(METRICS.getExecutorPoolName());
-RatioGauge metaMetrics = (RatioGauge) METRICS.getMetricRegistry()
-.getMetrics().get(METRICS.getMetaPoolName());
-assertEquals(Ratio.of(0, 3).getValue(), executorMetrics.getValue(), 0);
-assertEquals(Double.NaN, metaMetrics.getValue(), 0);
+
+// RatioGauge executorMetrics = (RatioGauge)
+// metricsConnection.getMetricRegistry().getMetrics()
+// .get(metricsConnection.getExecutorPoolName());
+// RatioGauge metaMetrics = (RatioGauge)
+// metricsConnection.getMetricRegistry().getMetrics()
+// .get(metricsConnection.getMetaPoolName());
+// assertEquals(Ratio.of(0, 3).getValue(), executorMetrics.getValue(), 0);
+// assertEquals(Double.NaN, metaMetrics.getValue(), 0);
{code}
This should just be removed (not left commented out).
> Migrate Client Metrics from codahale to hbase-metrics
> -
>
> Key: HBASE-18409
> URL: https://issues.apache.org/jira/browse/HBASE-18409
> Project: HBase
> Issue Type: Improvement
> Components: Client, java, metrics
>Affects Versions: 3.0.0
>Reporter: Ronald Macmaster
> Labels: newbie
> Fix For: 3.0.0
>
> Attachments:
> 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch,
> 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Currently, the metrics for hbase-client are tailored for reporting via a
> client-side JMX server.
> The MetricsConnection handles the metrics management and reporting via the
> metrics platform from codahale.
> This approach worked well for hbase-1.3.1 when the metrics platform was still
> relatively young, but it could be improved by using the new
> hbase-metrics-api.
> Now that we have an actual hbase-metrics-api that master, regionserver,
> zookeeper, and other daemons use, it would be good to also allow the client
> to leverage the metrics-api.
> Then, the client could also report its metrics via Hadoop's metrics2 if
> desired or through another platform that utilizes the hbase-metrics-api.
> If left alone, client metrics will continue to be only barely visible through
> a client-side JMX server.
> The migration to the new metrics-api could be done by simply changing the
> Metrics data types from codahale types to hbase-metrics types without
> changing the metrics signatures of MetricsConnection unless completely
> necessary.
> The codahale MetricsRegistry would also have to be exchanged for a
> hbase-metrics MetricsRegistry.
> I found this to be a necessary change after attempting to implement my own
> Reporter to use within the MetricsConnection class.
> I was attempting to create a HadoopMetrics2Reporter that extends the codahale
> ScheduledReporter and reports the MetricsConnection metrics to Hadoop's
> metrics2 system.
> The already existing infrastructure in the hbase-metrics and
> hbase-metrics-api projects could be easily leveraged for a cleaner solution.
> If completed successfully, users could instead access their client-side
> metrics through the hbase-metrics-api.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129869#comment-16129869
]
Josh Elser commented on HBASE-18409:
{code}
+ public static final String CLIENT_SIDE_JMX_ENABLED_KEY =
"hbase.client.jmx.enable";
{code}
This appears to be unused.
> Migrate Client Metrics from codahale to hbase-metrics
> -
>
> Key: HBASE-18409
> URL: https://issues.apache.org/jira/browse/HBASE-18409
> Project: HBase
> Issue Type: Improvement
> Components: Client, java, metrics
>Affects Versions: 3.0.0
>Reporter: Ronald Macmaster
> Labels: newbie
> Fix For: 3.0.0
>
> Attachments:
> 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch,
> 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Currently, the metrics for hbase-client are tailored for reporting via a
> client-side JMX server.
> The MetricsConnection handles the metrics management and reporting via the
> metrics platform from codahale.
> This approach worked well for hbase-1.3.1 when the metrics platform was still
> relatively young, but it could be improved by using the new
> hbase-metrics-api.
> Now that we have an actual hbase-metrics-api that master, regionserver,
> zookeeper, and other daemons use, it would be good to also allow the client
> to leverage the metrics-api.
> Then, the client could also report its metrics via Hadoop's metrics2 if
> desired or through another platform that utilizes the hbase-metrics-api.
> If left alone, client metrics will continue to be only barely visible through
> a client-side JMX server.
> The migration to the new metrics-api could be done by simply changing the
> Metrics data types from codahale types to hbase-metrics types without
> changing the metrics signatures of MetricsConnection unless completely
> necessary.
> The codahale MetricsRegistry would also have to be exchanged for a
> hbase-metrics MetricsRegistry.
> I found this to be a necessary change after attempting to implement my own
> Reporter to use within the MetricsConnection class.
> I was attempting to create a HadoopMetrics2Reporter that extends the codahale
> ScheduledReporter and reports the MetricsConnection metrics to Hadoop's
> metrics2 system.
> The already existing infrastructure in the hbase-metrics and
> hbase-metrics-api projects could be easily leveraged for a cleaner solution.
> If completed successfully, users could instead access their client-side
> metrics through the hbase-metrics-api.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129838#comment-16129838 ] Josh Elser commented on HBASE-18409: Also, [~ndimiduk] for his client-metrics expertise! > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129837#comment-16129837
]
Josh Elser commented on HBASE-18409:
Hi, [~macmaster]! Mighty nice patch you've put together here.
Isolating the metrics aggregation behind the new hbase-metrics API is a great
idea, IMO.
{code}
+ /// Other Bases ///
...
+ /// Other Bases ///
{code}
Maybe tone these down? :) Just a leading comment character would be sufficient
{code}
+this.regionStats = new HashMap();
+this.cacheDroppingExceptions = new HashMap();
{code}
You can omit the types on the RHS. The bare diamond operator is sufficient.
{code}
+ @VisibleForTesting
+ protected final Map regionStats;
+ @VisibleForTesting
+ protected final Map cacheDroppingExceptions;
{code}
These were ConcurrentHashMaps above. Are we sure that we don't need the
concurrency-safety now?
{code}
+ private final String POOL_EXECUTOR_ACTIVE_KEY = "executorPoolActiveThreads";
+ private final String POOL_EXECUTOR_ACTIVE_DESC = "number of active threads
in the executor pool";
+ private final String POOL_META_ACTIVE_KEY = "metaPoolActiveThreads";
+ private final String POOL_META_ACTIVE_DESC = "number of active threads in
the meta pool";
{code}
These are net-new metrics? Should they be spun out into a different JIRA issue
(we try to limit one "thing" per JIRA issue).
FYI [~stack], I think you had done a review on the hbase-metrics API patch when
Enis brought it in. Thought you might be interested in this one too.
> Migrate Client Metrics from codahale to hbase-metrics
> -
>
> Key: HBASE-18409
> URL: https://issues.apache.org/jira/browse/HBASE-18409
> Project: HBase
> Issue Type: Improvement
> Components: Client, java, metrics
>Affects Versions: 3.0.0
>Reporter: Ronald Macmaster
> Labels: newbie
> Fix For: 3.0.0
>
> Attachments:
> 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch,
> 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Currently, the metrics for hbase-client are tailored for reporting via a
> client-side JMX server.
> The MetricsConnection handles the metrics management and reporting via the
> metrics platform from codahale.
> This approach worked well for hbase-1.3.1 when the metrics platform was still
> relatively young, but it could be improved by using the new
> hbase-metrics-api.
> Now that we have an actual hbase-metrics-api that master, regionserver,
> zookeeper, and other daemons use, it would be good to also allow the client
> to leverage the metrics-api.
> Then, the client could also report its metrics via Hadoop's metrics2 if
> desired or through another platform that utilizes the hbase-metrics-api.
> If left alone, client metrics will continue to be only barely visible through
> a client-side JMX server.
> The migration to the new metrics-api could be done by simply changing the
> Metrics data types from codahale types to hbase-metrics types without
> changing the metrics signatures of MetricsConnection unless completely
> necessary.
> The codahale MetricsRegistry would also have to be exchanged for a
> hbase-metrics MetricsRegistry.
> I found this to be a necessary change after attempting to implement my own
> Reporter to use within the MetricsConnection class.
> I was attempting to create a HadoopMetrics2Reporter that extends the codahale
> ScheduledReporter and reports the MetricsConnection metrics to Hadoop's
> metrics2 system.
> The already existing infrastructure in the hbase-metrics and
> hbase-metrics-api projects could be easily leveraged for a cleaner solution.
> If completed successfully, users could instead access their client-side
> metrics through the hbase-metrics-api.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[ https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124431#comment-16124431 ] Ronald Macmaster commented on HBASE-18409: -- What do you all think? Can we apply this patch? > Migrate Client Metrics from codahale to hbase-metrics > - > > Key: HBASE-18409 > URL: https://issues.apache.org/jira/browse/HBASE-18409 > Project: HBase > Issue Type: Improvement > Components: Client, java, metrics >Affects Versions: 3.0.0 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0 > > Attachments: > 0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch, > 0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, the metrics for hbase-client are tailored for reporting via a > client-side JMX server. > The MetricsConnection handles the metrics management and reporting via the > metrics platform from codahale. > This approach worked well for hbase-1.3.1 when the metrics platform was still > relatively young, but it could be improved by using the new > hbase-metrics-api. > Now that we have an actual hbase-metrics-api that master, regionserver, > zookeeper, and other daemons use, it would be good to also allow the client > to leverage the metrics-api. > Then, the client could also report its metrics via Hadoop's metrics2 if > desired or through another platform that utilizes the hbase-metrics-api. > If left alone, client metrics will continue to be only barely visible through > a client-side JMX server. > The migration to the new metrics-api could be done by simply changing the > Metrics data types from codahale types to hbase-metrics types without > changing the metrics signatures of MetricsConnection unless completely > necessary. > The codahale MetricsRegistry would also have to be exchanged for a > hbase-metrics MetricsRegistry. > I found this to be a necessary change after attempting to implement my own > Reporter to use within the MetricsConnection class. > I was attempting to create a HadoopMetrics2Reporter that extends the codahale > ScheduledReporter and reports the MetricsConnection metrics to Hadoop's > metrics2 system. > The already existing infrastructure in the hbase-metrics and > hbase-metrics-api projects could be easily leveraged for a cleaner solution. > If completed successfully, users could instead access their client-side > metrics through the hbase-metrics-api. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105971#comment-16105971
]
Hadoop QA commented on HBASE-18409:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
53s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}133m
59s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
53s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}195m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879446/0002-HBASE-18409-MetricsConnection-client-metrics-migrati.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile xml |
| uname | Linux d9ac73b8d402 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-18409) Migrate Client Metrics from codahale to hbase-metrics
[
https://issues.apache.org/jira/browse/HBASE-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105721#comment-16105721
]
Hadoop QA commented on HBASE-18409:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
33s{color} | {color:red} hbase-hadoop2-compat generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
59s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-hadoop2-compat |
| | Should org.apache.hadoop.hbase.client.MetricsClientSourceImpl$RegionStats
be a _static_ inner class? At MetricsClientSourceImpl.java:inner class? At
MetricsClientSourceImpl.java:[lines 189-198] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.13.1 Server=1.13.1 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879398/0001-HBASE-18409-MetricsConnection-client-metrics-migration.patch
|
| Optional Tests | asflicense javac
