[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261015#comment-13261015 ] Hudson commented on HBASE-3614: --- Integrated in HBase-0.94 #145 (See [https://builds.apache.org/job/HBase-0.94/145/]) HBASE-5836 Backport per region metrics from HBASE-3614 to 0.94.1 (Revision 1329958) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/OperationMetrics.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionMetricsStorage.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261219#comment-13261219 ] Hudson commented on HBASE-3614: --- Integrated in HBase-0.94-security #20 (See [https://builds.apache.org/job/HBase-0.94-security/20/]) HBASE-5836 Backport per region metrics from HBASE-3614 to 0.94.1 (Revision 1329958) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/OperationMetrics.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionMetricsStorage.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258394#comment-13258394 ] Elliott Clark commented on HBASE-3614: -- @Todd the time period is configurable. It's set through the hadoop-metrics hbase.period. Whenever the thread that publishes metrics comes around the averages are re-set. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[
https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257787#comment-13257787
]
stack commented on HBASE-3614:
--
Since you renamed RegionOperationMetrics, is this right now:
{code}
+ private final OperationMetrics regionMetrics;
{code}
Should it be named metrics or operationMetrics?
Whats 'unknown' in the following? + //null will be treated as unknown.
We are updating metrics w/o attributing them to a cf?
Fix misspell 'Inctement' in hbase-site change
Patch is good to go after addressing above. Good stuff.
Expose per-region request rate metrics
--
Key: HBASE-3614
URL: https://issues.apache.org/jira/browse/HBASE-3614
Project: HBase
Issue Type: Improvement
Components: metrics, regionserver
Reporter: Gary Helmling
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch,
HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch,
HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, Screen Shot
2012-04-17 at 2.41.27 PM.png
We currently export metrics on request rates for each region server, and this
can help with identifying uneven load at a high level. But once you see a
given server under high load, you're forced to extrapolate based on your
application patterns and the data it's serving what the likely culprit is.
This can and should be much easier if we just exported request rate metrics
per-region on each server.
Dynamically updating the metrics keys based on assigned regions may pose some
minor challenges, but this seems a very valuable diagnostic tool to have
available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257797#comment-13257797 ] Elliott Clark commented on HBASE-3614: -- And I beat you to the rename :-) is opMetrics alright. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257847#comment-13257847 ] Hadoop QA commented on HBASE-3614: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523401/HBASE-3614-8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1582//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1582//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1582//console This message is automatically generated. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257927#comment-13257927 ] Hudson commented on HBASE-3614: --- Integrated in HBase-TRUNK #2788 (See [https://builds.apache.org/job/HBase-TRUNK/2788/]) HBASE-3614 Expose per-region request rate metrics (Revision 1328140) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/OperationMetrics.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java * /hbase/trunk/src/main/resources/hbase-default.xml * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257978#comment-13257978 ] Todd Lipcon commented on HBASE-3614: Just getting to this after it's committed.. but: what's the time scale of avgtime on these metrics? Averages that are since boot are kind of useless. We should either track the sum of the time, or do a time-biased metric like Shaneal added recently in other places. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258011#comment-13258011 ] stack commented on HBASE-3614: -- @Todd This issue just exposes metrics that were already being collected per region. I believe its over the metrics reporting period (5 seconds?). Want that changed? Metrics could do w/ a revamp/edit for sure. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258013#comment-13258013 ] Todd Lipcon commented on HBASE-3614: Gotcha. Apologies for my cluelessness :) Seems fine. Though I think Shaneal's percentile stuff would be a nice improvement, what we've got now is at least useful. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258022#comment-13258022 ] Hudson commented on HBASE-3614: --- Integrated in HBase-TRUNK-security #176 (See [https://builds.apache.org/job/HBase-TRUNK-security/176/]) HBASE-3614 Expose per-region request rate metrics (Revision 1328140) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/OperationMetrics.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java * /hbase/trunk/src/main/resources/hbase-default.xml * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[
https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257226#comment-13257226
]
stack commented on HBASE-3614:
--
This could be final: '+ private RegionOperationMetrics regionMetrics;'?
100 chars per line.
Just pass HRegionInfo altogether to the below?
{code}
+this.regionMetrics = new RegionOperationMetrics(conf,
this.regionInfo.getTableNameAsString(), this.regionInfo.getEncodedName());
{code}
Err... your replacement is better than what was there previously in the below:
{code}
-final String metricPrefix = SchemaMetrics.generateSchemaMetricsPrefix(
-getTableDesc().getNameAsString(), familyMap.keySet());
-if (!metricPrefix.isEmpty()) {
- RegionMetricsStorage.incrTimeVaryingMetric(metricPrefix + delete_,
after - now);
-}
+this.regionMetrics.updateDeleteMetrics(familyMap.keySet(), after-now);
{code}
Whats happening here?
{code}
+if (cfSet == null) {
+ cfSet = put.getFamilyMap().keySet();
+} else {
+ cfSetConsistent = cfSetConsistent put.equals(cfSet);
{code}
Do we have to get the column family set each time through? It never changes
(currently) while the region is open.
Whats a cfSetConsistent? A comment would help?
Yeah, I don't follow this stuff:
{code}
+ //See if the column families were consistent through the whole thing.
+ //if they were then keep them. If they were not then pass a null.
+ //null will be treated as unknown.
{code}
Should be hbase.metrics.region.exposeOperationTimes instead of
hbase.metrics.exposeOperationTimes to convey its on/off for per-region metrics?
This patch is great.
Expose per-region request rate metrics
--
Key: HBASE-3614
URL: https://issues.apache.org/jira/browse/HBASE-3614
Project: HBase
Issue Type: Improvement
Components: metrics, regionserver
Reporter: Gary Helmling
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch,
HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, Screen Shot
2012-04-17 at 2.41.27 PM.png
We currently export metrics on request rates for each region server, and this
can help with identifying uneven load at a high level. But once you see a
given server under high load, you're forced to extrapolate based on your
application patterns and the data it's serving what the likely culprit is.
This can and should be much easier if we just exported request rate metrics
per-region on each server.
Dynamically updating the metrics keys based on assigned regions may pose some
minor challenges, but this seems a very valuable diagnostic tool to have
available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13256122#comment-13256122 ] Enis Soztutar commented on HBASE-3614: -- This is fine for JMX, but these per-region metrics will be exposed via the Hadoop metrics system as well right? If so, the large number of regions and the fact that regions are dynamic would make it useless for ganglia (or similar). So, I think we should either not expose them through the hadoop metrics, but do itJMX only, or make per-region metrics configurable IMO. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13256149#comment-13256149 ] Elliott Clark commented on HBASE-3614: -- I think per region and per cf metrics being configurable is probably the right way to go. I'll add that in the next go around. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[
https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13255304#comment-13255304
]
stack commented on HBASE-3614:
--
FYI 100 chars per line max and space around operators (this won't fly:
cfSetConsistent?cfSet:null)
I like how you are removing metrics stuff from HRegion out to a region scoped
metrics class.
'+public class RegionMetrics {' needs a class comment saying what its all
about. Does the class need to be public? Can it be scoped to this package
only?
Collect all the data members at the top of the class. Thats whats usually done
in this code base.
So put the tablename etc. in RegionMetric before the constructor etc. rather
than after.
Does this need to be public generateRegionMetricsPrefix?
What do these new metrics look like? Is this all it takes to expose them?
Some regionnames are going to be really long. Should you use the region
encoded name instead of the full name? Do you think we even need the table
name as prefix?
Good stuff Elliott.
Expose per-region request rate metrics
--
Key: HBASE-3614
URL: https://issues.apache.org/jira/browse/HBASE-3614
Project: HBase
Issue Type: Improvement
Components: metrics, regionserver
Reporter: Gary Helmling
Assignee: Elliott Clark
Priority: Minor
Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch
We currently export metrics on request rates for each region server, and this
can help with identifying uneven load at a high level. But once you see a
given server under high load, you're forced to extrapolate based on your
application patterns and the data it's serving what the likely culprit is.
This can and should be much easier if we just exported request rate metrics
per-region on each server.
Dynamically updating the metrics keys based on assigned regions may pose some
minor challenges, but this seems a very valuable diagnostic tool to have
available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13004817#comment-13004817 ] Jonathan Gray commented on HBASE-3614: -- I'm not sure if there is a JIRA yet, but some guys at FB did a bunch of work on doing per-family metrics. They did work to dynamically generate new metric names, etc. I think we could work on this at the same time we start to think about using the info for better load balancing and such. This could obviously come first. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Priority: Minor We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
