[jira] [Comment Edited] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527834#comment-15527834 ] Dustin Pho edited comment on HBASE-16661 at 9/27/16 11:57 PM: -- Good call on the description! I missed that completely. was (Author: pho): Good call! > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16661.001.patch, HBASE-16661.002.patch, > HBASE-16661.003.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16661: --- Attachment: HBASE-16661.003.patch Good call! > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16661.001.patch, HBASE-16661.002.patch, > HBASE-16661.003.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16657: --- Attachment: with-patch-changes.png > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch, HBASE-16657.002.patch, > with-patch-changes.png, without-patch-changes.png > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16657: --- Attachment: HBASE-16657.002.patch Changed format of lastMajorCompactionTs to "-MM-dd HH:mm ZZ" > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch, HBASE-16657.002.patch, > without-patch-changes.png > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16657: --- Attachment: (was: with-patch-changes.png) > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch, HBASE-16657.002.patch, > without-patch-changes.png > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16661: --- Attachment: HBASE-16661.002.patch Logging the IOException in MetricsRegionWrapperImpl#getLastMajorCompactionAge. Also fixed the typo majorCompactioOnly => majorCompactionOnly in HRegion#getOldestHfileTs and Region#getOldestHfileTs. > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16661.001.patch, HBASE-16661.002.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518029#comment-15518029 ] Dustin Pho commented on HBASE-16661: Regarding 2) -- `LOG.fatal(...)` over `LOG.error()` for this case? Should MetricsRegionWrapperImpl#getLastMajorCompactionAge return -1 in this case (and in the case where a major compaction has not happened)? Returning 0 implies that a major compaction just occurred. > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16661.001.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514698#comment-15514698 ] Dustin Pho commented on HBASE-16657: Regarding manual steps taken to verify -- 1. use major_compact in bin/hbase shell 2. Go to the corresponding :/rs-status#regionCompactStats. The last major compaction should be updated. I don't think my changes affect the failed junit tests. > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch, with-patch-changes.png, > without-patch-changes.png > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16657: --- Attachment: without-patch-changes.png > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch, with-patch-changes.png, > without-patch-changes.png > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16657: --- Attachment: with-patch-changes.png > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch, with-patch-changes.png > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16657: --- Status: Patch Available (was: Open) > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16657: --- Attachment: HBASE-16657.001.patch Added a column for last compaction time under RegionServer UI > Regions > Compaction Metrics > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16657.001.patch > > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-16657) Expose per-region last major compaction timestamp in RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho reassigned HBASE-16657: -- Assignee: Dustin Pho > Expose per-region last major compaction timestamp in RegionServer UI > > > Key: HBASE-16657 > URL: https://issues.apache.org/jira/browse/HBASE-16657 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > > HBASE-12859 added some tracking for the last major compaction completed for > each region. However, this is currently only exposed through the cluster > status reporting and the Admin API. Since the regionserver is already > reporting this information, it would be nice to fold it in somewhere to the > region listing in the regionserver UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16661: --- Status: Patch Available (was: Open) > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16661.001.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16661) Add last major compaction age to per-region metrics
[ https://issues.apache.org/jira/browse/HBASE-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16661: --- Attachment: HBASE-16661.001.patch Added LastMajorCompactionAge to the per-region metrics > Add last major compaction age to per-region metrics > --- > > Key: HBASE-16661 > URL: https://issues.apache.org/jira/browse/HBASE-16661 > Project: HBase > Issue Type: Improvement >Reporter: Gary Helmling >Assignee: Dustin Pho >Priority: Minor > Attachments: HBASE-16661.001.patch > > > After HBASE-12859, we can now track the last major compaction timestamp for > each region. However, this is only exposed through cluster status reporting > and the admin API. > We have similar per-region metrics around storefile age, but none that > filters on major compaction specifically. > Let's add a metric for last major compaction age. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16540) Scan should do additional validation on start and stop row
[ https://issues.apache.org/jira/browse/HBASE-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16540: --- Attachment: HBASE-16540.002.patch updated the javadocs to reference the constant used > Scan should do additional validation on start and stop row > -- > > Key: HBASE-16540 > URL: https://issues.apache.org/jira/browse/HBASE-16540 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Gary Helmling >Assignee: Dustin Pho > Attachments: HBASE-16540.002.patch, HBASE-16540.patch > > > Scan.setStartRow() and setStopRow() should validate the byte[] passed to > ensure it meets the criteria for a row key. If the byte[] length is greater > that Short.MAX_VALUE, we should throw an IllegalArgumentException in order to > fast fail and prevent server-side errors being thrown and retried. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16540) Scan should do additional validation on start and stop row
[ https://issues.apache.org/jira/browse/HBASE-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16540: --- Status: Patch Available (was: Open) > Scan should do additional validation on start and stop row > -- > > Key: HBASE-16540 > URL: https://issues.apache.org/jira/browse/HBASE-16540 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Gary Helmling >Assignee: Dustin Pho > Attachments: HBASE-16540.patch > > > Scan.setStartRow() and setStopRow() should validate the byte[] passed to > ensure it meets the criteria for a row key. If the byte[] length is greater > that Short.MAX_VALUE, we should throw an IllegalArgumentException in order to > fast fail and prevent server-side errors being thrown and retried. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16540) Scan should do additional validation on start and stop row
[ https://issues.apache.org/jira/browse/HBASE-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Pho updated HBASE-16540: --- Attachment: HBASE-16540.patch Added a validator for row keys passed to setStartRow and setStopRow such that they must be less than or equal to HConstant.MAX_ROW_LENGTH (Short.MAX_VALUE). > Scan should do additional validation on start and stop row > -- > > Key: HBASE-16540 > URL: https://issues.apache.org/jira/browse/HBASE-16540 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Gary Helmling >Assignee: Dustin Pho > Attachments: HBASE-16540.patch > > > Scan.setStartRow() and setStopRow() should validate the byte[] passed to > ensure it meets the criteria for a row key. If the byte[] length is greater > that Short.MAX_VALUE, we should throw an IllegalArgumentException in order to > fast fail and prevent server-side errors being thrown and retried. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
