[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[
https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17268717#comment-17268717
]
Hudson commented on HBASE-25329:
Results for branch branch-1
[build #76 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/76/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/76//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/76//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/76//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Dump region hashes in logs for the regions that are stuck in transition for
> more than a configured amount of time
> -
>
> Key: HBASE-25329
> URL: https://issues.apache.org/jira/browse/HBASE-25329
> Project: HBase
> Issue Type: Improvement
>Reporter: Caroline
>Assignee: Caroline
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.2.7, 2.5.0, 2.4.1, 2.3.5
>
> Attachments: HBASE-25329.branch-1.000.patch,
> HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch
>
>
> We have metrics for number of RITs as well as number of RITs above a certain
> threshold, but we don't have any way of keeping track of the region hashes of
> those RITs. It would be beneficial to emit those region hashes as a metric,
> as well as log them, so that we don't accidentally lose this information for
> debugging the RIT at a later tiime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[
https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266625#comment-17266625
]
Hudson commented on HBASE-25329:
Results for branch branch-2.3
[build #153 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/153/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/153/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/153/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/153/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/153/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dump region hashes in logs for the regions that are stuck in transition for
> more than a configured amount of time
> -
>
> Key: HBASE-25329
> URL: https://issues.apache.org/jira/browse/HBASE-25329
> Project: HBase
> Issue Type: Improvement
>Reporter: Caroline
>Assignee: Caroline
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.1, 2.3.5
>
> Attachments: HBASE-25329.branch-1.000.patch,
> HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch
>
>
> We have metrics for number of RITs as well as number of RITs above a certain
> threshold, but we don't have any way of keeping track of the region hashes of
> those RITs. It would be beneficial to emit those region hashes as a metric,
> as well as log them, so that we don't accidentally lose this information for
> debugging the RIT at a later tiime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[
https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266359#comment-17266359
]
Hudson commented on HBASE-25329:
Results for branch branch-2.2
[build #156 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/156/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/156//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/156//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/156//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dump region hashes in logs for the regions that are stuck in transition for
> more than a configured amount of time
> -
>
> Key: HBASE-25329
> URL: https://issues.apache.org/jira/browse/HBASE-25329
> Project: HBase
> Issue Type: Improvement
>Reporter: Caroline
>Assignee: Caroline
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.1, 2.3.5
>
> Attachments: HBASE-25329.branch-1.000.patch,
> HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch
>
>
> We have metrics for number of RITs as well as number of RITs above a certain
> threshold, but we don't have any way of keeping track of the region hashes of
> those RITs. It would be beneficial to emit those region hashes as a metric,
> as well as log them, so that we don't accidentally lose this information for
> debugging the RIT at a later tiime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266328#comment-17266328 ] Huaxiang Sun commented on HBASE-25329: -- Thanks [~vjasani], let's keep it for 2.3.5 for now, unless a new RC effort will happen. I will make sure new jiras after RC4 tag have correct fixVersion. > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.4.1, 2.3.5 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266086#comment-17266086 ] Viraj Jasani commented on HBASE-25329: -- FYI [~huaxiangsun] > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[
https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265635#comment-17265635
]
Hudson commented on HBASE-25329:
Results for branch branch-2.4
[build #32 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/32/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/32/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/32/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/32/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/32/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dump region hashes in logs for the regions that are stuck in transition for
> more than a configured amount of time
> -
>
> Key: HBASE-25329
> URL: https://issues.apache.org/jira/browse/HBASE-25329
> Project: HBase
> Issue Type: Improvement
>Reporter: Caroline
>Assignee: Caroline
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1
>
> Attachments: HBASE-25329.branch-1.000.patch,
> HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch
>
>
> We have metrics for number of RITs as well as number of RITs above a certain
> threshold, but we don't have any way of keeping track of the region hashes of
> those RITs. It would be beneficial to emit those region hashes as a metric,
> as well as log them, so that we don't accidentally lose this information for
> debugging the RIT at a later tiime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[
https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264818#comment-17264818
]
Hudson commented on HBASE-25329:
Results for branch master
[build #185 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/185/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/185/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/185/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/185/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dump region hashes in logs for the regions that are stuck in transition for
> more than a configured amount of time
> -
>
> Key: HBASE-25329
> URL: https://issues.apache.org/jira/browse/HBASE-25329
> Project: HBase
> Issue Type: Improvement
>Reporter: Caroline
>Assignee: Caroline
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1
>
> Attachments: HBASE-25329.branch-1.000.patch,
> HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch
>
>
> We have metrics for number of RITs as well as number of RITs above a certain
> threshold, but we don't have any way of keeping track of the region hashes of
> those RITs. It would be beneficial to emit those region hashes as a metric,
> as well as log them, so that we don't accidentally lose this information for
> debugging the RIT at a later tiime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[
https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264440#comment-17264440
]
Hudson commented on HBASE-25329:
Results for branch branch-2
[build #157 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/157/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/157/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/157/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/157/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/157/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dump region hashes in logs for the regions that are stuck in transition for
> more than a configured amount of time
> -
>
> Key: HBASE-25329
> URL: https://issues.apache.org/jira/browse/HBASE-25329
> Project: HBase
> Issue Type: Improvement
>Reporter: Caroline
>Assignee: Caroline
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1
>
> Attachments: HBASE-25329.branch-1.000.patch,
> HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch
>
>
> We have metrics for number of RITs as well as number of RITs above a certain
> threshold, but we don't have any way of keeping track of the region hashes of
> those RITs. It would be beneficial to emit those region hashes as a metric,
> as well as log them, so that we don't accidentally lose this information for
> debugging the RIT at a later tiime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264372#comment-17264372 ] Andrew Kyle Purtell commented on HBASE-25329: - Thanks [~vjasani]. Thanks for the contribution [~caroliney14]! > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264369#comment-17264369 ] Caroline commented on HBASE-25329: -- Thanks [~vjasani] > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.1 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264345#comment-17264345 ] Viraj Jasani commented on HBASE-25329: -- Sure thing, let me commit and resolve this Jira. Will reopen later for branch-2.3, 2.2 and branch-1 backports. > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5, 2.4.2 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264341#comment-17264341 ] Andrew Kyle Purtell commented on HBASE-25329: - [~vjasani][~caroliney14] If you want to make 2.4.1RC0, commit to branch-2.4 within the next hour or two. > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5, 2.4.2 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263990#comment-17263990 ] Viraj Jasani commented on HBASE-25329: -- PR#2761 is merged to master and it was cleanly applied to branch-2 without any manual intervention. Closed PR#2763 as it is not needed anymore. Will take a look at branch-1 PR but before committing changes to branch-1, we are still left with few backports on various active branch-2.x. > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5, 2.4.2 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263962#comment-17263962 ] Caroline commented on HBASE-25329: -- Thanks [~vjasani] for merging the patch into master. I've updated the branch-1 PR as well; feel free to take a look: [https://github.com/apache/hbase/pull/2762/files] Can you explain how the change has been merged into branch-2? Isn't the branch-2 PR still open? (The one that was merged had a base branch of master, right?) > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5, 2.4.2 > > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263915#comment-17263915 ] Viraj Jasani commented on HBASE-25329: -- Thanks [~caroliney14] for the updated patch. The changes are merged to master and branch-2 so far. Let's wait for some release preparations to complete before further backports. > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260132#comment-17260132 ] Caroline commented on HBASE-25329: -- [~apurtell] [~vjasani] [~bharathv] Thanks for the feedback. I'll remove the metrics additions from the patches since this particular Jira concerns dumping RITs into the logs. Do you have any suggestions on how often we should be logging the RITs? For the idea of adding a /rit endpoint in master, I've created a Jira to track it. Would love to hear suggestions about what info to expose on the endpoint. If someone could provide a servlet code example that would also be helpful, since I'm unfamiliar with them in this code base. Thanks! > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259960#comment-17259960 ] Andrew Kyle Purtell commented on HBASE-25329: - FWIW I also like the idea of the new servlet on master to emit structured records regards current RIT for alterting, analysis, debugging, etc. > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259955#comment-17259955 ] Andrew Kyle Purtell commented on HBASE-25329: - bq. Patch seems fine but was curious if we considered adding something like a /rit end point (servlet) on master? I am just getting back up to speed after vacation but let me say the current patch is not ok in my opinion. Please fix the below issues (IMHO): Description is "Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time", but - Only the first N hashes are logged. - Patch introduces new metrics. (I share Bharath's view that the usefulness of the new metrics is questionable.) In my opinion RITs are too transient to have a metric for each RIT instance. Metrics about RITs, like total count of RITs in the current metrics reporting period, these are reasonable metrics and we already have them. Per region hash RIT metrics represent objects and data with a limited lifespan and usefulness and are best exported and processed by some other means, such as the debug dump servlet (functionality we already have). This is just my opinion and may not be shared by others. If this patch proposes to "dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time", then it should actually dump all of the found RITs. Otherwise in my opinion it fails to achieve the basic aim. While there does need to be a limit on total line length of an emitted log line, it is certainly possible to emit as many log lines as needed to dump all RIT in progress. > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[
https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259451#comment-17259451
]
Viraj Jasani commented on HBASE-25329:
--
{quote}It's much easier to parse this file rather than parse a metric which is
plain hashes.
{quote}
Absolutely, having a new endpoint on master would be really great and should
benefit in parsing it! +1 to this idea.
> Dump region hashes in logs for the regions that are stuck in transition for
> more than a configured amount of time
> -
>
> Key: HBASE-25329
> URL: https://issues.apache.org/jira/browse/HBASE-25329
> Project: HBase
> Issue Type: Improvement
>Reporter: Caroline
>Assignee: Caroline
>Priority: Minor
> Attachments: HBASE-25329.branch-1.000.patch,
> HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch
>
>
> We have metrics for number of RITs as well as number of RITs above a certain
> threshold, but we don't have any way of keeping track of the region hashes of
> those RITs. It would be beneficial to emit those region hashes as a metric,
> as well as log them, so that we don't accidentally lose this information for
> debugging the RIT at a later tiime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259327#comment-17259327 ] Bharath Vissapragada commented on HBASE-25329: -- Patch seems fine but was curious if we considered adding something like a /rit end point (servlet) on master? Looks like the patch caps the number of RITs to 20 and emits a new metric that is a concatenation of hashes. IMO it is more useful to have a nice parseable (json) end point that has richer metadata (like server ritDuration and other RegionState attributes). It's much easier to parse this file rather than parse a metric which is plain hashes. Thoughts? > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248238#comment-17248238 ] Siddhi Mehta commented on HBASE-25329: -- FYI [~apurtell] [~dmanning] > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247088#comment-17247088 ] Caroline commented on HBASE-25329: -- [~vjasani] Thanks for the heads-up, didn't know patch support was being deprecated. I've created 3 PRs: [https://github.com/apache/hbase/pull/2761] [https://github.com/apache/hbase/pull/2762] [https://github.com/apache/hbase/pull/2763] > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25329) Dump region hashes in logs for the regions that are stuck in transition for more than a configured amount of time
[ https://issues.apache.org/jira/browse/HBASE-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247063#comment-17247063 ] Viraj Jasani commented on HBASE-25329: -- [~caroliney14] If you could create PRs including backport changes, that would be great. patch file support is almost on the verge of deprecation (or maybe already deprecated). Thanks > Dump region hashes in logs for the regions that are stuck in transition for > more than a configured amount of time > - > > Key: HBASE-25329 > URL: https://issues.apache.org/jira/browse/HBASE-25329 > Project: HBase > Issue Type: Improvement >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Attachments: HBASE-25329.branch-1.000.patch, > HBASE-25329.branch-2.000.patch, HBASE-25329.master.000.patch > > > We have metrics for number of RITs as well as number of RITs above a certain > threshold, but we don't have any way of keeping track of the region hashes of > those RITs. It would be beneficial to emit those region hashes as a metric, > as well as log them, so that we don't accidentally lose this information for > debugging the RIT at a later tiime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
