[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569317#comment-16569317
]
Hudson commented on HBASE-20847:
Results for branch branch-2.0
[build #632 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/632/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/632//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/632//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/632//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-20847-addendum.patch,
> HBASE-20847-branch-2.0-v1.patch, HBASE-20847-branch-2.0.patch,
> HBASE-20847-v1.patch, HBASE-20847-v2.patch, HBASE-20847-v3.patch,
> HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541646#comment-16541646
]
Hudson commented on HBASE-20847:
Results for branch master
[build #394 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/394/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/394//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/394//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/394//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-20847-addendum.patch,
> HBASE-20847-branch-2.0-v1.patch, HBASE-20847-branch-2.0.patch,
> HBASE-20847-v1.patch, HBASE-20847-v2.patch, HBASE-20847-v3.patch,
> HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541154#comment-16541154
]
Hudson commented on HBASE-20847:
Results for branch branch-2
[build #972 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/972/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/972//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/972//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/972//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-20847-addendum.patch,
> HBASE-20847-branch-2.0-v1.patch, HBASE-20847-branch-2.0.patch,
> HBASE-20847-v1.patch, HBASE-20847-v2.patch, HBASE-20847-v3.patch,
> HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541134#comment-16541134
]
Hudson commented on HBASE-20847:
Results for branch branch-2.1
[build #50 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/50/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/50//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/50//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/50//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-20847-addendum.patch,
> HBASE-20847-branch-2.0-v1.patch, HBASE-20847-branch-2.0.patch,
> HBASE-20847-v1.patch, HBASE-20847-v2.patch, HBASE-20847-v3.patch,
> HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540766#comment-16540766
]
Hudson commented on HBASE-20847:
Results for branch branch-2.1
[build #49 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/49/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/49//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/49//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/49//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-20847-addendum.patch,
> HBASE-20847-branch-2.0-v1.patch, HBASE-20847-branch-2.0.patch,
> HBASE-20847-v1.patch, HBASE-20847-v2.patch, HBASE-20847-v3.patch,
> HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540756#comment-16540756
]
Hudson commented on HBASE-20847:
Results for branch branch-2
[build #971 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/971/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/971//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/971//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/971//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-20847-addendum.patch,
> HBASE-20847-branch-2.0-v1.patch, HBASE-20847-branch-2.0.patch,
> HBASE-20847-v1.patch, HBASE-20847-v2.patch, HBASE-20847-v3.patch,
> HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540460#comment-16540460 ] stack commented on HBASE-20847: --- +1 on v3. I like explanation in releaseExclusiveLock. On backport to branch-2.0, I'm worried there'll be side-effects and that we need more than this fix. Let me make a separate issue for backport to branch-2.0. Will evaluate before pushing a 2.0.2. I can do it. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-addendum.patch, > HBASE-20847-branch-2.0-v1.patch, HBASE-20847-branch-2.0.patch, > HBASE-20847-v1.patch, HBASE-20847-v2.patch, HBASE-20847-v3.patch, > HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540336#comment-16540336
]
Hadoop QA commented on HBASE-20847:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}118m
51s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20847 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12931167/HBASE-20847-addendum.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 05cf9c955e1e 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a838f7631f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13594/testReport/ |
| Max. process+thread count | 4845 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13594/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> The parent proc
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539920#comment-16539920
]
Hadoop QA commented on HBASE-20847:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
44s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hbase-procedure: The patch generated 0 new + 1
unchanged - 5 fixed = 1 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} hbase-server: The patch generated 0 new + 26
unchanged - 1 fixed = 26 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
46s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 38s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 71m 55s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterProcedureScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-20847 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12931148/HBASE-20847-branch-2.0.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ac695
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539814#comment-16539814 ] Duo Zhang commented on HBASE-20847: --- Pushed to branch-2.1+. The patch for branch-2.0 is a little different, so let's wait for the pre commit result. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-branch-2.0.patch, HBASE-20847-v1.patch, > HBASE-20847-v2.patch, HBASE-20847-v3.patch, HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539773#comment-16539773 ] Duo Zhang commented on HBASE-20847: --- Will commit after fixing the checkstyle issue. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847-v3.patch, HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539629#comment-16539629
]
Hadoop QA commented on HBASE-20847:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hbase-procedure: The patch generated 0 new + 1
unchanged - 5 fixed = 1 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
9s{color} | {color:red} hbase-server: The patch generated 1 new + 22 unchanged
- 1 fixed = 23 total (was 23) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
46s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20847 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/1293/HBASE-20847-v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6fd9b51ac926 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkin
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539557#comment-16539557
]
Allan Yang commented on HBASE-20847:
{quote}
We need to release the exclusive lock if we are scheduling sub procedures. This
is what we do for now.
{quote}
Are we doing this now? IIRC, we will hold the exclusive lock the whole life
time even if we schedule sub procedures(if holdLock = true).
Just reviewed V3 patch, +1 on this patch, I think it totally resolves the
problem of HBASE-20846. You can add the test case in HBASE-20846 to this patch,
and resolve HBASE-20846 after this one.
For restoring locks after master restarts, I think we can open another issue to
implement it, since it is a big change in AMv2.
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch,
> HBASE-20847-v3.patch, HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539524#comment-16539524 ] Duo Zhang commented on HBASE-20847: --- The root cause for TestAdmin2 is that, we only have 5 rpc handlers so all of them will be blocked by the createTable request, as now the AssignProcedure can hold the shared lock while the parent CreateTableProcedure holds the exclusive lock, so all other broken CreateTableProcedure will hang there and wait for the AssignProcedures, but AssignProcedure will lead to a reportRegionTransition call, which can not be executed as all the rpc handlers have been consumed by createTable... It is strange that the requests from user are mixed up with the system requests... I temporarily fixed it by setting a larger handler count, need dig later... > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847-v3.patch, HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539462#comment-16539462 ] Duo Zhang commented on HBASE-20847: --- The problem is that, in ProcedureExecutor will schedule the sub procedures before releasing the lock(if needed, i.e, holdLock is false), this breaks our assumption above, where we assume that the the parent procedure will not release the lock until the sub procedures finishes... Anyway, I think remove the assumption is still fine, as in Java, you are free release the WriteLock before releasing the readLock. Let me change the code. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538686#comment-16538686 ] Duo Zhang commented on HBASE-20847: --- OK the failed UTs are related. Seems we have something wrong with the count of sharedLock accounting. Let me dig more... > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538649#comment-16538649 ] stack commented on HBASE-20847: --- I +1'd this patch up on rb. Its cleanup and the refactors to lockAndQueue are improvements pushing down the messy parent checking distributed around the codebase. I agree with the trySharedLock change. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538646#comment-16538646
]
Duo Zhang commented on HBASE-20847:
---
{quote}
You are arguing we should never take an exclusive lock on a table? For a
modifytableprocedure?
{quote}
We need to release the exclusive lock if we are scheduling sub procedures. This
is what we do for now.
{quote}
Assign can only proceed after WAL logs have been split... so if an SCP and a
ModifyTableProcedure at same time, MTP should wait until SCP has finished log
splitting before proceeding SCP should wait till MTP has done
assigning/unassiging before it tries assigning?
{quote}
We need to hold the lock. You can think from opposite, what if the SCP
schedules AssignProcedure and then there comes a ModifyTableProcedure? Here we
do not need to wait so long if the MTP comes first. We just schedule the
AssignProcedures, but it will blocked if MTP has held the exclusive lock. And
after MTP schedules Unassign/AssignProcedures, it will release the exclusive
lock and we can go on.
The lock recovery after master restarts will be addressed by HBASE-20846, it is
another story. Here the problem is we need to aqcuire the shared lock every
time even if the procedure has a parent.
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch,
> HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538623#comment-16538623 ] stack commented on HBASE-20847: --- bq. For table lock this should not happen, as region assign can happen at any time if there is a server crash as it needs to hold the shared lock, so if we hold the exclusive lock for the whole life of a table procedure then it will hurt the availability... You are arguing we should never take an exclusive lock on a table? For a modifytableprocedure? bq. as region assign can happen at any time if there is a server crash Assign can only proceed after WAL logs have been split... so if an SCP and a ModifyTableProcedure at same time, MTP should wait until SCP has finished log splitting before proceeding SCP should wait till MTP has done assigning/unassiging before it tries assigning? bq. But for other procedures, such as peer related procedures, we will hold the lock for the whole life time. Post-crash, how does this lock get reinstated? Currently there is no means, right? Scheduling the parent to run before the sub-procedure would go against the Pv2 grain. Does this mean that as part of the load process we should be re-instituting locks? (Is this what HBASE-20846 is supposed to be doing? Sounds like it. If so, great) > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538577#comment-16538577 ] Duo Zhang commented on HBASE-20847: --- Let me check the failed UT. Seems related. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538574#comment-16538574
]
Hadoop QA commented on HBASE-20847:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hbase-procedure: The patch generated 0 new + 1
unchanged - 5 fixed = 1 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} hbase-server: The patch generated 0 new + 8
unchanged - 1 fixed = 8 total (was 9) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}129m 44s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}176m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHStore |
| | hadoop.hbase.client.TestMobRestoreSnapshotFromClient |
| | hadoop.hbase.client.TestAdmin2 |
| | hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas |
| | hadoop.hbase.client.TestRestoreSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20847 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12930981
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538355#comment-16538355 ] Duo Zhang commented on HBASE-20847: --- Any other concerns? [~stack] And [~allan163], the patch here is similar to yours in HBASE-20846, where we always need to acquire shared lock, no matter whether we have a parent or not. Could you please also take a simple look? Thanks. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847-v2.patch, > HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537861#comment-16537861
]
Duo Zhang commented on HBASE-20847:
---
{quote}
Are we talking about the post-master crash scenario? Is the suggestion that
sub-procedures just assume and put back in place the locks that were taken out
by parent procedures when they run post-crash? And this will be ok as long as
the parent waits on all sub-procedures to complete before it lets-go of shared
or exclusive lock? (Which is how it currently works?).
{quote}
No, we will use HBASE-20846 to address the master restart problem. And here the
problem is that, the parent procedure may not hold the shared/exclusive table
lock in the normal cases, for example, SCP will not hold any table locks, but
it will schedule AssignProcedures to make regions online. So we should change
the implementation.
In HBASE-20846, the problem is that we need to restore the lock state before
starting workers when master restarts.
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-20847-v1.patch, HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537857#comment-16537857
]
Duo Zhang commented on HBASE-20847:
---
{quote}
What of the case where the parent procedure takes an exclusive lock for the
life of the Procedure?
{quote}
For table lock this should not happen, as region assign can happen at any time
if there is a server crash as it needs to hold the shared lock, so if we hold
the exclusive lock for the whole life of a table procedure then it will hurt
the availability...
But for other procedures, such as peer related procedures, we will hold the
lock for the whole life time. Of course until now we only acquire exclusive
lock for peer procedures, but the procedure framework is general, we should at
least make it work for most common cases. In java, you are free the hold the
readLock any times if you already hold the writeLock, so I think we use the
same rule here is fine?
> The parent procedure of RegionTransitionProcedure may not have the table lock
> -
>
> Key: HBASE-20847
> URL: https://issues.apache.org/jira/browse/HBASE-20847
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-20847-v1.patch, HBASE-20847.patch
>
>
> For example, SCP can also schedule AssignProcedure and obviously it will not
> hold the table lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537294#comment-16537294
]
Hadoop QA commented on HBASE-20847:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hbase-procedure: The patch generated 0 new + 1
unchanged - 5 fixed = 1 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} hbase-server: The patch generated 0 new + 5
unchanged - 1 fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 11s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
41s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}121m 34s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas |
| | hadoop.hbase.client.TestMobRestoreSnapshotFromClient |
| | hadoop.hbase.client.TestAdmin2 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20847 |
| JIRA Patch URL |
https://issues.apache.org/jira/
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537074#comment-16537074 ] stack commented on HBASE-20847: --- What of the case where the parent procedure takes an exclusive lock for the life of the Procedure? The sub-procedure might put in place an exclusive lock while it runs but it won't restore the parent procedures' lifetime lock. Is this an argument that parent procedure should run before sub-procedures to at least restore any locks it had taken (shared or exclusve, lifetime or step-duration). > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537045#comment-16537045 ] stack commented on HBASE-20847: --- bq. The locks which have already been held by parent procedures should also be considered as held by sub procedures. This is OK as we can make sure that the parent procedure will not release the lock before the sub procedures, as it can only be executed again after all the sub procedures have finished. Are we talking about the post-master crash scenario? Is the suggestion that sub-procedures assume locks that were taken out by parent procedures? > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536949#comment-16536949 ] Duo Zhang commented on HBASE-20847: --- Review board link: https://reviews.apache.org/r/67854/ > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847-v1.patch, HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[
https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536820#comment-16536820
]
Hadoop QA commented on HBASE-20847:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
6s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} hbase-procedure: The patch generated 0 new + 3
unchanged - 3 fixed = 3 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} hbase-server: The patch generated 0 new + 5
unchanged - 1 fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 57s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
44s{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
56s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 53s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 79m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Nullcheck of table at line 702 of value previously dereferenced in
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.waitRegions(Procedure,
TableName, RegionInfo[]) At MasterProcedureScheduler.java:702 of value
previously dereferenced in
org.apache.hadoop.hbase.master.procedure
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536762#comment-16536762 ] Duo Zhang commented on HBASE-20847: --- No UT yet. Will try to implement a UT soon. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-20847.patch > > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20847) The parent procedure of RegionTransitionProcedure may not have the table lock
[ https://issues.apache.org/jira/browse/HBASE-20847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536686#comment-16536686 ] Duo Zhang commented on HBASE-20847: --- And for the waitRegions method, I think we should apply the patch in HBASE-20846, i.e, always try to acquire the shared lock. But the implementation of procedure lock needs a bit modification. If the parent procedure already held the exclusive lock, instead of returning false to let the procedure wait, we should return true to let the procedure go on. The locks which have already been held by parent procedures should also be considered as held by sub procedures. This is OK as we can make sure that the parent procedure will not release the lock before the sub procedures, as it can only be executed again after all the sub procedures have finished. Copied the comment from parent here. And the logic is not simple, as when releasing a lock we need to wake other procedures. Anyway let me try to prepare a patch. > The parent procedure of RegionTransitionProcedure may not have the table lock > - > > Key: HBASE-20847 > URL: https://issues.apache.org/jira/browse/HBASE-20847 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Priority: Major > > For example, SCP can also schedule AssignProcedure and obviously it will not > hold the table lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
