[jira] [Commented] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394093#comment-16394093
]
Hadoop QA commented on HBASE-20153:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
46s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}168m 19s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}219m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20153 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913835/HBASE-20153.patch |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
shadedjars hadoopcheck xml compile |
| uname | Linux 1fdb05ed2827 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dd6f4525e7 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| shellcheck | v0.4.4 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11897/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11897/testReport/ |
| Max. process+thread count | 5074 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11897/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> enable error-prone analysis in precommit
>
>
[jira] [Commented] (HBASE-19665) Add table based replication peers/queues storage back
[
https://issues.apache.org/jira/browse/HBASE-19665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394066#comment-16394066
]
Hadoop QA commented on HBASE-19665:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 30s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}195m
46s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}236m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19665 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913867/HBASE-19665.v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux da1952f57542 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / dd6f4525e7 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs
[jira] [Commented] (HBASE-20167) Optimize the implementation of ReplicationSourceWALReader
[
https://issues.apache.org/jira/browse/HBASE-20167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394063#comment-16394063
]
Hadoop QA commented on HBASE-20167:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} hbase-server: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}199m 50s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}238m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL
|
| | hadoop.hbase.replication.regionserver.TestGlobalThrottler |
| | hadoop.hbase.replication.TestReplicationSource |
| |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL |
| | hadoop.hbase.replication.regionserver.TestWALEntryStream |
| | hadoop.hbase.replication.TestReplicationSyncUpTool |
| | hadoop.hbase.replication.TestReplicationDroppedTables |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20167 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913862/HBASE-20167.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux aaa5f167dd19 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-20169) NPE when calling HBTU.shutdownMiniCluster
[
https://issues.apache.org/jira/browse/HBASE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394060#comment-16394060
]
stack commented on HBASE-20169:
---
np. Assigned myself.
> NPE when calling HBTU.shutdownMiniCluster
> -
>
> Key: HBASE-20169
> URL: https://issues.apache.org/jira/browse/HBASE-20169
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
>
> This usually happens when some master or rs has already been down before we
> calling shutdownMiniCluster.
> See
> https://builds.apache.org/job/HBASE-Flaky-Tests/27223/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManagerMetrics/org_apache_hadoop_hbase_master_TestAssignmentManagerMetrics/
> and also
> http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34873/testReport/junit/org.apache.hadoop.hbase.master/TestRestartCluster/testRetainAssignmentOnRestart/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:156)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20169) NPE when calling HBTU.shutdownMiniCluster
[
https://issues.apache.org/jira/browse/HBASE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reassigned HBASE-20169:
-
Assignee: stack
> NPE when calling HBTU.shutdownMiniCluster
> -
>
> Key: HBASE-20169
> URL: https://issues.apache.org/jira/browse/HBASE-20169
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Major
>
> This usually happens when some master or rs has already been down before we
> calling shutdownMiniCluster.
> See
> https://builds.apache.org/job/HBASE-Flaky-Tests/27223/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManagerMetrics/org_apache_hadoop_hbase_master_TestAssignmentManagerMetrics/
> and also
> http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34873/testReport/junit/org.apache.hadoop.hbase.master/TestRestartCluster/testRetainAssignmentOnRestart/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:156)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20169) NPE when calling HBTU.shutdownMiniCluster
[
https://issues.apache.org/jira/browse/HBASE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20169:
--
Priority: Critical (was: Major)
> NPE when calling HBTU.shutdownMiniCluster
> -
>
> Key: HBASE-20169
> URL: https://issues.apache.org/jira/browse/HBASE-20169
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
>
> This usually happens when some master or rs has already been down before we
> calling shutdownMiniCluster.
> See
> https://builds.apache.org/job/HBASE-Flaky-Tests/27223/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManagerMetrics/org_apache_hadoop_hbase_master_TestAssignmentManagerMetrics/
> and also
> http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34873/testReport/junit/org.apache.hadoop.hbase.master/TestRestartCluster/testRetainAssignmentOnRestart/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:156)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20146) Regions are stuck while opening when WAL is disabled
[
https://issues.apache.org/jira/browse/HBASE-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394049#comment-16394049
]
Hudson commented on HBASE-20146:
Results for branch branch-1
[build #245 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Regions are stuck while opening when WAL is disabled
>
>
> Key: HBASE-20146
> URL: https://issues.apache.org/jira/browse/HBASE-20146
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>
> Attachments: HBASE-20146.patch, HBASE-20146.v1.patch
>
>
> On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false,
> to disable the WAL for complete cluster, after restarting HBase service,
> regions are not getting opened leading to HMaster abort as Namespace table
> regions are not getting assigned.
> jstack for region open:
> {noformat}
> "RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0
> tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x87554448> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> This used to work with HBase 1.0.2 version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394050#comment-16394050
]
Hudson commented on HBASE-20164:
Results for branch branch-1
[build #245 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[
https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394051#comment-16394051
]
Hudson commented on HBASE-20162:
Results for branch branch-1
[build #245 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/245//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [nightly] depending on pipeline execution we sometimes refer to the wrong
> workspace
> ---
>
> Key: HBASE-20162
> URL: https://issues.apache.org/jira/browse/HBASE-20162
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20162.0.patch
>
>
> we set BASEDIR at the top of our pipeline to point at the component checkout
> within WORKSPACE.
> but!
> a) at that point WORKSPACE is the workspace for the launching task
> b) sometimes our parallel executions get a task with a different local
> WORKSPACE to allow for coexisting on the same build host
> c) when this happens our parallel stages are referring to some other absolute
> path on the host
> d) in most cases we're referring to dev-support files like e.g. the nightly
> build script or machine info script that are the same across branches so
> things are fine if we aren't running at the same time as a job that's
> overwritting them
> e) we also refer to the Dockerfile this way, so weird bugs I'm sure.
> f) we build the source tarball from here, so that's probably broken subtly
> g) sometimes that other directory _doesn't exist at all_ and we fail with
> confusing messages about stuff not found
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20146) Regions are stuck while opening when WAL is disabled
[
https://issues.apache.org/jira/browse/HBASE-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394045#comment-16394045
]
Chia-Ping Tsai commented on HBASE-20146:
Should we update the seq id of cells passed to DisabledWALProvider? Is the view
of scanner impacted by the lack of update to seq id?
> Regions are stuck while opening when WAL is disabled
>
>
> Key: HBASE-20146
> URL: https://issues.apache.org/jira/browse/HBASE-20146
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>
> Attachments: HBASE-20146.patch, HBASE-20146.v1.patch
>
>
> On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false,
> to disable the WAL for complete cluster, after restarting HBase service,
> regions are not getting opened leading to HMaster abort as Namespace table
> regions are not getting assigned.
> jstack for region open:
> {noformat}
> "RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0
> tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x87554448> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> This used to work with HBase 1.0.2 version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20132) Change the "KV" to "Cell" for web UI
[ https://issues.apache.org/jira/browse/HBASE-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394036#comment-16394036 ] Chia-Ping Tsai commented on HBASE-20132: I have pushed the patch to branch-2 and master. Keep this open to wait the feedback from [~stack] > Change the "KV" to "Cell" for web UI > > > Key: HBASE-20132 > URL: https://issues.apache.org/jira/browse/HBASE-20132 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Minor > Labels: beginner, beginners > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20132.master.001.patch > > > grep the source code. The related words which should be revised are shown > below. > # Num. Compacting KVs > # Num. Compacted KVs > # Remaining KVs > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19404) Fix DisabledWALProvider
[
https://issues.apache.org/jira/browse/HBASE-19404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-19404:
---
Resolution: Duplicate
Status: Resolved (was: Patch Available)
see HBASE-20146
> Fix DisabledWALProvider
> ---
>
> Key: HBASE-19404
> URL: https://issues.apache.org/jira/browse/HBASE-19404
> Project: HBase
> Issue Type: Task
> Components: wal
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19404.v0.patch, HBASE-19404.v1.patch
>
>
> {{TestFromClientSide3}} fails with {{DisabledWALProvider}}. If no use case
> exists for {{DisabledWALProvider}}, we can just remove it.
> Any suggestions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve fault tolerance in backup merge operation
[
https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394032#comment-16394032
]
Ted Yu commented on HBASE-19969:
Vlad:
Can you address checkstyle warnings ?
{code}
./hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupMerge.java:33:import
org.junit.Assert;: Wrong order for 'org.junit.Assert' import. [ImportOrder]
./hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupMerge.java:39:import
static org.junit.Assert.assertTrue;: Wrong order for
'org.junit.Assert.assertTrue' import. [ImportOrder]
{code}
> Improve fault tolerance in backup merge operation
> -
>
> Key: HBASE-19969
> URL: https://issues.apache.org/jira/browse/HBASE-19969
> Project: HBase
> Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch,
> HBASE-19969-v3.patch
>
>
> Some file system operations are not fault tolerant during merge. We delete
> backup data in a backup file system, then copy new data over to backup
> destination. Deletes can be partial, copy can fail as well
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve fault tolerance in backup merge operation
[
https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394028#comment-16394028
]
Hadoop QA commented on HBASE-19969:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
27s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-backup: The patch generated 2 new + 0 unchanged
- 0 fixed = 2 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
18s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19969 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913857/HBASE-19969-v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux bf486dc27f43 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / dd6f4525e7 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11898/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/11898/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
[jira] [Updated] (HBASE-20120) Remove some unused classes/ java files from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20120: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the patch. [~uagashe] > Remove some unused classes/ java files from hbase-server > > > Key: HBASE-20120 > URL: https://issues.apache.org/jira/browse/HBASE-20120 > Project: HBase > Issue Type: Improvement >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 3.0.0, 2.1.0 > > Attachments: hbase-20120.master.001.patch > > > Following classes/ Java files are not used, can be removed: > hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProtoUtil.java > hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSMapRUtils.java > hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseWALEntryFilter.java > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoOpHeapMemoryTuner.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[
https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394024#comment-16394024
]
Hudson commented on HBASE-20162:
Results for branch master
[build #257 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/257/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [nightly] depending on pipeline execution we sometimes refer to the wrong
> workspace
> ---
>
> Key: HBASE-20162
> URL: https://issues.apache.org/jira/browse/HBASE-20162
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20162.0.patch
>
>
> we set BASEDIR at the top of our pipeline to point at the component checkout
> within WORKSPACE.
> but!
> a) at that point WORKSPACE is the workspace for the launching task
> b) sometimes our parallel executions get a task with a different local
> WORKSPACE to allow for coexisting on the same build host
> c) when this happens our parallel stages are referring to some other absolute
> path on the host
> d) in most cases we're referring to dev-support files like e.g. the nightly
> build script or machine info script that are the same across branches so
> things are fine if we aren't running at the same time as a job that's
> overwritting them
> e) we also refer to the Dockerfile this way, so weird bugs I'm sure.
> f) we build the source tarball from here, so that's probably broken subtly
> g) sometimes that other directory _doesn't exist at all_ and we fail with
> confusing messages about stuff not found
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[
https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394021#comment-16394021
]
Hudson commented on HBASE-16179:
Results for branch master
[build #257 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/257/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix compilation errors when building hbase-spark against Spark 2.0
> --
>
> Key: HBASE-16179
> URL: https://issues.apache.org/jira/browse/HBASE-16179
> Project: HBase
> Issue Type: Bug
> Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Labels: build
> Fix For: 3.0.0
>
> Attachments: 16179.v0.txt, 16179.v1.txt, 16179.v1.txt, 16179.v10.txt,
> 16179.v11.txt, 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt,
> 16179.v15.txt, 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt,
> 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, 16179.v25.txt,
> 16179.v26.txt, 16179.v27.txt, 16179.v28.txt, 16179.v28.txt, 16179.v29.txt,
> 16179.v30.txt, 16179.v31.txt, 16179.v32.txt, 16179.v33.txt, 16179.v34.txt,
> 16179.v35.txt, 16179.v36.txt, 16179.v37.txt, 16179.v38.txt, 16179.v38.txt,
> 16179.v4.txt, 16179.v5.txt, 16179.v7.txt, 16179.v8.txt, 16179.v9.txt,
> HBASE-16179.v29.patch
>
>
> I tried building hbase-spark module against Spark-2.0 snapshot and got the
> following compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer
> accessible to downstream projects.
> hbase-spark module should not depend on such classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20024) TestMergeTableRegionsProcedure is STILL flakey
[
https://issues.apache.org/jira/browse/HBASE-20024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394022#comment-16394022
]
Hudson commented on HBASE-20024:
Results for branch master
[build #257 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/257/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> TestMergeTableRegionsProcedure is STILL flakey
> --
>
> Key: HBASE-20024
> URL: https://issues.apache.org/jira/browse/HBASE-20024
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20024.branch-2.001.patch,
> hbase-20024.master.001.patch, hbase-20024.master.002.patch,
> hbase-20024.master.003.patch, hbase-20024.master.004.patch
>
>
> This is a follow-on from HBASE-20015. Root issue is that merge does not
> support rollback once it has hit the point-of-no-return; it can only
> roll-forward at this point.
> HBASE-18018 added abort to all procedures. HBASE-18016 added ignoring abort
> to the truncate procedure to get around flakeyness. HBASE-20022 is a new
> issue to figure what to do w/ unabortable procedures.
> Meantime, merge and split have PONR and the procedure executor test harness
> does abort regardless making these tests flakies. Adding an ignore of the
> abort once into the PONR makes sense (Always? HBASE-20022 is for those as yet
> unknown cases where it does not). Let me ignore to merge and split in this
> issue to address flakeyness.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20146) Regions are stuck while opening when WAL is disabled
[
https://issues.apache.org/jira/browse/HBASE-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394020#comment-16394020
]
Hudson commented on HBASE-20146:
Results for branch master
[build #257 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/257/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Regions are stuck while opening when WAL is disabled
>
>
> Key: HBASE-20146
> URL: https://issues.apache.org/jira/browse/HBASE-20146
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>
> Attachments: HBASE-20146.patch, HBASE-20146.v1.patch
>
>
> On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false,
> to disable the WAL for complete cluster, after restarting HBase service,
> regions are not getting opened leading to HMaster abort as Namespace table
> regions are not getting assigned.
> jstack for region open:
> {noformat}
> "RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0
> tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x87554448> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> This used to work with HBase 1.0.2 version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394023#comment-16394023
]
Hudson commented on HBASE-20164:
Results for branch master
[build #257 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/257/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/257//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20169) NPE when calling HBTU.shutdownMiniCluster
[
https://issues.apache.org/jira/browse/HBASE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394018#comment-16394018
]
Duo Zhang commented on HBASE-20169:
---
Do not have any ideas on how to find out the real problem yet, it is not easy
to reproduce and the stack trace is meaningless...
So if you have time please take a look at it sir.
Thanks.
> NPE when calling HBTU.shutdownMiniCluster
> -
>
> Key: HBASE-20169
> URL: https://issues.apache.org/jira/browse/HBASE-20169
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Priority: Major
>
> This usually happens when some master or rs has already been down before we
> calling shutdownMiniCluster.
> See
> https://builds.apache.org/job/HBASE-Flaky-Tests/27223/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManagerMetrics/org_apache_hadoop_hbase_master_TestAssignmentManagerMetrics/
> and also
> http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34873/testReport/junit/org.apache.hadoop.hbase.master/TestRestartCluster/testRetainAssignmentOnRestart/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:156)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20167) Optimize the implementation of ReplicationSourceWALReader
[ https://issues.apache.org/jira/browse/HBASE-20167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394015#comment-16394015 ] Duo Zhang commented on HBASE-20167: --- Review board link: https://reviews.apache.org/r/66018/ > Optimize the implementation of ReplicationSourceWALReader > - > > Key: HBASE-20167 > URL: https://issues.apache.org/jira/browse/HBASE-20167 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20167.patch > > > After HBASE-20148, serial replication will be an option for peer. Since an > instance of ReplicationSourceWALReader can only belongs to one peer, we do > not need to add the so many 'if' in the implementation of readWALEntries to > check whether we should consider serial replication. We can just make a sub > class or something similiar for serial replication to make the code clean. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20120) Remove some unused classes/ java files from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394006#comment-16394006 ] Chia-Ping Tsai edited comment on HBASE-20120 at 3/10/18 4:33 AM: - Will commit it to master and branch-2 was (Author: chia7712): Will commit it later. > Remove some unused classes/ java files from hbase-server > > > Key: HBASE-20120 > URL: https://issues.apache.org/jira/browse/HBASE-20120 > Project: HBase > Issue Type: Improvement >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 3.0.0, 2.1.0 > > Attachments: hbase-20120.master.001.patch > > > Following classes/ Java files are not used, can be removed: > hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProtoUtil.java > hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSMapRUtils.java > hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseWALEntryFilter.java > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoOpHeapMemoryTuner.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20120) Remove some unused classes/ java files from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20120: --- Fix Version/s: (was: 2.0.0) 2.1.0 3.0.0 > Remove some unused classes/ java files from hbase-server > > > Key: HBASE-20120 > URL: https://issues.apache.org/jira/browse/HBASE-20120 > Project: HBase > Issue Type: Improvement >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 3.0.0, 2.1.0 > > Attachments: hbase-20120.master.001.patch > > > Following classes/ Java files are not used, can be removed: > hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProtoUtil.java > hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSMapRUtils.java > hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseWALEntryFilter.java > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoOpHeapMemoryTuner.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20132) Change the "KV" to "Cell" for web UI
[ https://issues.apache.org/jira/browse/HBASE-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20132: --- Fix Version/s: (was: 2.0.0) 2.1.0 3.0.0 > Change the "KV" to "Cell" for web UI > > > Key: HBASE-20132 > URL: https://issues.apache.org/jira/browse/HBASE-20132 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Minor > Labels: beginner, beginners > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20132.master.001.patch > > > grep the source code. The related words which should be revised are shown > below. > # Num. Compacting KVs > # Num. Compacted KVs > # Remaining KVs > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20132) Change the "KV" to "Cell" for web UI
[ https://issues.apache.org/jira/browse/HBASE-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394014#comment-16394014 ] Chia-Ping Tsai commented on HBASE-20132: Will push it to master and branch-2. [~stack] Do you want this patch to branch-2.0? > Change the "KV" to "Cell" for web UI > > > Key: HBASE-20132 > URL: https://issues.apache.org/jira/browse/HBASE-20132 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Guangxu Cheng >Priority: Minor > Labels: beginner, beginners > Fix For: 2.0.0 > > Attachments: HBASE-20132.master.001.patch > > > grep the source code. The related words which should be revised are shown > below. > # Num. Compacting KVs > # Num. Compacted KVs > # Remaining KVs > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[
https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394011#comment-16394011
]
Hudson commented on HBASE-20162:
Results for branch branch-2.0
[build #13 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/13/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/13//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/13//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/13//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [nightly] depending on pipeline execution we sometimes refer to the wrong
> workspace
> ---
>
> Key: HBASE-20162
> URL: https://issues.apache.org/jira/browse/HBASE-20162
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20162.0.patch
>
>
> we set BASEDIR at the top of our pipeline to point at the component checkout
> within WORKSPACE.
> but!
> a) at that point WORKSPACE is the workspace for the launching task
> b) sometimes our parallel executions get a task with a different local
> WORKSPACE to allow for coexisting on the same build host
> c) when this happens our parallel stages are referring to some other absolute
> path on the host
> d) in most cases we're referring to dev-support files like e.g. the nightly
> build script or machine info script that are the same across branches so
> things are fine if we aren't running at the same time as a job that's
> overwritting them
> e) we also refer to the Dockerfile this way, so weird bugs I'm sure.
> f) we build the source tarball from here, so that's probably broken subtly
> g) sometimes that other directory _doesn't exist at all_ and we fail with
> confusing messages about stuff not found
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[
https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394010#comment-16394010
]
Hudson commented on HBASE-20162:
Results for branch branch-2
[build #466 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [nightly] depending on pipeline execution we sometimes refer to the wrong
> workspace
> ---
>
> Key: HBASE-20162
> URL: https://issues.apache.org/jira/browse/HBASE-20162
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20162.0.patch
>
>
> we set BASEDIR at the top of our pipeline to point at the component checkout
> within WORKSPACE.
> but!
> a) at that point WORKSPACE is the workspace for the launching task
> b) sometimes our parallel executions get a task with a different local
> WORKSPACE to allow for coexisting on the same build host
> c) when this happens our parallel stages are referring to some other absolute
> path on the host
> d) in most cases we're referring to dev-support files like e.g. the nightly
> build script or machine info script that are the same across branches so
> things are fine if we aren't running at the same time as a job that's
> overwritting them
> e) we also refer to the Dockerfile this way, so weird bugs I'm sure.
> f) we build the source tarball from here, so that's probably broken subtly
> g) sometimes that other directory _doesn't exist at all_ and we fail with
> confusing messages about stuff not found
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394009#comment-16394009
]
Hudson commented on HBASE-20164:
Results for branch branch-2
[build #466 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20024) TestMergeTableRegionsProcedure is STILL flakey
[
https://issues.apache.org/jira/browse/HBASE-20024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394008#comment-16394008
]
Hudson commented on HBASE-20024:
Results for branch branch-2
[build #466 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/466//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> TestMergeTableRegionsProcedure is STILL flakey
> --
>
> Key: HBASE-20024
> URL: https://issues.apache.org/jira/browse/HBASE-20024
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20024.branch-2.001.patch,
> hbase-20024.master.001.patch, hbase-20024.master.002.patch,
> hbase-20024.master.003.patch, hbase-20024.master.004.patch
>
>
> This is a follow-on from HBASE-20015. Root issue is that merge does not
> support rollback once it has hit the point-of-no-return; it can only
> roll-forward at this point.
> HBASE-18018 added abort to all procedures. HBASE-18016 added ignoring abort
> to the truncate procedure to get around flakeyness. HBASE-20022 is a new
> issue to figure what to do w/ unabortable procedures.
> Meantime, merge and split have PONR and the procedure executor test harness
> does abort regardless making these tests flakies. Adding an ignore of the
> abort once into the PONR makes sense (Always? HBASE-20022 is for those as yet
> unknown cases where it does not). Let me ignore to merge and split in this
> issue to address flakeyness.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394007#comment-16394007
]
Hadoop QA commented on HBASE-20153:
---
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at
https://builds.apache.org/job/PreCommit-HBASE-Build/11897/console in case of
problems.
> enable error-prone analysis in precommit
>
>
> Key: HBASE-20153
> URL: https://issues.apache.org/jira/browse/HBASE-20153
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20153.patch
>
>
> We've done a lot of work to get rid of the error-prone errors, we should make
> sure they stay out. Let's enable errorProne profile and analysis in precommit.
> [~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the
> mvn compile precommit check but was unable to unravel that thread. Any help
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20120) Remove some unused classes/ java files from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394006#comment-16394006 ] Chia-Ping Tsai commented on HBASE-20120: Will commit it later. > Remove some unused classes/ java files from hbase-server > > > Key: HBASE-20120 > URL: https://issues.apache.org/jira/browse/HBASE-20120 > Project: HBase > Issue Type: Improvement >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0 > > Attachments: hbase-20120.master.001.patch > > > Following classes/ Java files are not used, can be removed: > hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProtoUtil.java > hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSMapRUtils.java > hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseWALEntryFilter.java > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoOpHeapMemoryTuner.java > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393999#comment-16393999
]
Sean Busbey commented on HBASE-20153:
-
looks like it took ~an hour to get to run the unit tests, then the unit tests
got 3 hours into their thing and teh node, H18, died.
I'll redo the job.
> enable error-prone analysis in precommit
>
>
> Key: HBASE-20153
> URL: https://issues.apache.org/jira/browse/HBASE-20153
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20153.patch
>
>
> We've done a lot of work to get rid of the error-prone errors, we should make
> sure they stay out. Let's enable errorProne profile and analysis in precommit.
> [~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the
> mvn compile precommit check but was unable to unravel that thread. Any help
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393998#comment-16393998
]
Sean Busbey commented on HBASE-20153:
-
let's go see where precommit went!
> enable error-prone analysis in precommit
>
>
> Key: HBASE-20153
> URL: https://issues.apache.org/jira/browse/HBASE-20153
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20153.patch
>
>
> We've done a lot of work to get rid of the error-prone errors, we should make
> sure they stay out. Let's enable errorProne profile and analysis in precommit.
> [~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the
> mvn compile precommit check but was unable to unravel that thread. Any help
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19969) Improve fault tolerance in backup merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-19969: --- Summary: Improve fault tolerance in backup merge operation (was: Improve FT in merge operation) > Improve fault tolerance in backup merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch, > HBASE-19969-v3.patch > > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[
https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393984#comment-16393984
]
Hudson commented on HBASE-20162:
Results for branch branch-1.4
[build #250 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [nightly] depending on pipeline execution we sometimes refer to the wrong
> workspace
> ---
>
> Key: HBASE-20162
> URL: https://issues.apache.org/jira/browse/HBASE-20162
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20162.0.patch
>
>
> we set BASEDIR at the top of our pipeline to point at the component checkout
> within WORKSPACE.
> but!
> a) at that point WORKSPACE is the workspace for the launching task
> b) sometimes our parallel executions get a task with a different local
> WORKSPACE to allow for coexisting on the same build host
> c) when this happens our parallel stages are referring to some other absolute
> path on the host
> d) in most cases we're referring to dev-support files like e.g. the nightly
> build script or machine info script that are the same across branches so
> things are fine if we aren't running at the same time as a job that's
> overwritting them
> e) we also refer to the Dockerfile this way, so weird bugs I'm sure.
> f) we build the source tarball from here, so that's probably broken subtly
> g) sometimes that other directory _doesn't exist at all_ and we fail with
> confusing messages about stuff not found
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393983#comment-16393983
]
Hudson commented on HBASE-20164:
Results for branch branch-1.4
[build #250 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/250//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393975#comment-16393975
]
Hudson commented on HBASE-20164:
Results for branch branch-1.3
[build #259 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393978#comment-16393978
]
Hudson commented on HBASE-20164:
Results for branch branch-1.2
[build #262 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[
https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393976#comment-16393976
]
Hudson commented on HBASE-20162:
Results for branch branch-1.3
[build #259 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/259//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [nightly] depending on pipeline execution we sometimes refer to the wrong
> workspace
> ---
>
> Key: HBASE-20162
> URL: https://issues.apache.org/jira/browse/HBASE-20162
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20162.0.patch
>
>
> we set BASEDIR at the top of our pipeline to point at the component checkout
> within WORKSPACE.
> but!
> a) at that point WORKSPACE is the workspace for the launching task
> b) sometimes our parallel executions get a task with a different local
> WORKSPACE to allow for coexisting on the same build host
> c) when this happens our parallel stages are referring to some other absolute
> path on the host
> d) in most cases we're referring to dev-support files like e.g. the nightly
> build script or machine info script that are the same across branches so
> things are fine if we aren't running at the same time as a job that's
> overwritting them
> e) we also refer to the Dockerfile this way, so weird bugs I'm sure.
> f) we build the source tarball from here, so that's probably broken subtly
> g) sometimes that other directory _doesn't exist at all_ and we fail with
> confusing messages about stuff not found
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[
https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393977#comment-16393977
]
Hudson commented on HBASE-20162:
Results for branch branch-1.2
[build #262 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/262//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [nightly] depending on pipeline execution we sometimes refer to the wrong
> workspace
> ---
>
> Key: HBASE-20162
> URL: https://issues.apache.org/jira/browse/HBASE-20162
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20162.0.patch
>
>
> we set BASEDIR at the top of our pipeline to point at the component checkout
> within WORKSPACE.
> but!
> a) at that point WORKSPACE is the workspace for the launching task
> b) sometimes our parallel executions get a task with a different local
> WORKSPACE to allow for coexisting on the same build host
> c) when this happens our parallel stages are referring to some other absolute
> path on the host
> d) in most cases we're referring to dev-support files like e.g. the nightly
> build script or machine info script that are the same across branches so
> things are fine if we aren't running at the same time as a job that's
> overwritting them
> e) we also refer to the Dockerfile this way, so weird bugs I'm sure.
> f) we build the source tarball from here, so that's probably broken subtly
> g) sometimes that other directory _doesn't exist at all_ and we fail with
> confusing messages about stuff not found
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve FT in merge operation
[
https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393967#comment-16393967
]
Hadoop QA commented on HBASE-19969:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hbase-backup: The patch generated 2 new + 0 unchanged
- 0 fixed = 2 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 55s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
57s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19969 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913857/HBASE-19969-v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 69ead85f1c66 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dd6f4525e7 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11894/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/11894/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
[jira] [Updated] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16179: --- Release Note: As of this JIRA, Spark version is upgraded from 1.6 to 2.1.1 > Fix compilation errors when building hbase-spark against Spark 2.0 > -- > > Key: HBASE-16179 > URL: https://issues.apache.org/jira/browse/HBASE-16179 > Project: HBase > Issue Type: Bug > Components: spark >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Labels: build > Fix For: 3.0.0 > > Attachments: 16179.v0.txt, 16179.v1.txt, 16179.v1.txt, 16179.v10.txt, > 16179.v11.txt, 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, > 16179.v15.txt, 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, > 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, 16179.v25.txt, > 16179.v26.txt, 16179.v27.txt, 16179.v28.txt, 16179.v28.txt, 16179.v29.txt, > 16179.v30.txt, 16179.v31.txt, 16179.v32.txt, 16179.v33.txt, 16179.v34.txt, > 16179.v35.txt, 16179.v36.txt, 16179.v37.txt, 16179.v38.txt, 16179.v38.txt, > 16179.v4.txt, 16179.v5.txt, 16179.v7.txt, 16179.v8.txt, 16179.v9.txt, > HBASE-16179.v29.patch > > > I tried building hbase-spark module against Spark-2.0 snapshot and got the > following compilation errors: > http://pastebin.com/bg3w247a > Some Spark classes such as DataTypeParser and Logging are no longer > accessible to downstream projects. > hbase-spark module should not depend on such classes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393964#comment-16393964
]
Hudson commented on HBASE-20164:
Results for branch branch-2.0
[build #12 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/12/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/12//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/12//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20024) TestMergeTableRegionsProcedure is STILL flakey
[
https://issues.apache.org/jira/browse/HBASE-20024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393963#comment-16393963
]
Hudson commented on HBASE-20024:
Results for branch branch-2.0
[build #12 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/12/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/12//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/12//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> TestMergeTableRegionsProcedure is STILL flakey
> --
>
> Key: HBASE-20024
> URL: https://issues.apache.org/jira/browse/HBASE-20024
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20024.branch-2.001.patch,
> hbase-20024.master.001.patch, hbase-20024.master.002.patch,
> hbase-20024.master.003.patch, hbase-20024.master.004.patch
>
>
> This is a follow-on from HBASE-20015. Root issue is that merge does not
> support rollback once it has hit the point-of-no-return; it can only
> roll-forward at this point.
> HBASE-18018 added abort to all procedures. HBASE-18016 added ignoring abort
> to the truncate procedure to get around flakeyness. HBASE-20022 is a new
> issue to figure what to do w/ unabortable procedures.
> Meantime, merge and split have PONR and the procedure executor test harness
> does abort regardless making these tests flakies. Adding an ignore of the
> abort once into the PONR makes sense (Always? HBASE-20022 is for those as yet
> unknown cases where it does not). Let me ignore to merge and split in this
> issue to address flakeyness.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19665) Add table based replication peers/queues storage back
[ https://issues.apache.org/jira/browse/HBASE-19665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393948#comment-16393948 ] Zheng Hu commented on HBASE-19665: -- Re-attached the patch.v3 for fix checkstyle & findbugs. All UT passed. Ping [~zghaobac] & [~Apache9] for reviewing. > Add table based replication peers/queues storage back > - > > Key: HBASE-19665 > URL: https://issues.apache.org/jira/browse/HBASE-19665 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Guanghao Zhang >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-19665.v1.patch, HBASE-19665.v2.patch, > HBASE-19665.v3.patch, HBASE-19665.v3.patch > > > We removed them after HBASE-19397. So open a issue to track this thing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20131) NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
[
https://issues.apache.org/jira/browse/HBASE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20131:
--
Issue Type: Sub-task (was: Bug)
Parent: HBASE-20152
> NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
> ---
>
> Key: HBASE-20131
> URL: https://issues.apache.org/jira/browse/HBASE-20131
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20131.001.patch
>
>
> I believe the error is that a MoveRegionProcedure comes in via ChaosMonkey
> for an unassigned region that was from a disabled table (also due to CM)
> which causes an NPE as we try to set a null original location into the
> protobuf which fails.
> {noformat}
> 2018-03-02 23:07:00,146 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=23,queue=2,port=2] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos$MoveRegionStateData$Builder.setSourceServer(MasterProcedureProtos.java:26127)
> at
> org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.serializeStateData(MoveRegionProcedure.java:133)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProtoProcedure(ProcedureUtil.java:198)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeEntry(ProcedureWALFormat.java:211)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeInsert(ProcedureWALFormat.java:222)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:490)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:863)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:832)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:111)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:561)
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1707)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1324)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){noformat}
> IntegrationTestLoadAndVerify also failed, but I'm not sure if it's related to
> this, or just a problem with the test. The test failed because the table was
> left offline after it was disabled, and appears to not have been re-enabled.
> Still debugging that side..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19665) Add table based replication peers/queues storage back
[ https://issues.apache.org/jira/browse/HBASE-19665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-19665: - Attachment: HBASE-19665.v3.patch > Add table based replication peers/queues storage back > - > > Key: HBASE-19665 > URL: https://issues.apache.org/jira/browse/HBASE-19665 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Guanghao Zhang >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-19665.v1.patch, HBASE-19665.v2.patch, > HBASE-19665.v3.patch, HBASE-19665.v3.patch > > > We removed them after HBASE-19397. So open a issue to track this thing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20137) TestRSGroups is flakey
[ https://issues.apache.org/jira/browse/HBASE-20137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20137: -- Status: Open (was: Patch Available) Cancelling patch. This patch went the wrong route. It allowed unassign complete if it could not schedule a server expire (Because concurrent server crash procedure in progress). Problem with this is that if the unassign were done as part of a move, we'd next go to the assign step and could online a region before its logs had split. Will be back. Working in umbrella issue HBASE-20152 first. > TestRSGroups is flakey > -- > > Key: HBASE-20137 > URL: https://issues.apache.org/jira/browse/HBASE-20137 > Project: HBase > Issue Type: Bug > Components: flakey >Affects Versions: 2.0.0-beta-2 >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-20137.branch-2.001.patch, > HBASE-20137.branch-2.002.patch, HBASE-20137.branch-2.003.patch, > HBASE-20137.branch-2.003.patch > > > It was the single test that failed the hbase-2 nightlies in #440 at the > hadoop2 stage. > The failure manifests as a timeout. It actually has an interesting cause > calling into question some of the clauses in > UnassignProcedure#remoteCallFailed. > We are running a disabletable concurrent with a shutdown. pid=309 is the > disable. pid=311 is the interesting one. The below is a little hard to read > -- the exception 'message' is the the current procedure as a String... hard > to parse, fixing -- but we are trying to unassign as part of a the > disabletable. Our RPC fails because the server we are trying to rpc too is > currently being processed as crashed (pid=308 is a servercrashprocedure for > this server). As part of the processing of the failed RPC we will expire the > server -- if we can't RPC to it, it must be gone. The current procedure is > then suspended until it gets woken up by the servercrashprocedure triggered > by the expire only in this case we are shutting down so the expire is > ignored... The current procedure is left in its suspend state. This prevents > the Master going down. So we time out. > 2018-03-05 11:29:22,507 INFO [PEWorker-13] > assignment.RegionTransitionProcedure(213): Dispatch pid=311, ppid=309, > state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524; rit=CLOSING, > location=1cfd208ff882,40584,1520249102524 > 2018-03-05 11:29:22,508 WARN [PEWorker-13] > assignment.RegionTransitionProcedure(187): Remote call failed pid=311, > ppid=309, state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524; rit=CLOSING, > location=1cfd208ff882,40584,1520249102524; exception=pid=311, ppid=309, > state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524 to 1cfd208ff882,40584,1520249102524 > 2018-03-05 11:29:22,508 WARN [PEWorker-13] > assignment.UnassignProcedure(276): Expiring server pid=311, ppid=309, > state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524; rit=CLOSING, > location=1cfd208ff882,40584,1520249102524, > exception=org.apache.hadoop.hbase.master.assignment.FailedRemoteDispatchException: > pid=311, ppid=309, state=RUNNABLE:REGION_TRANSITION_DISPATCH; > UnassignProcedure table=Group_ns:testKillRS, > region=de7534c208a06502537cd95c248b3043, > server=1cfd208ff882,40584,1520249102524 to 1cfd208ff882,40584,1520249102524 > 2018-03-05 11:29:22,508 WARN [PEWorker-13] master.ServerManager(580): > Expiration of 1cfd208ff882,40584,1520249102524 but server shutdown already in > progress > I need to cater for case where the expire server is rejected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393945#comment-16393945 ] Josh Elser commented on HBASE-19969: Looks like my comments were addressed. +1 from me if you're happy with it, Ted. > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch, > HBASE-19969-v3.patch > > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393940#comment-16393940 ] Ted Yu commented on HBASE-19969: [~elserj]: Do you want to take another look at v3 ? > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch, > HBASE-19969-v3.patch > > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20169) NPE when calling HBTU.shutdownMiniCluster
[
https://issues.apache.org/jira/browse/HBASE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393931#comment-16393931
]
stack commented on HBASE-20169:
---
Shout if you want me to take it [~Apache9]
> NPE when calling HBTU.shutdownMiniCluster
> -
>
> Key: HBASE-20169
> URL: https://issues.apache.org/jira/browse/HBASE-20169
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Priority: Major
>
> This usually happens when some master or rs has already been down before we
> calling shutdownMiniCluster.
> See
> https://builds.apache.org/job/HBASE-Flaky-Tests/27223/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManagerMetrics/org_apache_hadoop_hbase_master_TestAssignmentManagerMetrics/
> and also
> http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34873/testReport/junit/org.apache.hadoop.hbase.master/TestRestartCluster/testRetainAssignmentOnRestart/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:156)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20148) Make serial replication as a option for a peer instead of a table
[ https://issues.apache.org/jira/browse/HBASE-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20148: -- Component/s: Replication > Make serial replication as a option for a peer instead of a table > - > > Key: HBASE-20148 > URL: https://issues.apache.org/jira/browse/HBASE-20148 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20148-v1.patch, HBASE-20148-v2.patch, > HBASE-20148-v3.patch, HBASE-20148.patch > > > Discussed with [~zghaobac] and [~openinx] offline, HBASE-20147 can only be > solved by adding new stages when creating a new replication peer. It will > also effect he processing even if you do not have any table with serial > replication. And also, the new logic introduced in ReplicationSource related > classes will also be executed even if you do not have serial replication > tables. > So, we think that it maybe a better choice to move the option to > ReplicationPeerConfig. Since if there is one table in the peer requires > serial replication, then all the tables in this peer will be automatically > 'serial' since the wal for these tables are mixed with the one which requires > serial replication. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20167) Optimize the implementation of ReplicationSourceWALReader
[ https://issues.apache.org/jira/browse/HBASE-20167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20167: -- Assignee: Duo Zhang Fix Version/s: 3.0.0 Status: Patch Available (was: Open) > Optimize the implementation of ReplicationSourceWALReader > - > > Key: HBASE-20167 > URL: https://issues.apache.org/jira/browse/HBASE-20167 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20167.patch > > > After HBASE-20148, serial replication will be an option for peer. Since an > instance of ReplicationSourceWALReader can only belongs to one peer, we do > not need to add the so many 'if' in the implementation of readWALEntries to > check whether we should consider serial replication. We can just make a sub > class or something similiar for serial replication to make the code clean. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20167) Optimize the implementation of ReplicationSourceWALReader
[ https://issues.apache.org/jira/browse/HBASE-20167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20167: -- Attachment: HBASE-20167.patch > Optimize the implementation of ReplicationSourceWALReader > - > > Key: HBASE-20167 > URL: https://issues.apache.org/jira/browse/HBASE-20167 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20167.patch > > > After HBASE-20148, serial replication will be an option for peer. Since an > instance of ReplicationSourceWALReader can only belongs to one peer, we do > not need to add the so many 'if' in the implementation of readWALEntries to > check whether we should consider serial replication. We can just make a sub > class or something similiar for serial replication to make the code clean. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20148) Make serial replication as a option for a peer instead of a table
[ https://issues.apache.org/jira/browse/HBASE-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20148: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks [~openinx] for reviewing. > Make serial replication as a option for a peer instead of a table > - > > Key: HBASE-20148 > URL: https://issues.apache.org/jira/browse/HBASE-20148 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20148-v1.patch, HBASE-20148-v2.patch, > HBASE-20148-v3.patch, HBASE-20148.patch > > > Discussed with [~zghaobac] and [~openinx] offline, HBASE-20147 can only be > solved by adding new stages when creating a new replication peer. It will > also effect he processing even if you do not have any table with serial > replication. And also, the new logic introduced in ReplicationSource related > classes will also be executed even if you do not have serial replication > tables. > So, we think that it maybe a better choice to move the option to > ReplicationPeerConfig. Since if there is one table in the peer requires > serial replication, then all the tables in this peer will be automatically > 'serial' since the wal for these tables are mixed with the one which requires > serial replication. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20169) NPE when calling HBTU.shutdownMiniCluster
Duo Zhang created HBASE-20169:
-
Summary: NPE when calling HBTU.shutdownMiniCluster
Key: HBASE-20169
URL: https://issues.apache.org/jira/browse/HBASE-20169
Project: HBase
Issue Type: Bug
Components: test
Reporter: Duo Zhang
This usually happens when some master or rs has already been down before we
calling shutdownMiniCluster.
See
https://builds.apache.org/job/HBASE-Flaky-Tests/27223/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManagerMetrics/org_apache_hadoop_hbase_master_TestAssignmentManagerMetrics/
and also
http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34873/testReport/junit/org.apache.hadoop.hbase.master/TestRestartCluster/testRetainAssignmentOnRestart/
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:156)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20169) NPE when calling HBTU.shutdownMiniCluster
[
https://issues.apache.org/jira/browse/HBASE-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393907#comment-16393907
]
Duo Zhang commented on HBASE-20169:
---
[~stack] FYI.
> NPE when calling HBTU.shutdownMiniCluster
> -
>
> Key: HBASE-20169
> URL: https://issues.apache.org/jira/browse/HBASE-20169
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Priority: Major
>
> This usually happens when some master or rs has already been down before we
> calling shutdownMiniCluster.
> See
> https://builds.apache.org/job/HBASE-Flaky-Tests/27223/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManagerMetrics/org_apache_hadoop_hbase_master_TestAssignmentManagerMetrics/
> and also
> http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34873/testReport/junit/org.apache.hadoop.hbase.master/TestRestartCluster/testRetainAssignmentOnRestart/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:156)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19598: -- Resolution: Fixed Assignee: Duo Zhang (was: stack) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Almost disappear on the flakey list. Will open a new issue to address remaining problems(not only for TestAssignmentManager). > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-19598-addendum.patch, > HBASE-19598.master.001.patch, HBASE-19598.master.002.patch, > HBASE-19598.master.003.patch, HBASE-19598.master.003.patch, > HBASE-19598.master.004.patch, HBASE-19598.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[
https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393900#comment-16393900
]
Duo Zhang commented on HBASE-19598:
---
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics.after(TestAssignmentManagerMetrics.java:100)
{noformat}
The stacktrace is a bit strange, I see this in log
{noformat}
2018-03-09 19:43:30,445 INFO [Time-limited test]
hbase.HBaseTestingUtility(1103): Shutting down minicluster
{noformat}
This means we have already entered the shutdownMiniCluster method in HBTU, so
the NPE should be in the shutdownMiniCluster method. I've also saw this for
TestRestartCluster once on the flakey test finder. Let me close this issue and
open a new one.
> Fix TestAssignmentManagerMetrics flaky test
> ---
>
> Key: HBASE-19598
> URL: https://issues.apache.org/jira/browse/HBASE-19598
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0-beta-1
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-19598-addendum.patch,
> HBASE-19598.master.001.patch, HBASE-19598.master.002.patch,
> HBASE-19598.master.003.patch, HBASE-19598.master.003.patch,
> HBASE-19598.master.004.patch, HBASE-19598.patch, TestUtil.java
>
>
> TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that
> commit 010012cbcb broke it (HBASE-18946).
> The test method runs successfully, but it cannot shut the minicluster down,
> and hangs forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20131) NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
[
https://issues.apache.org/jira/browse/HBASE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393892#comment-16393892
]
Josh Elser commented on HBASE-20131:
Yep, figured something along those lines would be needed. Have at it. Thanks!
> NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
> ---
>
> Key: HBASE-20131
> URL: https://issues.apache.org/jira/browse/HBASE-20131
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20131.001.patch
>
>
> I believe the error is that a MoveRegionProcedure comes in via ChaosMonkey
> for an unassigned region that was from a disabled table (also due to CM)
> which causes an NPE as we try to set a null original location into the
> protobuf which fails.
> {noformat}
> 2018-03-02 23:07:00,146 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=23,queue=2,port=2] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos$MoveRegionStateData$Builder.setSourceServer(MasterProcedureProtos.java:26127)
> at
> org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.serializeStateData(MoveRegionProcedure.java:133)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProtoProcedure(ProcedureUtil.java:198)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeEntry(ProcedureWALFormat.java:211)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeInsert(ProcedureWALFormat.java:222)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:490)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:863)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:832)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:111)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:561)
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1707)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1324)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){noformat}
> IntegrationTestLoadAndVerify also failed, but I'm not sure if it's related to
> this, or just a problem with the test. The test failed because the table was
> left offline after it was disabled, and appears to not have been re-enabled.
> Still debugging that side..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve FT in merge operation
[
https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393887#comment-16393887
]
Vladimir Rodionov commented on HBASE-19969:
---
{quote}
As I said previously, if the delete() call fails, the rename() would fail as
well.
I think IOE can be thrown for both failures.
{quote}
Done.
{quote}
getBackupTmpDirForBackupId is no longer in the patch. Can you describe related
changes ?
{quote}
It is now getBackupTmpDirPathForBackupId
> Improve FT in merge operation
> -
>
> Key: HBASE-19969
> URL: https://issues.apache.org/jira/browse/HBASE-19969
> Project: HBase
> Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch,
> HBASE-19969-v3.patch
>
>
> Some file system operations are not fault tolerant during merge. We delete
> backup data in a backup file system, then copy new data over to backup
> destination. Deletes can be partial, copy can fail as well
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393889#comment-16393889 ] Vladimir Rodionov commented on HBASE-19969: --- Patch v3. > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch, > HBASE-19969-v3.patch > > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-19969: -- Attachment: HBASE-19969-v3.patch > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch, > HBASE-19969-v3.patch > > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20131) NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
[
https://issues.apache.org/jira/browse/HBASE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393880#comment-16393880
]
stack commented on HBASE-20131:
---
Its hard to compare branch-1 and branch-2. Also, general idea was that
Procedure would be contained in that all logic would be inside the Procedure
itself. The MoveProcedure has no prepare step as most others do where they
check that they are ok to run (see for example the AssignProcedure#start...).
This seems like oversight.
We should add a prepare to the MoveProcedure where it checks if it can run. If
table is offline or master is stopped or cluster is going down, we should not
run. Probably good if this method was exposed so could be called just inside
the RsRPCServer#moveRegion method It could call it to do basic checks early
so we can fail early and send failure directly back to user.
I was going to do a review of all our procedures to make sure a prep. Mind if I
add to your patch [~elserj]? Thanks sir.
> NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
> ---
>
> Key: HBASE-20131
> URL: https://issues.apache.org/jira/browse/HBASE-20131
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20131.001.patch
>
>
> I believe the error is that a MoveRegionProcedure comes in via ChaosMonkey
> for an unassigned region that was from a disabled table (also due to CM)
> which causes an NPE as we try to set a null original location into the
> protobuf which fails.
> {noformat}
> 2018-03-02 23:07:00,146 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=23,queue=2,port=2] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos$MoveRegionStateData$Builder.setSourceServer(MasterProcedureProtos.java:26127)
> at
> org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.serializeStateData(MoveRegionProcedure.java:133)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProtoProcedure(ProcedureUtil.java:198)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeEntry(ProcedureWALFormat.java:211)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeInsert(ProcedureWALFormat.java:222)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:490)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:863)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:832)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:111)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:561)
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1707)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1324)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){noformat}
> IntegrationTestLoadAndVerify also failed, but I'm not sure if it's related to
> this, or just a problem with the test. The test failed because the table was
> left offline after it was disabled, and appears to not have been re-enabled.
> Still debugging that side..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve FT in merge operation
[
https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393859#comment-16393859
]
Ted Yu commented on HBASE-19969:
{code}
+if (!fs.delete(destPath, true)) {
+ System.out.println("Failed to delete " + destPath);
+}
+boolean res = fs.rename(tmpPath, destPath);
{code}
As I said previously, if the delete() call fails, the rename() would fail as
well.
I think IOE can be thrown for both failures.
getBackupTmpDirForBackupId is no longer in the patch. Can you describe related
changes ?
> Improve FT in merge operation
> -
>
> Key: HBASE-19969
> URL: https://issues.apache.org/jira/browse/HBASE-19969
> Project: HBase
> Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch
>
>
> Some file system operations are not fault tolerant during merge. We delete
> backup data in a backup file system, then copy new data over to backup
> destination. Deletes can be partial, copy can fail as well
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20133) Calculate correct assignment and build region movement plans for mis-placed regions in one pass
[ https://issues.apache.org/jira/browse/HBASE-20133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20133: --- Hadoop Flags: Reviewed Fix Version/s: 3.0.0 > Calculate correct assignment and build region movement plans for mis-placed > regions in one pass > --- > > Key: HBASE-20133 > URL: https://issues.apache.org/jira/browse/HBASE-20133 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-20133.master.000.patch, > HBASE-20133.master.001.patch, HBASE-20133.master.002.patch > > > In RSGroupBasedLoadBalancer#balanceCluster(clusterState), the logic could be > improved: > correctAssignment() builds a map for mis-placed and placed regions. For > mis-placed regions, the key(ServerName) is BOGUS_SERVER_NAME. Then the logic > gets those mis-paced regions out and calls findServerForRegion() several > times to find out the current host server, in order to build RegionPlan for > movement. > Some logic in correctAssignment() and findServerForRegion() could be merged > so as to build both corrected assignment and RegionPlan for mis-placed region > in one pass. As a result, findServerForRegion() could be removed if I get it > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20133) Calculate correct assignment and build region movement plans for mis-placed regions in one pass
[ https://issues.apache.org/jira/browse/HBASE-20133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393850#comment-16393850 ] Ted Yu commented on HBASE-20133: Adding debug log next to the line above, I found coverage by TestRSGroups. So test coverage issue passes. Going to commit soon. > Calculate correct assignment and build region movement plans for mis-placed > regions in one pass > --- > > Key: HBASE-20133 > URL: https://issues.apache.org/jira/browse/HBASE-20133 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Attachments: HBASE-20133.master.000.patch, > HBASE-20133.master.001.patch, HBASE-20133.master.002.patch > > > In RSGroupBasedLoadBalancer#balanceCluster(clusterState), the logic could be > improved: > correctAssignment() builds a map for mis-placed and placed regions. For > mis-placed regions, the key(ServerName) is BOGUS_SERVER_NAME. Then the logic > gets those mis-paced regions out and calls findServerForRegion() several > times to find out the current host server, in order to build RegionPlan for > movement. > Some logic in correctAssignment() and findServerForRegion() could be merged > so as to build both corrected assignment and RegionPlan for mis-placed region > in one pass. As a result, findServerForRegion() could be removed if I get it > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393845#comment-16393845 ] Vladimir Rodionov commented on HBASE-19969: --- Patch v2 addresses your comments, [~yuzhih...@gmail.com], [~elserj] > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch > > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-19969: -- Attachment: HBASE-19969-v2.patch > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-19969-v1.patch, HBASE-19969-v2.patch > > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[ https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393836#comment-16393836 ] Hudson commented on HBASE-20164: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1085 (See [https://builds.apache.org/job/HBase-1.2-IT/1085/]) HBASE-20164 add hadoopcheck link to table footer (busbey: rev fd43ff465a32dbdbd08ba7d21d9e26ed4fa0c2ea) * (edit) dev-support/hbase-personality.sh > failed hadoopcheck should add footer link > - > > Key: HBASE-20164 > URL: https://issues.apache.org/jira/browse/HBASE-20164 > Project: HBase > Issue Type: Bug > Components: community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3 > > Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch, > HBASE-20164.wip.patch > > > thought for sure this already had an issue, [~busbey], but I can't find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20152) [AMv2] DisableTableProcedure versus ServerCrashProcedure
[
https://issues.apache.org/jira/browse/HBASE-20152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393824#comment-16393824
]
stack commented on HBASE-20152:
---
For the deadlock scenario, we might be able to get away w/ simple fixes
* We were re-assigning a region though there was an unassign outstanding
against the crashed server. This was a bug that had us doing an unnecessary
assign and upping likelihood of deadlock (unassign and concurrent assign).
* Deadlock happens because the unassign happens AFTER SCP cancels outstanding
RITs but before our SCP Assign is scheduled. Rerunning the RIT cancel procedure
after all assigns have been created and scheduled hopefully takes care of the
case where an outstanding unassign has the region lock so the SCP assign can't
progress; the re-run of the cancel of outstanding RITs against crashed server
will cancel any unassigns that might have been lucky enough to have been
scheduled.
The above should fix scenario #1.
Scenario #2 'Delete of Online Region' is tough. It happened while we had the
patch over in HBASE-20137 was in place which allowed the UnassignProcedure
progress when it needed to wait on SCP. I won't spend more time on it.
For Scenario #3, 'Move after Cluster Shutdown', just need to retrofit
Procedures so likes of Move, etc., don't run when cluster shutdown and Master
stop has been set (probably don't want to turn off AMv2 Procedures completely
since we'll likely want to put shutdown itself into a Procedure...)
> [AMv2] DisableTableProcedure versus ServerCrashProcedure
>
>
> Key: HBASE-20152
> URL: https://issues.apache.org/jira/browse/HBASE-20152
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
>
> Seeing a small spate of issues where disabled tables/regions are being
> assigned. Usually they happen when a DisableTableProcedure is running
> concurrent with a ServerCrashProcedure. See below. See associated
> HBASE-20131. This is umbrella issue for fixing.
> h3. Deadlock
> From HBASE-20137, 'TestRSGroups is Flakey',
> https://issues.apache.org/jira/browse/HBASE-20137?focusedCommentId=16390325=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16390325
> {code}
> * SCP is running because a server was aborted in test.
> * SCP starts AssignProcedure of region X from crashed server.
> * DisableTable Procedure runs because test has finished and we're doing
> table delete. Queues
> * UnassignProcedure for region X.
> * Disable Unassign gets Lock on region X first.
> * SCP AssignProcedure tries to get lock, waits on lock.
> * DisableTable Procedure UnassignProcedure RPC fails because server is down
> (Thats why the SCP).
> * Tries to expire the server it failed the RPC against. Fails (currently
> being SCP'd).
> * DisableTable Procedure Unassign is suspended. It is a suspend with lock on
> region X held
> * SCP can't run because lock on X is held
> * Test timesout.
> {code}
> Here is the actual log from around the deadlock. pid=308 is the SCP. pid=309
> is the disable table:
> {code}
> 2018-03-05 11:29:21,224 DEBUG [PEWorker-7]
> procedure.ServerCrashProcedure(225): Done splitting WALs pid=308,
> state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS; ServerCrashProcedure
> server=1cfd208ff882,40584,1520249102524, splitWal=true, meta=false
> 2018-03-05 11:29:21,300 INFO
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
> rsgroup.RSGroupAdminServer(371): Move server done: default=>appInfo
> 2018-03-05 11:29:21,307 INFO
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
> rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(279):
> Client=jenkins//172.17.0.2 list rsgroup
> 2018-03-05 11:29:21,312 INFO [Time-limited test] client.HBaseAdmin$15(901):
> Started disable of Group_ns:testKillRS
> 2018-03-05 11:29:21,313 INFO
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
> master.HMaster$7(2278): Client=jenkins//172.17.0.2 disable Group_ns:testKillRS
> 2018-03-05 11:29:21,384 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1495): Initialized subprocedures=[{pid=310,
> ppid=308, state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
> table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043}]
> 2018-03-05 11:29:21,534 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
> procedure2.ProcedureExecutor(865): Stored pid=309,
> state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure
> table=Group_ns:testKillRS
> 2018-03-05 11:29:21,542 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
> master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
> 2018-03-05 11:29:21,644 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[ https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393820#comment-16393820 ] Hudson commented on HBASE-20162: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1084 (See [https://builds.apache.org/job/HBase-1.2-IT/1084/]) HBASE-20162 [nightly] depending on pipeline execution we sometimes refer (busbey: rev d317aa4a10df19a106f837d98a4bfcf7938ba255) * (edit) dev-support/hbase_nightly_yetus.sh * (edit) dev-support/Jenkinsfile > [nightly] depending on pipeline execution we sometimes refer to the wrong > workspace > --- > > Key: HBASE-20162 > URL: https://issues.apache.org/jira/browse/HBASE-20162 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3 > > Attachments: HBASE-20162.0.patch > > > we set BASEDIR at the top of our pipeline to point at the component checkout > within WORKSPACE. > but! > a) at that point WORKSPACE is the workspace for the launching task > b) sometimes our parallel executions get a task with a different local > WORKSPACE to allow for coexisting on the same build host > c) when this happens our parallel stages are referring to some other absolute > path on the host > d) in most cases we're referring to dev-support files like e.g. the nightly > build script or machine info script that are the same across branches so > things are fine if we aren't running at the same time as a job that's > overwritting them > e) we also refer to the Dockerfile this way, so weird bugs I'm sure. > f) we build the source tarball from here, so that's probably broken subtly > g) sometimes that other directory _doesn't exist at all_ and we fail with > confusing messages about stuff not found -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20124) Make hbase-spark module work with hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393807#comment-16393807
]
Ted Yu commented on HBASE-20124:
hadoopcheck failure was not related:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install)
on project hbase-thrift: Failed to install metadata
org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not
parse metadata
/home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml:
in epilog non whitespace content is not allowed but got / (position: END_TAG
seen ...\n/... @25:2) -> [Help 1]
{code}
> Make hbase-spark module work with hadoop3
> -
>
> Key: HBASE-20124
> URL: https://issues.apache.org/jira/browse/HBASE-20124
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20124.v1.txt, 20124.v2.txt, 20124.v3.txt
>
>
> The following error can be observed when running tests in hbase-spark module
> against hadoop3:
> {code}
> HBaseDStreamFunctionsSuite:
> *** RUN ABORTED ***
> java.lang.NoClassDefFoundError: org/apache/hadoop/ipc/ExternalCall
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:746)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:640)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:979)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> ...
> Cause: java.lang.ClassNotFoundException: org.apache.hadoop.ipc.ExternalCall
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:746)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> {code}
> The dependency tree shows mixture of hadoop 2.7.4 and hadoop3 for the
> hbase-spark module.
> This should be addressed by adding proper profile in pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[ https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393805#comment-16393805 ] Hudson commented on HBASE-20162: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #372 (See [https://builds.apache.org/job/HBase-1.3-IT/372/]) HBASE-20162 [nightly] depending on pipeline execution we sometimes refer (busbey: rev 0d81bd171b377dc90051124a49d5f4d837e2139e) * (edit) dev-support/Jenkinsfile * (edit) dev-support/hbase_nightly_yetus.sh > [nightly] depending on pipeline execution we sometimes refer to the wrong > workspace > --- > > Key: HBASE-20162 > URL: https://issues.apache.org/jira/browse/HBASE-20162 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3 > > Attachments: HBASE-20162.0.patch > > > we set BASEDIR at the top of our pipeline to point at the component checkout > within WORKSPACE. > but! > a) at that point WORKSPACE is the workspace for the launching task > b) sometimes our parallel executions get a task with a different local > WORKSPACE to allow for coexisting on the same build host > c) when this happens our parallel stages are referring to some other absolute > path on the host > d) in most cases we're referring to dev-support files like e.g. the nightly > build script or machine info script that are the same across branches so > things are fine if we aren't running at the same time as a job that's > overwritting them > e) we also refer to the Dockerfile this way, so weird bugs I'm sure. > f) we build the source tarball from here, so that's probably broken subtly > g) sometimes that other directory _doesn't exist at all_ and we fail with > confusing messages about stuff not found -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20164) failed hadoopcheck should add footer link
[ https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20164: Fix Version/s: 1.2.8 > failed hadoopcheck should add footer link > - > > Key: HBASE-20164 > URL: https://issues.apache.org/jira/browse/HBASE-20164 > Project: HBase > Issue Type: Bug > Components: community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3 > > Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch, > HBASE-20164.wip.patch > > > thought for sure this already had an issue, [~busbey], but I can't find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[ https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20162: Resolution: Fixed Fix Version/s: 1.4.3 1.2.8 1.3.3 1.5.0 2.1.0 3.0.0 2.0.0 Status: Resolved (was: Patch Available) > [nightly] depending on pipeline execution we sometimes refer to the wrong > workspace > --- > > Key: HBASE-20162 > URL: https://issues.apache.org/jira/browse/HBASE-20162 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3 > > Attachments: HBASE-20162.0.patch > > > we set BASEDIR at the top of our pipeline to point at the component checkout > within WORKSPACE. > but! > a) at that point WORKSPACE is the workspace for the launching task > b) sometimes our parallel executions get a task with a different local > WORKSPACE to allow for coexisting on the same build host > c) when this happens our parallel stages are referring to some other absolute > path on the host > d) in most cases we're referring to dev-support files like e.g. the nightly > build script or machine info script that are the same across branches so > things are fine if we aren't running at the same time as a job that's > overwritting them > e) we also refer to the Dockerfile this way, so weird bugs I'm sure. > f) we build the source tarball from here, so that's probably broken subtly > g) sometimes that other directory _doesn't exist at all_ and we fail with > confusing messages about stuff not found -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20162) [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
[ https://issues.apache.org/jira/browse/HBASE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393754#comment-16393754 ] Sean Busbey commented on HBASE-20162: - Mike +1d over on the PR > [nightly] depending on pipeline execution we sometimes refer to the wrong > workspace > --- > > Key: HBASE-20162 > URL: https://issues.apache.org/jira/browse/HBASE-20162 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0, 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-20162.0.patch > > > we set BASEDIR at the top of our pipeline to point at the component checkout > within WORKSPACE. > but! > a) at that point WORKSPACE is the workspace for the launching task > b) sometimes our parallel executions get a task with a different local > WORKSPACE to allow for coexisting on the same build host > c) when this happens our parallel stages are referring to some other absolute > path on the host > d) in most cases we're referring to dev-support files like e.g. the nightly > build script or machine info script that are the same across branches so > things are fine if we aren't running at the same time as a job that's > overwritting them > e) we also refer to the Dockerfile this way, so weird bugs I'm sure. > f) we build the source tarball from here, so that's probably broken subtly > g) sometimes that other directory _doesn't exist at all_ and we fail with > confusing messages about stuff not found -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20146) Regions are stuck while opening when WAL is disabled
[
https://issues.apache.org/jira/browse/HBASE-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393749#comment-16393749
]
Hudson commented on HBASE-20146:
Results for branch branch-1.3
[build #258 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/258/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/258//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/258//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/258//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Regions are stuck while opening when WAL is disabled
>
>
> Key: HBASE-20146
> URL: https://issues.apache.org/jira/browse/HBASE-20146
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>
> Attachments: HBASE-20146.patch, HBASE-20146.v1.patch
>
>
> On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false,
> to disable the WAL for complete cluster, after restarting HBase service,
> regions are not getting opened leading to HMaster abort as Namespace table
> regions are not getting assigned.
> jstack for region open:
> {noformat}
> "RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0
> tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x87554448> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> This used to work with HBase 1.0.2 version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20124) Make hbase-spark module work with hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393739#comment-16393739
]
Hadoop QA commented on HBASE-20124:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 8m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
5s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
1s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
6s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
10s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20124 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913839/20124.v3.txt |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux 353685cb2678 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 28279697c2 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| hadoopcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/11893/artifact/patchprocess/patch-javac-2.6.5.txt
|
| hadoopcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/11893/artifact/patchprocess/patch-javac-2.7.4.txt
|
| hadoopcheck |
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[ https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393727#comment-16393727 ] Sean Busbey commented on HBASE-20164: - branch-1.2? > failed hadoopcheck should add footer link > - > > Key: HBASE-20164 > URL: https://issues.apache.org/jira/browse/HBASE-20164 > Project: HBase > Issue Type: Bug > Components: community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3 > > Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch, > HBASE-20164.wip.patch > > > thought for sure this already had an issue, [~busbey], but I can't find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20124) Make hbase-spark module work with hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20124:
---
Attachment: 20124.v3.txt
> Make hbase-spark module work with hadoop3
> -
>
> Key: HBASE-20124
> URL: https://issues.apache.org/jira/browse/HBASE-20124
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20124.v1.txt, 20124.v2.txt, 20124.v3.txt
>
>
> The following error can be observed when running tests in hbase-spark module
> against hadoop3:
> {code}
> HBaseDStreamFunctionsSuite:
> *** RUN ABORTED ***
> java.lang.NoClassDefFoundError: org/apache/hadoop/ipc/ExternalCall
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:746)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:640)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:979)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> ...
> Cause: java.lang.ClassNotFoundException: org.apache.hadoop.ipc.ExternalCall
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:746)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> {code}
> The dependency tree shows mixture of hadoop 2.7.4 and hadoop3 for the
> hbase-spark module.
> This should be addressed by adding proper profile in pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20152) [AMv2] DisableTableProcedure versus ServerCrashProcedure
[
https://issues.apache.org/jira/browse/HBASE-20152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20152:
--
Description:
Seeing a small spate of issues where disabled tables/regions are being
assigned. Usually they happen when a DisableTableProcedure is running
concurrent with a ServerCrashProcedure. See below. See associated HBASE-20131.
This is umbrella issue for fixing.
h3. Deadlock
>From HBASE-20137, 'TestRSGroups is Flakey',
>https://issues.apache.org/jira/browse/HBASE-20137?focusedCommentId=16390325=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16390325
{code}
* SCP is running because a server was aborted in test.
* SCP starts AssignProcedure of region X from crashed server.
* DisableTable Procedure runs because test has finished and we're doing table
delete. Queues
* UnassignProcedure for region X.
* Disable Unassign gets Lock on region X first.
* SCP AssignProcedure tries to get lock, waits on lock.
* DisableTable Procedure UnassignProcedure RPC fails because server is down
(Thats why the SCP).
* Tries to expire the server it failed the RPC against. Fails (currently being
SCP'd).
* DisableTable Procedure Unassign is suspended. It is a suspend with lock on
region X held
* SCP can't run because lock on X is held
* Test timesout.
{code}
Here is the actual log from around the deadlock. pid=308 is the SCP. pid=309 is
the disable table:
{code}
2018-03-05 11:29:21,224 DEBUG [PEWorker-7] procedure.ServerCrashProcedure(225):
Done splitting WALs pid=308, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS;
ServerCrashProcedure server=1cfd208ff882,40584,1520249102524, splitWal=true,
meta=false
2018-03-05 11:29:21,300 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
rsgroup.RSGroupAdminServer(371): Move server done: default=>appInfo
2018-03-05 11:29:21,307 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(279):
Client=jenkins//172.17.0.2 list rsgroup
2018-03-05 11:29:21,312 INFO [Time-limited test] client.HBaseAdmin$15(901):
Started disable of Group_ns:testKillRS
2018-03-05 11:29:21,313 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.HMaster$7(2278): Client=jenkins//172.17.0.2 disable Group_ns:testKillRS
2018-03-05 11:29:21,384 INFO [PEWorker-9] procedure2.ProcedureExecutor(1495):
Initialized subprocedures=[{pid=310, ppid=308,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043}]
2018-03-05 11:29:21,534 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
procedure2.ProcedureExecutor(865): Stored pid=309,
state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure
table=Group_ns:testKillRS
2018-03-05 11:29:21,542 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:21,644 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:21,847 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:22,118 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1944): Put
{"totalColumns":1,"row":"Group_ns:testKillRS","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":1520249362117}]},"ts":1520249362117}
2018-03-05 11:29:22,123 INFO [PEWorker-5] hbase.MetaTableAccessor(1646):
Updated table Group_ns:testKillRS state to DISABLING in META
2018-03-05 11:29:22,148 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:22,345 INFO [PEWorker-5] procedure2.ProcedureExecutor(1495):
Initialized subprocedures=[{pid=311, ppid=309,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043,
server=1cfd208ff882,40584,1520249102524}]
2018-03-05 11:29:22,503 INFO [PEWorker-13]
procedure.MasterProcedureScheduler(571): pid=311, ppid=309,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043,
server=1cfd208ff882,40584,1520249102524,
Group_ns:testKillRS,,1520249357742.de7534c208a06502537cd95c248b3043.
2018-03-05 11:29:22,503 INFO [PEWorker-5]
procedure.MasterProcedureScheduler(571): pid=310, ppid=308,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043,
Group_ns:testKillRS,,1520249357742.de7534c208a06502537cd95c248b3043.
2018-03-05 11:29:22,504 INFO [PEWorker-13] assignment.RegionStateStore(189):
pid=311 updating hbase:meta
[jira] [Comment Edited] (HBASE-20152) [AMv2] DisableTableProcedure versus ServerCrashProcedure
[
https://issues.apache.org/jira/browse/HBASE-20152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390901#comment-16390901
]
stack edited comment on HBASE-20152 at 3/9/18 10:26 PM:
New scenario seen on internal cluster (not sure what the test is doing... kinda
crazy killing of cluster and then restarts..)
h3. Procedure after Cluster Shutdown
{code}
* Cluster shutdown is set. This means cluster down flag is set, we expire
servers, but no ServerCrashProcedure gets scheduled.
2018-03-03 05:51:33,852 INFO org.apache.hadoop.hbase.master.ServerManager:
Cluster shutdown set; quasar-lxosrm-3.vpc.cloudera.com,22101,1520080141368
expired; onlineServers=2
2018-03-03 05:51:33,859 INFO
org.apache.hadoop.hbase.master.RegionServerTracker: RegionServer ephemeral node
deleted, processing expiration
[quasar-lxosrm-2.vpc.cloudera.com,22101,1520080154394]
* Just after, a Move Region event comes in
2018-03-03 05:51:33,999 INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: pid=158,
state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
hri=hbase:namespace,,1520062075237.d2854c6a96bb191d04a3f285c0eef210.,
source=quasar-lxosrm-2.vpc.cloudera.com,22101,1520080154394, destination=
hbase:namespace hbase:namespace,,1520062075237.d2854c6a96bb191d04a3f285c0eef210.
* Move region tries to unassign but RPC fails with java.io.IOException: Call to
quasar-lxosrm-2.vpc.cloudera.com/172.26.11.155:22101 failed on local exception:
org.apache.hadoop.hbase.ipc.StoppedRpcClientException
* Can't schedule an SCP. Fails because server is going down already not
present as online. Procedure suspended.
* New Master, Unassign procedure rerun. Does dispatch to non-existent server.
* Fails to schedule SCP because server is long gone.
* Procedure is Suspended. Stuck forever.
{code}
was (Author: stack):
New scenario seen on internal cluster (not sure what the test is doing... kinda
crazy killing of cluster and then restarts..)
h3. Procedure after Cluster Shutdown
{code}
* Cluster shutdown is set. This means cluster down flag is set, we expire
servers, but no ServerCrashProcedure gets scheduled.
2018-03-03 05:51:33,852 INFO org.apache.hadoop.hbase.master.ServerManager:
Cluster shutdown set; quasar-lxosrm-3.vpc.cloudera.com,22101,1520080141368
expired; onlineServers=2
2018-03-03 05:51:33,859 INFO
org.apache.hadoop.hbase.master.RegionServerTracker: RegionServer ephemeral node
deleted, processing expiration
[quasar-lxosrm-2.vpc.cloudera.com,22101,1520080154394]
* Just after, a Move Region event comes in
2018-03-03 05:51:33,999 INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: pid=158,
state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
hri=hbase:namespace,,1520062075237.d2854c6a96bb191d04a3f285c0eef210.,
source=quasar-lxosrm-2.vpc.cloudera.com,22101,1520080154394, destination=
hbase:namespace hbase:namespace,,1520062075237.d2854c6a96bb191d04a3f285c0eef210.
* Move region tries to unassign but RPC fails with java.io.IOException: Call to
quasar-lxosrm-2.vpc.cloudera.com/172.26.11.155:22101 failed on local exception:
org.apache.hadoop.hbase.ipc.StoppedRpcClientException
* Can't schedule an SCP. Fails because server is going down already not
present as online. Procedure suspended.
* New Master, Unassign procedure rerun. Does dispatch to non-existent server.
* Fails to schedule SCP because server is long gone.
* Procedure is Suspended. Stuck forever.
{code}
Here is the actual log from around the deadlock. pid=308 is the SCP. pid=309 is
the disable table:
{code}
2018-03-05 11:29:21,224 DEBUG [PEWorker-7] procedure.ServerCrashProcedure(225):
Done splitting WALs pid=308, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS;
ServerCrashProcedure server=1cfd208ff882,40584,1520249102524, splitWal=true,
meta=false
2018-03-05 11:29:21,300 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
rsgroup.RSGroupAdminServer(371): Move server done: default=>appInfo
2018-03-05 11:29:21,307 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(279):
Client=jenkins//172.17.0.2 list rsgroup
2018-03-05 11:29:21,312 INFO [Time-limited test] client.HBaseAdmin$15(901):
Started disable of Group_ns:testKillRS
2018-03-05 11:29:21,313 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.HMaster$7(2278): Client=jenkins//172.17.0.2 disable Group_ns:testKillRS
2018-03-05 11:29:21,384 INFO [PEWorker-9] procedure2.ProcedureExecutor(1495):
Initialized subprocedures=[{pid=310, ppid=308,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043}]
2018-03-05 11:29:21,534 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
procedure2.ProcedureExecutor(865): Stored pid=309,
state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure
[jira] [Comment Edited] (HBASE-20152) [AMv2] DisableTableProcedure versus ServerCrashProcedure
[
https://issues.apache.org/jira/browse/HBASE-20152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390901#comment-16390901
]
stack edited comment on HBASE-20152 at 3/9/18 10:18 PM:
New scenario seen on internal cluster (not sure what the test is doing... kinda
crazy killing of cluster and then restarts..)
h3. Procedure after Cluster Shutdown
{code}
* Cluster shutdown is set. This means cluster down flag is set, we expire
servers, but no ServerCrashProcedure gets scheduled.
2018-03-03 05:51:33,852 INFO org.apache.hadoop.hbase.master.ServerManager:
Cluster shutdown set; quasar-lxosrm-3.vpc.cloudera.com,22101,1520080141368
expired; onlineServers=2
2018-03-03 05:51:33,859 INFO
org.apache.hadoop.hbase.master.RegionServerTracker: RegionServer ephemeral node
deleted, processing expiration
[quasar-lxosrm-2.vpc.cloudera.com,22101,1520080154394]
* Just after, a Move Region event comes in
2018-03-03 05:51:33,999 INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: pid=158,
state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
hri=hbase:namespace,,1520062075237.d2854c6a96bb191d04a3f285c0eef210.,
source=quasar-lxosrm-2.vpc.cloudera.com,22101,1520080154394, destination=
hbase:namespace hbase:namespace,,1520062075237.d2854c6a96bb191d04a3f285c0eef210.
* Move region tries to unassign but RPC fails with java.io.IOException: Call to
quasar-lxosrm-2.vpc.cloudera.com/172.26.11.155:22101 failed on local exception:
org.apache.hadoop.hbase.ipc.StoppedRpcClientException
* Can't schedule an SCP. Fails because server is going down already not
present as online. Procedure suspended.
* New Master, Unassign procedure rerun. Does dispatch to non-existent server.
* Fails to schedule SCP because server is long gone.
* Procedure is Suspended. Stuck forever.
{code}
Here is the actual log from around the deadlock. pid=308 is the SCP. pid=309 is
the disable table:
{code}
2018-03-05 11:29:21,224 DEBUG [PEWorker-7] procedure.ServerCrashProcedure(225):
Done splitting WALs pid=308, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS;
ServerCrashProcedure server=1cfd208ff882,40584,1520249102524, splitWal=true,
meta=false
2018-03-05 11:29:21,300 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
rsgroup.RSGroupAdminServer(371): Move server done: default=>appInfo
2018-03-05 11:29:21,307 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(279):
Client=jenkins//172.17.0.2 list rsgroup
2018-03-05 11:29:21,312 INFO [Time-limited test] client.HBaseAdmin$15(901):
Started disable of Group_ns:testKillRS
2018-03-05 11:29:21,313 INFO
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.HMaster$7(2278): Client=jenkins//172.17.0.2 disable Group_ns:testKillRS
2018-03-05 11:29:21,384 INFO [PEWorker-9] procedure2.ProcedureExecutor(1495):
Initialized subprocedures=[{pid=310, ppid=308,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043}]
2018-03-05 11:29:21,534 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
procedure2.ProcedureExecutor(865): Stored pid=309,
state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure
table=Group_ns:testKillRS
2018-03-05 11:29:21,542 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:21,644 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:21,847 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:22,118 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1944): Put
{"totalColumns":1,"row":"Group_ns:testKillRS","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":1520249362117}]},"ts":1520249362117}
2018-03-05 11:29:22,123 INFO [PEWorker-5] hbase.MetaTableAccessor(1646):
Updated table Group_ns:testKillRS state to DISABLING in META
2018-03-05 11:29:22,148 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38498]
master.MasterRpcServices(1134): Checking to see if procedure is done pid=309
2018-03-05 11:29:22,345 INFO [PEWorker-5] procedure2.ProcedureExecutor(1495):
Initialized subprocedures=[{pid=311, ppid=309,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043,
server=1cfd208ff882,40584,1520249102524}]
2018-03-05 11:29:22,503 INFO [PEWorker-13]
procedure.MasterProcedureScheduler(571): pid=311, ppid=309,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure
table=Group_ns:testKillRS, region=de7534c208a06502537cd95c248b3043,
server=1cfd208ff882,40584,1520249102524,
[jira] [Commented] (HBASE-20124) Make hbase-spark module work with hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393687#comment-16393687
]
Mike Drob commented on HBASE-20124:
---
you shouldn't need to specify exclusions, they should be inherited from the
parent pom (hbase root pom)
also, your indent is off in a few places.
> Make hbase-spark module work with hadoop3
> -
>
> Key: HBASE-20124
> URL: https://issues.apache.org/jira/browse/HBASE-20124
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20124.v1.txt, 20124.v2.txt
>
>
> The following error can be observed when running tests in hbase-spark module
> against hadoop3:
> {code}
> HBaseDStreamFunctionsSuite:
> *** RUN ABORTED ***
> java.lang.NoClassDefFoundError: org/apache/hadoop/ipc/ExternalCall
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:746)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:640)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:979)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> ...
> Cause: java.lang.ClassNotFoundException: org.apache.hadoop.ipc.ExternalCall
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:746)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> {code}
> The dependency tree shows mixture of hadoop 2.7.4 and hadoop3 for the
> hbase-spark module.
> This should be addressed by adding proper profile in pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20164) failed hadoopcheck should add footer link
[ https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393676#comment-16393676 ] Hudson commented on HBASE-20164: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #371 (See [https://builds.apache.org/job/HBase-1.3-IT/371/]) HBASE-20164 add hadoopcheck link to table footer (mdrob: rev e02b28f9341e5df9bd8ea9190e24b15a0cc0540d) * (edit) dev-support/hbase-personality.sh > failed hadoopcheck should add footer link > - > > Key: HBASE-20164 > URL: https://issues.apache.org/jira/browse/HBASE-20164 > Project: HBase > Issue Type: Bug > Components: community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3 > > Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch, > HBASE-20164.wip.patch > > > thought for sure this already had an issue, [~busbey], but I can't find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393671#comment-16393671
]
Hadoop QA commented on HBASE-20153:
---
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at
https://builds.apache.org/job/PreCommit-HBASE-Build/11892/console in case of
problems.
> enable error-prone analysis in precommit
>
>
> Key: HBASE-20153
> URL: https://issues.apache.org/jira/browse/HBASE-20153
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20153.patch
>
>
> We've done a lot of work to get rid of the error-prone errors, we should make
> sure they stay out. Let's enable errorProne profile and analysis in precommit.
> [~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the
> mvn compile precommit check but was unable to unravel that thread. Any help
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20146) Regions are stuck while opening when WAL is disabled
[
https://issues.apache.org/jira/browse/HBASE-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393668#comment-16393668
]
Hudson commented on HBASE-20146:
Results for branch branch-1.2
[build #261 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/261/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/261//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/261//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/261//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Regions are stuck while opening when WAL is disabled
>
>
> Key: HBASE-20146
> URL: https://issues.apache.org/jira/browse/HBASE-20146
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.3.1
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 1.5.0, 1.2.7, 1.3.3, 1.4.3
>
> Attachments: HBASE-20146.patch, HBASE-20146.v1.patch
>
>
> On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false,
> to disable the WAL for complete cluster, after restarting HBase service,
> regions are not getting opened leading to HMaster abort as Namespace table
> regions are not getting assigned.
> jstack for region open:
> {noformat}
> "RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0
> tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x87554448> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> This used to work with HBase 1.0.2 version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20131) NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
[
https://issues.apache.org/jira/browse/HBASE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393657#comment-16393657
]
Josh Elser commented on HBASE-20131:
Hurm, for some reason HBASE-16816 was just never applied to branch-2 (or, I
guess, master at the time). I think that would catch most cases of this.
> NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
> ---
>
> Key: HBASE-20131
> URL: https://issues.apache.org/jira/browse/HBASE-20131
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20131.001.patch
>
>
> I believe the error is that a MoveRegionProcedure comes in via ChaosMonkey
> for an unassigned region that was from a disabled table (also due to CM)
> which causes an NPE as we try to set a null original location into the
> protobuf which fails.
> {noformat}
> 2018-03-02 23:07:00,146 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=23,queue=2,port=2] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos$MoveRegionStateData$Builder.setSourceServer(MasterProcedureProtos.java:26127)
> at
> org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.serializeStateData(MoveRegionProcedure.java:133)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProtoProcedure(ProcedureUtil.java:198)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeEntry(ProcedureWALFormat.java:211)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeInsert(ProcedureWALFormat.java:222)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:490)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:863)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:832)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:111)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:561)
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1707)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1324)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){noformat}
> IntegrationTestLoadAndVerify also failed, but I'm not sure if it's related to
> this, or just a problem with the test. The test failed because the table was
> left offline after it was disabled, and appears to not have been re-enabled.
> Still debugging that side..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393656#comment-16393656
]
Mike Drob commented on HBASE-20153:
---
patch file includes two commits - first one to add the check for precommit,
second one to trigger a full build (won't be included in final push)
let's see what QA thinks
> enable error-prone analysis in precommit
>
>
> Key: HBASE-20153
> URL: https://issues.apache.org/jira/browse/HBASE-20153
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20153.patch
>
>
> We've done a lot of work to get rid of the error-prone errors, we should make
> sure they stay out. Let's enable errorProne profile and analysis in precommit.
> [~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the
> mvn compile precommit check but was unable to unravel that thread. Any help
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-20153:
--
Assignee: Mike Drob
Status: Patch Available (was: Open)
> enable error-prone analysis in precommit
>
>
> Key: HBASE-20153
> URL: https://issues.apache.org/jira/browse/HBASE-20153
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20153.patch
>
>
> We've done a lot of work to get rid of the error-prone errors, we should make
> sure they stay out. Let's enable errorProne profile and analysis in precommit.
> [~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the
> mvn compile precommit check but was unable to unravel that thread. Any help
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20153) enable error-prone analysis in precommit
[
https://issues.apache.org/jira/browse/HBASE-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-20153:
--
Attachment: HBASE-20153.patch
> enable error-prone analysis in precommit
>
>
> Key: HBASE-20153
> URL: https://issues.apache.org/jira/browse/HBASE-20153
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Priority: Major
> Attachments: HBASE-20153.patch
>
>
> We've done a lot of work to get rid of the error-prone errors, we should make
> sure they stay out. Let's enable errorProne profile and analysis in precommit.
> [~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the
> mvn compile precommit check but was unable to unravel that thread. Any help
> is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20124) Make hbase-spark module work with hadoop3
[
https://issues.apache.org/jira/browse/HBASE-20124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393646#comment-16393646
]
Hadoop QA commented on HBASE-20124:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 10m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
38s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
42s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m
15s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
37s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20124 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913827/20124.v2.txt |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux c9e79d0c3487 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 974200fca1 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11891/testReport/ |
| Max. process+thread count | 934 (vs. ulimit of 1) |
| modules | C: hbase-spark U: hbase-spark |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11891/console |
| Powered by | Apache Yetus 0.7.0
[jira] [Updated] (HBASE-20164) failed hadoopcheck should add footer link
[ https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-20164: -- Fix Version/s: 1.3.3 > failed hadoopcheck should add footer link > - > > Key: HBASE-20164 > URL: https://issues.apache.org/jira/browse/HBASE-20164 > Project: HBase > Issue Type: Bug > Components: community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3 > > Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch, > HBASE-20164.wip.patch > > > thought for sure this already had an issue, [~busbey], but I can't find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20164) failed hadoopcheck should add footer link
[
https://issues.apache.org/jira/browse/HBASE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-20164:
--
Resolution: Fixed
Fix Version/s: 1.4.3
1.5.0
2.0.0
Status: Resolved (was: Patch Available)
thanks for review!
pushed to master, branch-{2, 2.0, 1, 1.4, 1.3}
fyi [~stack] [~apurtell] [~toffer]
> failed hadoopcheck should add footer link
> -
>
> Key: HBASE-20164
> URL: https://issues.apache.org/jira/browse/HBASE-20164
> Project: HBase
> Issue Type: Bug
> Components: community
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.4.3
>
> Attachments: HBASE-20164.patch, HBASE-20164.wip.1.patch,
> HBASE-20164.wip.patch
>
>
> thought for sure this already had an issue, [~busbey], but I can't find it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20131) NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
[
https://issues.apache.org/jira/browse/HBASE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393633#comment-16393633
]
Josh Elser commented on HBASE-20131:
Looks like there was some code dropped in branch-2 (for some reason)
branch-1
{code:java}
if (regionState == null) {
throw new UnknownRegionException(Bytes.toStringBinary(encodedRegionName));
} else if (!assignmentManager.getRegionStates()
.isRegionOnline(regionState.getRegion())) {
throw new HBaseIOException(
"moving region not onlined: " + regionState.getRegion() + ", "
+ regionState);
}{code}
branch-2
{code:java}
if (regionState != null) {
hri = regionState.getRegion();
} else {
throw new UnknownRegionException(Bytes.toStringBinary(encodedRegionName));
}{code}
If you try to {{move}} and offline region in branch-1, you hit the else-if
branch. Let me see if I can figure out a reason for that change..
> NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
> ---
>
> Key: HBASE-20131
> URL: https://issues.apache.org/jira/browse/HBASE-20131
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20131.001.patch
>
>
> I believe the error is that a MoveRegionProcedure comes in via ChaosMonkey
> for an unassigned region that was from a disabled table (also due to CM)
> which causes an NPE as we try to set a null original location into the
> protobuf which fails.
> {noformat}
> 2018-03-02 23:07:00,146 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=23,queue=2,port=2] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos$MoveRegionStateData$Builder.setSourceServer(MasterProcedureProtos.java:26127)
> at
> org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.serializeStateData(MoveRegionProcedure.java:133)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProtoProcedure(ProcedureUtil.java:198)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeEntry(ProcedureWALFormat.java:211)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeInsert(ProcedureWALFormat.java:222)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:490)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:863)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:832)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:111)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:561)
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1707)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1324)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){noformat}
> IntegrationTestLoadAndVerify also failed, but I'm not sure if it's related to
> this, or just a problem with the test. The test failed because the table was
> left offline after it was disabled, and appears to not have been re-enabled.
> Still debugging that side..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16179: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Fix compilation errors when building hbase-spark against Spark 2.0 > -- > > Key: HBASE-16179 > URL: https://issues.apache.org/jira/browse/HBASE-16179 > Project: HBase > Issue Type: Bug > Components: spark >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Labels: build > Fix For: 3.0.0 > > Attachments: 16179.v0.txt, 16179.v1.txt, 16179.v1.txt, 16179.v10.txt, > 16179.v11.txt, 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, > 16179.v15.txt, 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, > 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, 16179.v25.txt, > 16179.v26.txt, 16179.v27.txt, 16179.v28.txt, 16179.v28.txt, 16179.v29.txt, > 16179.v30.txt, 16179.v31.txt, 16179.v32.txt, 16179.v33.txt, 16179.v34.txt, > 16179.v35.txt, 16179.v36.txt, 16179.v37.txt, 16179.v38.txt, 16179.v38.txt, > 16179.v4.txt, 16179.v5.txt, 16179.v7.txt, 16179.v8.txt, 16179.v9.txt, > HBASE-16179.v29.patch > > > I tried building hbase-spark module against Spark-2.0 snapshot and got the > following compilation errors: > http://pastebin.com/bg3w247a > Some Spark classes such as DataTypeParser and Logging are no longer > accessible to downstream projects. > hbase-spark module should not depend on such classes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19697) Remove TestReplicationAdminUsingProcedure
[
https://issues.apache.org/jira/browse/HBASE-19697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393571#comment-16393571
]
Hudson commented on HBASE-19697:
Results for branch branch-2
[build #465 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Remove TestReplicationAdminUsingProcedure
> -
>
> Key: HBASE-19697
> URL: https://issues.apache.org/jira/browse/HBASE-19697
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-19697-HBASE-19397.patch
>
>
> It is useless now since we have already on procedure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19719) Fix checkstyle issues
[
https://issues.apache.org/jira/browse/HBASE-19719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393576#comment-16393576
]
Hudson commented on HBASE-19719:
Results for branch branch-2
[build #465 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/465//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix checkstyle issues
> -
>
> Key: HBASE-19719
> URL: https://issues.apache.org/jira/browse/HBASE-19719
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-19719-HBASE-19397.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
