[jira] [Work started] (HBASE-17823) Migrate to Apache Yetus Audience Annotations
[ https://issues.apache.org/jira/browse/HBASE-17823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-17823 started by Sean Busbey. --- > Migrate to Apache Yetus Audience Annotations > > > Key: HBASE-17823 > URL: https://issues.apache.org/jira/browse/HBASE-17823 > Project: HBase > Issue Type: Improvement > Components: API >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0 > > > Migrate from our own audience annotation handling to apache yetus' > implementation. > [discussion thread on > dev@hbase|https://lists.apache.org/thread.html/5a83d37c9c763b3fc4114231489a073167ac69dbade9774af5ca4fb4@%3Cdev.hbase.apache.org%3E] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[
https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133964#comment-16133964
]
Hadoop QA commented on HBASE-17442:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
1s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
10s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
18s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
22s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
24s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
28s{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.0. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
53s{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
17s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.0. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
44s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
10s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.2. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
10s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 40s{color}
| {color:red} hbase-server
[jira] [Created] (HBASE-18633) Add more info to understand the source/scenario of large batch requests exceeding threshold
Vikas Vishwakarma created HBASE-18633: - Summary: Add more info to understand the source/scenario of large batch requests exceeding threshold Key: HBASE-18633 URL: https://issues.apache.org/jira/browse/HBASE-18633 Project: HBase Issue Type: Bug Affects Versions: 1.3.1 Reporter: Vikas Vishwakarma Assignee: Vikas Vishwakarma In our controlled test env, we are seeing frequent Large batch operation detected warnings (as implemented in HBASE-18023). We are not running any client with large batch sizes on this test env, but we start seeing these warnings after some runtime. Maybe it is caused due to some error / retry scenario. Could also be related to Phoenix index updates based on surrounding activity in the logs. Need to add more info like table/region name and anything else that will enable debugging the source or the scenario in which these warnings occur. 2017-08-12 03:40:33,919 WARN [7,queue=0,port=16020] regionserver.RSRpcServices - Large batch operation detected (greater than 5000) (HBASE-18023). Requested Number of Rows: 7108 Client: xxx 2017-08-12 03:40:34,476 WARN [7,queue=0,port=16020] regionserver.RSRpcServices - Large batch operation detected (greater than 5000) (HBASE-18023). Requested Number of Rows: 7096 Client: xxx 2017-08-12 03:40:34,483 WARN [4,queue=0,port=16020] regionserver.RSRpcServices - Large batch operation detected (greater than 5000) (HBASE-18023). Requested Number of Rows: 7091 Client: xxx 2017-08-12 03:40:35,728 WARN [3,queue=0,port=16020] regionserver.RSRpcServices - Large batch operation detected (greater than 5000) (HBASE-18023). Requested Number of Rows: 7102 Client: xxx -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18448) Added support for refreshing HFiles through API and shell
[
https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133959#comment-16133959
]
Hadoop QA commented on HBASE-18448:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
38s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
18s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
53s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 21m
10s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
13s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 13m
26s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
4s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
25s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} hbase-protocol in the patch passed with JDK
v1.8.0_144. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} hbase-client in the patch passed with JDK
v1.8.0_144. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} hbase-server-jdk1.8.0_144 with JDK v1.8.0_144
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} hbase-examples in the patch passed with JDK
v1.8.0_144. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} hbase-protocol in the patch passed with JDK
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} hbase-client in the patch passed with JDK
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} hbase-server-jdk1.7.0_131 with JDK v1.7.0_131
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} hbase-examples in the patch passed with JDK
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml
[jira] [Updated] (HBASE-18621) Refactor ClusterOptions before applying to code base
[ https://issues.apache.org/jira/browse/HBASE-18621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18621: -- Status: Patch Available (was: Open) > Refactor ClusterOptions before applying to code base > > > Key: HBASE-18621 > URL: https://issues.apache.org/jira/browse/HBASE-18621 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan > Attachments: HBASE-18621.master.001.patch, > HBASE-18621.master.002.patch > > > So far, ClusterStatus.Options is not so clean that can be applied to code > base. > Refactoring it before next move. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18621) Refactor ClusterOptions before applying to code base
[ https://issues.apache.org/jira/browse/HBASE-18621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18621: -- Attachment: HBASE-18621.master.002.patch Fix java doc mistakes. "TestCoprocessorWhitelistMasterObserver" is not related, trigger QA again. > Refactor ClusterOptions before applying to code base > > > Key: HBASE-18621 > URL: https://issues.apache.org/jira/browse/HBASE-18621 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan > Attachments: HBASE-18621.master.001.patch, > HBASE-18621.master.002.patch > > > So far, ClusterStatus.Options is not so clean that can be applied to code > base. > Refactoring it before next move. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18621) Refactor ClusterOptions before applying to code base
[ https://issues.apache.org/jira/browse/HBASE-18621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18621: -- Status: Open (was: Patch Available) > Refactor ClusterOptions before applying to code base > > > Key: HBASE-18621 > URL: https://issues.apache.org/jira/browse/HBASE-18621 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan > Attachments: HBASE-18621.master.001.patch > > > So far, ClusterStatus.Options is not so clean that can be applied to code > base. > Refactoring it before next move. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[
https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133949#comment-16133949
]
Hadoop QA commented on HBASE-17442:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
9s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 16s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}109m 0s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}273m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
| |
[jira] [Updated] (HBASE-18577) shaded client includes several non-relocated third party dependencies
[ https://issues.apache.org/jira/browse/HBASE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18577: Attachment: HBASE-18577.WIP.2.patch -02 still a WIP, but now the relocation test passes for server and client - added a bunch of additional unincluded dependencies that had been included but unrelocated - relocated what looks like an optional thing in jets3t Not sure about testing here, given the lack of HBASE-15666. I guess I'll test client manually via YCSB's 1.2 binding, since that uses the shaded client. open to suggestions on how to test the mapreduce stuff in hbase-shaded-server. > shaded client includes several non-relocated third party dependencies > - > > Key: HBASE-18577 > URL: https://issues.apache.org/jira/browse/HBASE-18577 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0, 1.1.2, 1.3.0, 2.0.0-alpha-1 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-3, 1.1.13 > > Attachments: HBASE-18577.WIP.0.patch, HBASE-18577.WIP.-1.patch, > HBASE-18577.WIP.1.patch, HBASE-18577.WIP.2.patch > > > we have some unexpected unrelocated third party dependencies in our shaded > artifacts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18508: -- Attachment: HBASE-18508.HBASE-14070.HLC.003.patch Retry > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133942#comment-16133942
]
Hudson commented on HBASE-14498:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #177 (See
[https://builds.apache.org/job/HBase-1.3-IT/177/])
Revert "HBASE-14498 Master stuck in infinite loop when all Zookeeper (stack:
rev 92a2dfcd6cef87b1b2ec62a5e8912d3c26f9b30d)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (edit)
hbase-client/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperWatcher.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager |
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133941#comment-16133941
]
Hudson commented on HBASE-14498:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #244 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/244/])
Revert "HBASE-14498 Master stuck in infinite loop when all Zookeeper (stack:
rev 92a2dfcd6cef87b1b2ec62a5e8912d3c26f9b30d)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (edit)
hbase-client/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperWatcher.java
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133939#comment-16133939
]
stack commented on HBASE-14498:
---
I reverted from branch-1.3.
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here
> > region server retrieve master location from zookeeper only when they
> > couldn't connect to Master
[jira] [Commented] (HBASE-18623) Frequent failed to parse at EOF warnings from WALEntryStream
[
https://issues.apache.org/jira/browse/HBASE-18623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133935#comment-16133935
]
Sean Busbey commented on HBASE-18623:
-
+1.
> Frequent failed to parse at EOF warnings from WALEntryStream
>
>
> Key: HBASE-18623
> URL: https://issues.apache.org/jira/browse/HBASE-18623
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18623.patch
>
>
> Lots of messages like this
> {noformat}
> 2017-08-17 15:10:36,363 INFO
> [main-EventThread.replicationSource,2.replicationSource.replicationWALReaderThread.onyx%2C8120%2C1503007587035,2]
> regionserver.WALEntryStream: Reached the end of WAL file
> 'hdfs://localhost:8020/hbase-1/WALs/onyx,8120,1503007587035/onyx%2C8120%2C1503007587035.1503007826083'.
> It was not closed cleanly, so we did not parse 8 bytes of data.
> {noformat}
> Fix this, either the message (if harmless) or the underlying cause.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18632:
---
Status: Patch Available (was: Open)
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
Ted Yu created HBASE-18632:
--
Summary: TestMultiParallel#testFlushCommitsWithAbort fails in
master branch
Key: HBASE-18632
URL: https://issues.apache.org/jira/browse/HBASE-18632
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
This can be reproduced:
{code}
java.lang.AssertionError: Waiting timed out after [15,000] msec
at
org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
at
org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
{code}
The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18632:
---
Attachment: 18632.v1.txt
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18592) [hbase-thirdparty] Doc on new hbase-thirdparty dependency for the refguide
[ https://issues.apache.org/jira/browse/HBASE-18592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133930#comment-16133930 ] Hudson commented on HBASE-18592: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3558 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3558/]) HBASE-18592 [hbase-thirdparty] Doc on new hbase-thirdparty dependency (stack: rev b932d38b2a53c1764345fec44e1bbb38addf04ec) * (edit) src/main/asciidoc/_chapters/developer.adoc > [hbase-thirdparty] Doc on new hbase-thirdparty dependency for the refguide > -- > > Key: HBASE-18592 > URL: https://issues.apache.org/jira/browse/HBASE-18592 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Add a bit to the refguide on the new hbase-thirdparty lib and why it exists. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18623) Frequent failed to parse at EOF warnings from WALEntryStream
[
https://issues.apache.org/jira/browse/HBASE-18623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133926#comment-16133926
]
Hadoop QA commented on HBASE-18623:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 27s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}145m
30s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}202m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18623 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882695/HBASE-18623.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 4033d28d9340 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b932d38 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8171/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8171/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Frequent failed to parse at EOF warnings from WALEntryStream
>
>
> Key: HBASE-18623
> URL: https://issues.apache.org/jira/browse/HBASE-18623
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee:
[jira] [Commented] (HBASE-18631) Allow configuration of ChaosMonkey properties via hbase-site
[
https://issues.apache.org/jira/browse/HBASE-18631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133920#comment-16133920
]
Hadoop QA commented on HBASE-18631:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 28s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18631 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882711/HBASE-18631.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f17107dd5ce7 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / b932d38 |
| Default Java | 1.8.0_144 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8173/testReport/ |
| modules | C: hbase-it U: hbase-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8173/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Allow configuration of ChaosMonkey properties via hbase-site
>
>
> Key: HBASE-18631
> URL: https://issues.apache.org/jira/browse/HBASE-18631
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13
>
>
[jira] [Commented] (HBASE-18615) hbase-rest tests fail in hbase-2.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-18615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133914#comment-16133914
]
Hadoop QA commented on HBASE-18615:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
3s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
37s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
50s{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.0. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
37s{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
25s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.0. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
12s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
59s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.2. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 22s{color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}112m 30s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}179m 47s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
[jira] [Updated] (HBASE-18631) Allow configuration of ChaosMonkey properties via hbase-site
[ https://issues.apache.org/jira/browse/HBASE-18631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18631: --- Status: Patch Available (was: Open) > Allow configuration of ChaosMonkey properties via hbase-site > > > Key: HBASE-18631 > URL: https://issues.apache.org/jira/browse/HBASE-18631 > Project: HBase > Issue Type: Improvement > Components: integration tests >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13 > > Attachments: HBASE-18631.001.patch > > > I noticed in some internal test automation that the code was attempting to > configure some of the chaos-monkey actions via hbase-site.xml, but these > weren't taking effect. > After reading the code, I found that these can only be specified via a > special properties file otherwise specified. It would be nice to also allow > configuration via hbase-site. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18631) Allow configuration of ChaosMonkey properties via hbase-site
[ https://issues.apache.org/jira/browse/HBASE-18631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18631: --- Attachment: HBASE-18631.001.patch .001 Simple patch -- hard-codes the prefixes for CM configuration keys and enumerates all keys in the HBaseConfiguration, adding all those that match any prefix into the CM's Properties object. > Allow configuration of ChaosMonkey properties via hbase-site > > > Key: HBASE-18631 > URL: https://issues.apache.org/jira/browse/HBASE-18631 > Project: HBase > Issue Type: Improvement > Components: integration tests >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13 > > Attachments: HBASE-18631.001.patch > > > I noticed in some internal test automation that the code was attempting to > configure some of the chaos-monkey actions via hbase-site.xml, but these > weren't taking effect. > After reading the code, I found that these can only be specified via a > special properties file otherwise specified. It would be nice to also allow > configuration via hbase-site. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133874#comment-16133874
]
Hudson commented on HBASE-14498:
FAILURE: Integrated in Jenkins build HBase-1.4 #871 (See
[https://builds.apache.org/job/HBase-1.4/871/])
HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are
(stack: rev e0f5f3aa1020abe5e744506efd40fad42e76a44e)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* (edit)
hbase-client/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperWatcher.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager |
[jira] [Commented] (HBASE-18448) Added support for refreshing HFiles through API and shell
[ https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133870#comment-16133870 ] Ted Yu commented on HBASE-18448: RefreshHFiles.proto doesn't need to be in hbase-protocol . It should go to hbase-examples module. e.g. hbase-examples//src/main/protobuf/BulkDelete.proto > Added support for refreshing HFiles through API and shell > - > > Key: HBASE-18448 > URL: https://issues.apache.org/jira/browse/HBASE-18448 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.1 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0 > > Attachments: HBASE-18448.branch-1.001.patch, > HBASE-18448.branch-1.002.patch, HBASE-18448.branch-1.003.patch, > HBASE-18448.branch-1.004.patch, HBASE-18448.branch-1.005.patch > > > In the case where multiple HBase clusters are sharing a common rootDir, even > after flushing the data from > one cluster doesn't mean that other clusters (replicas) will automatically > pick the new HFile. Through this patch, > we are exposing the refresh HFiles API which when issued from a replica will > update the in-memory file handle list > with the newly added file. > This allows replicas to be consistent with the data written through the > primary cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18471) The DeleteFamily cell is skipped when StoreScanner seeks to next column
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133856#comment-16133856
]
Hudson commented on HBASE-18471:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #193 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/193/])
HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to
(chia7712: rev 658b01e26e08bb80a130671e6aaad58de258b376)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
> The DeleteFamily cell is skipped when StoreScanner seeks to next column
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.branch-1.2.v0.patch, HBASE-18471.v0.patch,
> HBASE-18471.v1.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18448) Added support for refreshing HFiles through API and shell
[ https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133855#comment-16133855 ] Ajay Jadhav commented on HBASE-18448: - I have made the suggested changes in the recent patch. Please take a look. > Added support for refreshing HFiles through API and shell > - > > Key: HBASE-18448 > URL: https://issues.apache.org/jira/browse/HBASE-18448 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.1 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0 > > Attachments: HBASE-18448.branch-1.001.patch, > HBASE-18448.branch-1.002.patch, HBASE-18448.branch-1.003.patch, > HBASE-18448.branch-1.004.patch, HBASE-18448.branch-1.005.patch > > > In the case where multiple HBase clusters are sharing a common rootDir, even > after flushing the data from > one cluster doesn't mean that other clusters (replicas) will automatically > pick the new HFile. Through this patch, > we are exposing the refresh HFiles API which when issued from a replica will > update the in-memory file handle list > with the newly added file. > This allows replicas to be consistent with the data written through the > primary cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18448) Added support for refreshing HFiles through API and shell
[ https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Jadhav updated HBASE-18448: Status: Open (was: Patch Available) > Added support for refreshing HFiles through API and shell > - > > Key: HBASE-18448 > URL: https://issues.apache.org/jira/browse/HBASE-18448 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1, 2.0.0 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0 > > Attachments: HBASE-18448.branch-1.001.patch, > HBASE-18448.branch-1.002.patch, HBASE-18448.branch-1.003.patch, > HBASE-18448.branch-1.004.patch, HBASE-18448.branch-1.005.patch > > > In the case where multiple HBase clusters are sharing a common rootDir, even > after flushing the data from > one cluster doesn't mean that other clusters (replicas) will automatically > pick the new HFile. Through this patch, > we are exposing the refresh HFiles API which when issued from a replica will > update the in-memory file handle list > with the newly added file. > This allows replicas to be consistent with the data written through the > primary cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18448) Added support for refreshing HFiles through API and shell
[ https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Jadhav updated HBASE-18448: Status: Patch Available (was: Open) > Added support for refreshing HFiles through API and shell > - > > Key: HBASE-18448 > URL: https://issues.apache.org/jira/browse/HBASE-18448 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.1, 2.0.0 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0 > > Attachments: HBASE-18448.branch-1.001.patch, > HBASE-18448.branch-1.002.patch, HBASE-18448.branch-1.003.patch, > HBASE-18448.branch-1.004.patch, HBASE-18448.branch-1.005.patch > > > In the case where multiple HBase clusters are sharing a common rootDir, even > after flushing the data from > one cluster doesn't mean that other clusters (replicas) will automatically > pick the new HFile. Through this patch, > we are exposing the refresh HFiles API which when issued from a replica will > update the in-memory file handle list > with the newly added file. > This allows replicas to be consistent with the data written through the > primary cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18448) Added support for refreshing HFiles through API and shell
[ https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Jadhav updated HBASE-18448: Attachment: HBASE-18448.branch-1.005.patch > Added support for refreshing HFiles through API and shell > - > > Key: HBASE-18448 > URL: https://issues.apache.org/jira/browse/HBASE-18448 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.1 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0 > > Attachments: HBASE-18448.branch-1.001.patch, > HBASE-18448.branch-1.002.patch, HBASE-18448.branch-1.003.patch, > HBASE-18448.branch-1.004.patch, HBASE-18448.branch-1.005.patch > > > In the case where multiple HBase clusters are sharing a common rootDir, even > after flushing the data from > one cluster doesn't mean that other clusters (replicas) will automatically > pick the new HFile. Through this patch, > we are exposing the refresh HFiles API which when issued from a replica will > update the in-memory file handle list > with the newly added file. > This allows replicas to be consistent with the data written through the > primary cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133833#comment-16133833
]
Hudson commented on HBASE-14498:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #255 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/255/])
HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are
(stack: rev 8b977d77851cb3b63b6eff32277bc182128b8201)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (edit)
hbase-client/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperWatcher.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO |
[jira] [Commented] (HBASE-18471) The DeleteFamily cell is skipped when StoreScanner seeks to next column
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133830#comment-16133830
]
Hudson commented on HBASE-18471:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3557 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3557/])
HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to
(chia7712: rev 56f9e1a60af224f8f3b0453932155ca46d44cf71)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/ScanQueryMatcher.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
> The DeleteFamily cell is skipped when StoreScanner seeks to next column
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.branch-1.2.v0.patch, HBASE-18471.v0.patch,
> HBASE-18471.v1.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18572) Delete can't remove the cells which have no visibility label
[
https://issues.apache.org/jira/browse/HBASE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133829#comment-16133829
]
Hudson commented on HBASE-18572:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3557 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3557/])
HBASE-18572 Delete can't remove the cells which have no visibility label
(chia7712: rev e9bafeb091549cce35f25a56688b6632578de74b)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
> Delete can't remove the cells which have no visibility label
>
>
> Key: HBASE-18572
> URL: https://issues.apache.org/jira/browse/HBASE-18572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18572.branch-1.v0.patch,
> HBASE-18572.branch-1.v0.patch, HBASE-18572.branch-1.v0.patch,
> HBASE-18572.ut.patch, HBASE-18572.v0.patch, HBASE-18572.v0.patch,
> HBASE-18572.v1.patch
>
>
> {code:title=DefaultVisibilityLabelServiceImpl.java}
> @Override
> public boolean matchVisibility(List putVisTags, Byte putTagsFormat,
> List deleteVisTags,
> Byte deleteTagsFormat) throws IOException {
> if ((deleteTagsFormat != null && deleteTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)
> && (putTagsFormat == null || putTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)) {
> if (putVisTags.isEmpty()) {
> // Early out if there are no tags in the cell
> return false;
> }
> if (putTagsFormat == null) {
> return matchUnSortedVisibilityTags(putVisTags, deleteVisTags);
> } else {
> return matchOrdinalSortedVisibilityTags(putVisTags, deleteVisTags);
> }
> }
> throw new IOException("Unexpected tag format passed for comparison,
> deleteTagsFormat : "
> + deleteTagsFormat + ", putTagsFormat : " + putTagsFormat);
> }
> {code}
> If the putVisTags is empty, it will return the false directly. Hence, the
> Delete#addColumn can't "see" the cell as well as deleting it. The
> Delete#addColumns, by contrast, doesn't invoke the
> cp#prePrepareTimeStampForDeleteVersion so it can remove the cell.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17442: - Attachment: HBASE-17442.branch-2.001.patch > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.branch-2.001.patch, HBASE-17442.master.001.patch, > HBASE-17442.v1.patch, HBASE-17442.v2.patch, HBASE-17442.v2.patch, > HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18623) Frequent failed to parse at EOF warnings from WALEntryStream
[
https://issues.apache.org/jira/browse/HBASE-18623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133826#comment-16133826
]
Andrew Purtell commented on HBASE-18623:
Going to commit trivial patch implementing Sean's suggestion, a logging only
change, tomorrow unless objection.
> Frequent failed to parse at EOF warnings from WALEntryStream
>
>
> Key: HBASE-18623
> URL: https://issues.apache.org/jira/browse/HBASE-18623
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18623.patch
>
>
> Lots of messages like this
> {noformat}
> 2017-08-17 15:10:36,363 INFO
> [main-EventThread.replicationSource,2.replicationSource.replicationWALReaderThread.onyx%2C8120%2C1503007587035,2]
> regionserver.WALEntryStream: Reached the end of WAL file
> 'hdfs://localhost:8020/hbase-1/WALs/onyx,8120,1503007587035/onyx%2C8120%2C1503007587035.1503007826083'.
> It was not closed cleanly, so we did not parse 8 bytes of data.
> {noformat}
> Fix this, either the message (if harmless) or the underlying cause.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133825#comment-16133825
]
Hudson commented on HBASE-14498:
FAILURE: Integrated in Jenkins build HBase-1.5 #18 (See
[https://builds.apache.org/job/HBase-1.5/18/])
HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are
(stack: rev 7a96682643ebfe1477a6c628359aba3e57ef1c9e)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (edit)
hbase-client/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperWatcher.java
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager |
[jira] [Updated] (HBASE-18623) Frequent failed to parse at EOF warnings from WALEntryStream
[
https://issues.apache.org/jira/browse/HBASE-18623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-18623:
---
Status: Patch Available (was: Open)
> Frequent failed to parse at EOF warnings from WALEntryStream
>
>
> Key: HBASE-18623
> URL: https://issues.apache.org/jira/browse/HBASE-18623
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18623.patch
>
>
> Lots of messages like this
> {noformat}
> 2017-08-17 15:10:36,363 INFO
> [main-EventThread.replicationSource,2.replicationSource.replicationWALReaderThread.onyx%2C8120%2C1503007587035,2]
> regionserver.WALEntryStream: Reached the end of WAL file
> 'hdfs://localhost:8020/hbase-1/WALs/onyx,8120,1503007587035/onyx%2C8120%2C1503007587035.1503007826083'.
> It was not closed cleanly, so we did not parse 8 bytes of data.
> {noformat}
> Fix this, either the message (if harmless) or the underlying cause.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18623) Frequent failed to parse at EOF warnings from WALEntryStream
[
https://issues.apache.org/jira/browse/HBASE-18623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-18623:
---
Attachment: HBASE-18623.patch
Trivial patch
> Frequent failed to parse at EOF warnings from WALEntryStream
>
>
> Key: HBASE-18623
> URL: https://issues.apache.org/jira/browse/HBASE-18623
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18623.patch
>
>
> Lots of messages like this
> {noformat}
> 2017-08-17 15:10:36,363 INFO
> [main-EventThread.replicationSource,2.replicationSource.replicationWALReaderThread.onyx%2C8120%2C1503007587035,2]
> regionserver.WALEntryStream: Reached the end of WAL file
> 'hdfs://localhost:8020/hbase-1/WALs/onyx,8120,1503007587035/onyx%2C8120%2C1503007587035.1503007826083'.
> It was not closed cleanly, so we did not parse 8 bytes of data.
> {noformat}
> Fix this, either the message (if harmless) or the underlying cause.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18623) Frequent failed to parse at EOF warnings from WALEntryStream
[
https://issues.apache.org/jira/browse/HBASE-18623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reassigned HBASE-18623:
--
Assignee: Andrew Purtell
Priority: Minor (was: Major)
Fix Version/s: 3.0.0
2.0.0
> Frequent failed to parse at EOF warnings from WALEntryStream
>
>
> Key: HBASE-18623
> URL: https://issues.apache.org/jira/browse/HBASE-18623
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18623.patch
>
>
> Lots of messages like this
> {noformat}
> 2017-08-17 15:10:36,363 INFO
> [main-EventThread.replicationSource,2.replicationSource.replicationWALReaderThread.onyx%2C8120%2C1503007587035,2]
> regionserver.WALEntryStream: Reached the end of WAL file
> 'hdfs://localhost:8020/hbase-1/WALs/onyx,8120,1503007587035/onyx%2C8120%2C1503007587035.1503007826083'.
> It was not closed cleanly, so we did not parse 8 bytes of data.
> {noformat}
> Fix this, either the message (if harmless) or the underlying cause.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18471) The DeleteFamily cell is skipped when StoreScanner seeks to next column
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133822#comment-16133822
]
Hudson commented on HBASE-18471:
FAILURE: Integrated in Jenkins build HBase-2.0 #354 (See
[https://builds.apache.org/job/HBase-2.0/354/])
HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to
(chia7712: rev bb2b6b86623233bcc12dc3276ae805c45241875b)
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/ScanQueryMatcher.java
> The DeleteFamily cell is skipped when StoreScanner seeks to next column
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.branch-1.2.v0.patch, HBASE-18471.v0.patch,
> HBASE-18471.v1.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133819#comment-16133819 ] Guanghao Zhang commented on HBASE-17442: It will be great if we can move all replication classes to a separate hbase-replication module. :-) bq. but it's coupled with server code. Yes. The hbase-server's replication classes may need more time to decouple... > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.master.001.patch, HBASE-17442.v1.patch, HBASE-17442.v2.patch, > HBASE-17442.v2.patch, HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18572) Delete can't remove the cells which have no visibility label
[
https://issues.apache.org/jira/browse/HBASE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133821#comment-16133821
]
Hudson commented on HBASE-18572:
FAILURE: Integrated in Jenkins build HBase-2.0 #354 (See
[https://builds.apache.org/job/HBase-2.0/354/])
HBASE-18572 Delete can't remove the cells which have no visibility label
(chia7712: rev c013bf8a7a196188b42fe68eab172654191323bc)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java
> Delete can't remove the cells which have no visibility label
>
>
> Key: HBASE-18572
> URL: https://issues.apache.org/jira/browse/HBASE-18572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18572.branch-1.v0.patch,
> HBASE-18572.branch-1.v0.patch, HBASE-18572.branch-1.v0.patch,
> HBASE-18572.ut.patch, HBASE-18572.v0.patch, HBASE-18572.v0.patch,
> HBASE-18572.v1.patch
>
>
> {code:title=DefaultVisibilityLabelServiceImpl.java}
> @Override
> public boolean matchVisibility(List putVisTags, Byte putTagsFormat,
> List deleteVisTags,
> Byte deleteTagsFormat) throws IOException {
> if ((deleteTagsFormat != null && deleteTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)
> && (putTagsFormat == null || putTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)) {
> if (putVisTags.isEmpty()) {
> // Early out if there are no tags in the cell
> return false;
> }
> if (putTagsFormat == null) {
> return matchUnSortedVisibilityTags(putVisTags, deleteVisTags);
> } else {
> return matchOrdinalSortedVisibilityTags(putVisTags, deleteVisTags);
> }
> }
> throw new IOException("Unexpected tag format passed for comparison,
> deleteTagsFormat : "
> + deleteTagsFormat + ", putTagsFormat : " + putTagsFormat);
> }
> {code}
> If the putVisTags is empty, it will return the false directly. Hence, the
> Delete#addColumn can't "see" the cell as well as deleting it. The
> Delete#addColumns, by contrast, doesn't invoke the
> cp#prePrepareTimeStampForDeleteVersion so it can remove the cell.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18518) Remove jersey1* dependencies from project and jersey1* jars from lib dir
[ https://issues.apache.org/jira/browse/HBASE-18518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133820#comment-16133820 ] Hudson commented on HBASE-18518: FAILURE: Integrated in Jenkins build HBase-2.0 #354 (See [https://builds.apache.org/job/HBase-2.0/354/]) Revert "HBASE-18518 Remove jersey1* dependencies form project and (stack: rev 3ac60fc0d87911c42460066a1d30678a2963582d) * (edit) hbase-rest/pom.xml * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * (edit) pom.xml Revert "Revert "HBASE-18518 Remove jersey1* dependencies form project (stack: rev 8c85b6b85336747a583d720d428a9e92caae38b9) * (edit) hbase-rest/pom.xml * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * (edit) pom.xml > Remove jersey1* dependencies from project and jersey1* jars from lib dir > > > Key: HBASE-18518 > URL: https://issues.apache.org/jira/browse/HBASE-18518 > Project: HBase > Issue Type: Task > Components: dependencies, pom, REST >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Samir Ahmic >Assignee: Samir Ahmic > Labels: cleanup > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18518-master-01.patch, HBASE-18518-master-02.patch > > > Recently i have opened https://issues.apache.org/jira/browse/HBASE-18506 and > it is clear that is caused by mixing jersey1 and jersey2 jars in classpath. > With https://issues.apache.org/jira/browse/HBASE-12894 we have introduced > jersey2 to project, and we also have bunch of transitive dependencies > (mainly from hadoop) on jersey1 which is not happiest situation since jersey1 > and jersey2 under same classpath can case runtime issues as it was case with > rest. > This task will have following steps > * Clean code and replace jersey1 constructs with jersey2 versions(there > should not be much of this) > * Add exclusions for transitive jersey1 dependencies in pom.xml > * Add exclusions in hadoop-two-compat.xml to prevent jersey1 jars in lib dir -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18615) hbase-rest tests fail in hbase-2.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-18615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133805#comment-16133805 ] Andrew Purtell commented on HBASE-18615: Many of the minicluster based tests are still not passing Failed tests: TestDeleteRow.testDeleteNonExistentColumn:36 expected:<415> but was:<200> TestGetAndPutResource.testLatestCellGetJSON:305 expected:<415> but was:<200> TestGetAndPutResource.testMetrics:538 expected:<500> but was:<200> TestGetAndPutResource.testMultiCellGetJson:519->RowResourceBase.checkValueJSON:512 expected:<404> but was:<200> TestGetAndPutResource.testSingleCellGetJSON:284 expected:<500> but was:<200> TestMultiRowResource.testMultiCellGetJSONNotFound:231 expected:<500> but was:<200> TestMultiRowResource.testMultiCellGetJSONNotFound:231 expected:<500> but was:<200> TestMultiRowResource.testMultiCellGetJSON:148 expected:<500> but was:<200> TestMultiRowResource.testMultiCellGetJSON:148 expected:<500> but was:<200> TestMultiRowResource.testMultiCellGetWithColsJSON:202 expected:<500> but was:<200> TestMultiRowResource.testMultiCellGetWithColsJSON:202 expected:<500> but was:<200> TestNamespacesInstanceResource.testGetNamespaceTablesAndCannotDeleteNamespace:207 expected:<200> but was:<500> TestNamespacesInstanceResource.testInvalidNamespacePostsAndPuts:268 expected:<400> but was:<415> TestNamespacesInstanceResource.testNamespaceCreateAndDeleteXMLAndJSON:306 expected:<403> but was:<415> TestNamespacesResource.testNamespaceListXMLandJSON:115 expected:<200> but was:<500> TestSchemaResource.testTableCreateAndDeletePB:198 expected:<403> but was:<201> TestSchemaResource.testTableCreateAndDeletePB:198 expected:<403> but was:<201> TestSchemaResource.testTableCreateAndDeleteXML:154 expected:<500> but was:<200> TestSchemaResource.testTableCreateAndDeleteXML:126 TestTableResource.testTableInfoJSON:264 expected:<500> but was:<200> TestTableResource.testTableListJSON:223 expected:<500> but was:<200> TestTableScan.testScanningUnknownColumnJson:460 expected:<200> but was:<500> TestTableScan.testSimpleScannerJson:207 expected:<200> but was:<500> TestTableScan.testStreamingJSON:317 expected:<200> but was:<500> TestVersionResource.doTestGetStorageClusterVersionJSON:174 TestVersionResource.testGetStargateVersionJSON:129 Tests run: 100, Failures: 26, Errors: 0, Skipped: 0 > hbase-rest tests fail in hbase-2.0.0-alpha2 > --- > > Key: HBASE-18615 > URL: https://issues.apache.org/jira/browse/HBASE-18615 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack > Attachments: 18615.2.txt, 18615.txt, 18615.v3.txt, > HBASE-18615.branch-2.001.patch > > > Pointed out by Andrew on VOTE mail for hbase-2.0.0-alpha2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18471) The DeleteFamily cell is skipped when StoreScanner seeks to next column
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133803#comment-16133803
]
Hudson commented on HBASE-18471:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #254 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/254/])
HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to
(chia7712: rev bc925b4e857e28a2377c8c325eaa2819ab7a7524)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
> The DeleteFamily cell is skipped when StoreScanner seeks to next column
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.branch-1.2.v0.patch, HBASE-18471.v0.patch,
> HBASE-18471.v1.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133802#comment-16133802 ] Appy commented on HBASE-17442: -- btw, i tried looking around hbase-server/.../replication/*.java classes to move more stuff, but it's coupled with server code. Few of the classes which i saw were loosely coupled and might be easy to move, but i didn't look at everything, so don't have much idea of full scope of that work. > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.master.001.patch, HBASE-17442.v1.patch, HBASE-17442.v2.patch, > HBASE-17442.v2.patch, HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133800#comment-16133800
]
Andrew Purtell commented on HBASE-14498:
Passes on branch-1.4 and branch-1. I'm guessing an issue only with 1.3.
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here
> > region server retrieve master location from zookeeper
[jira] [Commented] (HBASE-18628) ZKPermissionWatcher blocks all ZK notifications
[
https://issues.apache.org/jira/browse/HBASE-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133801#comment-16133801
]
Hadoop QA commented on HBASE-18628:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}156m
14s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}209m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18628 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882658/HBASE-18628.v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ab708dbe8621 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 56f9e1a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/8162/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8162/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8162/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> ZKPermissionWatcher blocks all ZK notifications
> ---
>
> Key: HBASE-18628
> URL: https://issues.apache.org/jira/browse/HBASE-18628
> Project: HBase
> Issue Type: Bug
>
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133799#comment-16133799 ] Appy commented on HBASE-17442: -- Uploaded new patch which is same as [~zghaobac]'s except that we are moving the classes to separate hbase-replication module. Since these classes are not needed in hbase-client, and since they don't need anything from hbase-server, there's not point moving them to hbase-server. Probably the best time to start hbase-replication module. However, separating interfaces and implementations into different packages within this module will be much more involved work (right now everything is in o.a.h.h.replication) and something for which i don't have bandwidth. Eventually, it'll be great if we can move out more replication pieces from hbase-server to this module, but for now, maybe this is great start? > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.master.001.patch, HBASE-17442.v1.patch, HBASE-17442.v2.patch, > HBASE-17442.v2.patch, HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-14498:
Unit test always failing on branch-1.3
Running org.apache.hadoop.hbase.zookeeper.TestZooKeeperWatcher
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 16.631 sec <<<
FAILURE! - in org.apache.hadoop.hbase.zookeeper.TestZooKeeperWatcher
testConnectionEvent(org.apache.hadoop.hbase.zookeeper.TestZooKeeperWatcher)
Time elapsed: 16.441 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.zookeeper.TestZooKeeperWatcher.testConnectionEvent(TestZooKeeperWatcher.java:105)
Results :
Failed tests:
TestZooKeeperWatcher.testConnectionEvent:105 null
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout
[jira] [Updated] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-17442: - Attachment: HBASE-17442.master.001.patch > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.master.001.patch, HBASE-17442.v1.patch, HBASE-17442.v2.patch, > HBASE-17442.v2.patch, HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133789#comment-16133789
]
stack commented on HBASE-18606:
---
This is test context. Why not install first? It works for me on linux too. If
you mvn install first does it work? Need more info please on your context.
stack@ve0524:~$ uname -a
Linux ve0524 3.13.0-83-generic #127-Ubuntu SMP Fri Mar 11 00:25:37 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux
stack@ve0524:~$ more /etc/issue
Ubuntu 14.04.4 LTS \n \l
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18627) Fix TestRegionServerReadRequestMetrics
[
https://issues.apache.org/jira/browse/HBASE-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133780#comment-16133780
]
Hadoop QA commented on HBASE-18627:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
9s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
55m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}181m 40s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
51s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}289m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver |
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.wal.TestWALFiltering |
| | org.apache.hadoop.hbase.TestZooKeeper |
| | org.apache.hadoop.hbase.wal.TestWALSplitCompressed |
| | org.apache.hadoop.hbase.quotas.TestRegionSizeUse |
| | org.apache.hadoop.hbase.util.TestHBaseFsckEncryption |
| | org.apache.hadoop.hbase.master.assignment.TestAssignmentOnRSCrash |
| | org.apache.hadoop.hbase.master.TestSplitLogManager |
| | org.apache.hadoop.hbase.master.balancer.TestFavoredNodeTableImport |
| | org.apache.hadoop.hbase.master.TestRollingRestart |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer |
| | org.apache.hadoop.hbase.wal.TestSecureWAL |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded |
| | org.apache.hadoop.hbase.master.TestDistributedLogSplitting |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential |
| | org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat2 |
| |
[jira] [Commented] (HBASE-18532) Improve cache related stats rendered on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133778#comment-16133778
]
Hadoop QA commented on HBASE-18532:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 25m 5s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
17s{color} | {color:green} hbase-external-blockcache in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 83m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.TestHeapSize |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18532 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882678/HBASE-18532.PATCH |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6990678006ea 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master /
[jira] [Comment Edited] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133777#comment-16133777
]
Ted Yu edited comment on HBASE-18606 at 8/18/17 11:32 PM:
--
You probably have run "mvn install" on the machine.
If you execute the following:
rm -rf ~/.m2/repository
Does the same command still pass for master branch ?
I was using Linux for today's runs.
was (Author: yuzhih...@gmail.com):
You probably have run "mvn install" on the machine.
If you execute the following:
rm ~/.m2/repository
Does the same command still pass for master branch ?
I was using Linux for today's runs.
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133777#comment-16133777
]
Ted Yu commented on HBASE-18606:
You probably have run "mvn install" on the machine.
If you execute the following:
rm ~/.m2/repository
Does the same command still pass for master branch ?
I was using Linux for today's runs.
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18615) hbase-rest tests fail in hbase-2.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-18615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18615: -- Attachment: HBASE-18615.branch-2.001.patch > hbase-rest tests fail in hbase-2.0.0-alpha2 > --- > > Key: HBASE-18615 > URL: https://issues.apache.org/jira/browse/HBASE-18615 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack > Attachments: 18615.2.txt, 18615.txt, 18615.v3.txt, > HBASE-18615.branch-2.001.patch > > > Pointed out by Andrew on VOTE mail for hbase-2.0.0-alpha2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18615) hbase-rest tests fail in hbase-2.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-18615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133774#comment-16133774 ] stack commented on HBASE-18615: --- Ok. This took a while. Now I can run REST in all three locations -- standalone out of a tarball, in-place inside a build dir after build, and unit tests pass. The patch is for branch-2 for now. The master and branch-2 have deviated a little around jersey/jackson handling. This article was of help: https://stackoverflow.com/questions/39458230/how-register-jacksonfeature-on-clientconfig It explained what the Jackson1Feature setting in RESTServer was supposed to do. Explained it dodgy and the proper alternative. It took me a while to then figure that the TestRESTServer class was different in branch-2 to master. I filtered out all of hadoops jersey1s. This helped for the case where we are running in-situ -- no jersey1 pollution. It seems like java will default to jersey1 if nothing else found so cut-it off by setting a jersey2 RuntimeDelegate as soon as the RESTServer starts. Remove all jackson1 references and includes of jersey-media-json-jackson1. Lets see how this patch does now. > hbase-rest tests fail in hbase-2.0.0-alpha2 > --- > > Key: HBASE-18615 > URL: https://issues.apache.org/jira/browse/HBASE-18615 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack > Attachments: 18615.2.txt, 18615.txt, 18615.v3.txt > > > Pointed out by Andrew on VOTE mail for hbase-2.0.0-alpha2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133769#comment-16133769
]
stack commented on HBASE-18606:
---
Are you on linux?
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18592) [hbase-thirdparty] Doc on new hbase-thirdparty dependency for the refguide
[
https://issues.apache.org/jira/browse/HBASE-18592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133768#comment-16133768
]
stack commented on HBASE-18592:
---
Looks like my claim above that I'd pushed the above doc was wrong. Pushed it
just now. Below is what I pushed. Added doc of the netty system property we
need from here on out.
{code}
+[[thirdparty]]
+=== The hbase-thirdparty dependency and shading/relocation
+
+A new project was created for the release of hbase-2.0.0. It was called
+`hbase-thirdparty`. This project exists only to provide the main hbase
+project with relocated -- or shaded -- versions of popular thirdparty
+libraries such as guava, netty, and protobuf. The mainline HBase project
+relies on the relocated versions of these libraries gotten from
hbase-thirdparty
+rather than on finding these classes in their usual locations. We do this so
+we can specify whatever the version we wish. If we don't relocate, we must
+harmonize our version to match that which hadoop and/or spark uses.
+
+For developers, this means you need to be careful referring to classes from
+netty, guava, protobuf, gson, etc. (see the hbase-thirdparty pom.xml for what
+it provides). Devs must refer to the hbase-thirdparty provided classes. In
+practice, this is usually not an issue (though it can be a bit of a pain). You
+will have to hunt for the relocated version of your particular class. You'll
+find it by prepending the general relocation prefix of
`org.apache.hadoop.hbase.shaded.`.
+For example if you are looking for `com.google.protobuf.Message`, the relocated
+version used by HBase internals can be found at
+`org.apache.hadoop.hbase.shaded.com.google.protobuf.Message`.
+
+For a few thirdparty libs, like protobuf (see the protobuf chapter in this book
+for the why), your IDE may give you both options -- the `com.google.protobuf.*`
+and the `org.apache.hadoop.hbase.shaded.com.google.protobuf.*` -- because both
+classes are on your CLASSPATH. Unless you are doing the particular juggling
+required in Coprocessor Endpoint development (again see above cited protobuf
+chapter), you'll want to use the shaded version, always.
+
+Of note, the relocation of netty is particular. The netty folks have put in
+place facility to aid relocation; it seems like shading netty is a popular
project.
+One case of this requires the setting of a peculiar system property on the JVM
+so that classes out in the bundld shared library (.so) can be found in their
+relocated location. Here is the property that needs to be set:
+
+`-Dorg.apache.hadoop.hbase.shaded.io.netty.packagePrefix=org.apache.hadoop.hbase.shaded.`
+
+(Note that the trailing '.' is required). Starting hbase normally or when
running
+test suites, the setting of this property is done for you. If you are doing
something
+out of the ordinary, starting hbase from your own context, you'll need to
provide
+this property on platforms that favor the bundled .so. See release notes on
HBASE-18271
+for more. The complaint you see is something like the following:
+`Cause: java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.`
+
+The `hbase-thirdparty` project has groupid of `org.apache.hbase.thirdparty`.
+As of this writing, it provides three jars; one for netty with an artifactid of
+`hbase-thirdparty-netty`, one for protobuf at `hbase-thirdparty-protobuf` and
then
+a jar for all else -- gson, guava -- at `hbase-thirdpaty-miscellaneous`.
+
+The hbase-thirdparty artifacts are a product produced by the Apache HBase
+project under the aegis of the HBase Project Management Committee. Releases
+are done via the usual voting project on the hbase dev mailing list. If issue
+in the hbase-thirdparty, use the hbase JIRA and mailing lists to post notice.
+
{code}
> [hbase-thirdparty] Doc on new hbase-thirdparty dependency for the refguide
> --
>
> Key: HBASE-18592
> URL: https://issues.apache.org/jira/browse/HBASE-18592
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
>
> Add a bit to the refguide on the new hbase-thirdparty lib and why it exists.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18621) Refactor ClusterOptions before applying to code base
[
https://issues.apache.org/jira/browse/HBASE-18621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133765#comment-16133765
]
Hadoop QA commented on HBASE-18621:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
20s{color} | {color:red} hbase-client generated 2 new + 0 unchanged - 0 fixed =
2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
45s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}154m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}214m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18621 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882656/HBASE-18621.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e5b45af35b7d 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 56f9e1a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| javadoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/8160/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
|
| unit |
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133752#comment-16133752
]
stack commented on HBASE-18606:
---
[~te...@apache.org] Works for me. Interested in more explicit instructions on
how to reproduce. I'm trying to shut down all cases where the property is
missing.
[~busbey] I have running doc on relocation here... HBASE-18592 will add
mention of the system property (says it was pushed but looks like it wasn't).
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[
https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133743#comment-16133743
]
Hadoop QA commented on HBASE-18508:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 32s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 73m 58s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}137m 37s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestStoreScanner |
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | org.apache.hadoop.hbase.regionserver.TestRowTooBig |
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.master.procedure.TestSafemodeBringsDownMaster |
| | org.apache.hadoop.hbase.client.TestMultipleTimestamps |
| | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.trace.TestHTraceHooks |
| | org.apache.hadoop.hbase.backup.TestRemoteBackup |
| | org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics |
| | org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi |
| | org.apache.hadoop.hbase.client.TestAsyncProcedureAdminApi |
| | org.apache.hadoop.hbase.regionserver.TestTimestampFilterSeekHint |
| |
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133736#comment-16133736
]
Sean Busbey commented on HBASE-18606:
-
Can we make sure we have an update to [the ref guide section on testing
application using the
HBaseTestingUtility|http://hbase.apache.org/book.html#_integration_testing_with_an_hbase_mini_cluster]
to call this config out? I'm fine with a follow on instead of here.
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133732#comment-16133732
]
Ted Yu commented on HBASE-18606:
I used master branch.
Same procedure: cd hbase-spark; mvn test ...
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133728#comment-16133728
]
stack commented on HBASE-18606:
---
What branch? Below is cd hbase-spark and then $ mvn test -PrunAllTests
{code}
- avro Or filter
KEY_FIELD STRING :key, A_FIELD STRING c:a, B_FIELD DOUBLE c:b, C_FIELD BINARY
c:c,
KEY_FIELD STRING :key, A_FIELD STRING c:a, B_FIELD DOUBLE c:b, C_FIELD BINARY
c:c,
HBaseCatalogSuite:
- basic
- parse MAP
- parse array
- parse MAp
- convert
- compatiblity
HBaseRDDFunctionsSuite:
Formatting using clusterid: testClusterID
- bulkput to test HBase client
- bulkDelete to test HBase client
- bulkGet to test HBase client
- bulkGet default converter to test HBase client
- foreachPartition with puts to test HBase client
- mapPartitions with Get from test HBase client
HBaseConnectionCacheSuite:
- all test cases
BulkLoadSuite:
Formatting using clusterid: testClusterID
- Wide Row Bulk Load: Test multi family and multi column tests with all default
HFile Configs.
- Wide Row Bulk Load: Test HBase client: Test Roll Over and using an implicit
call to bulk load
- Wide Row Bulk Load: Test multi family and multi column tests with one column
family with custom configs plus multi region
- Test partitioner
- Thin Row Bulk Load: Test multi family and multi column tests with all default
HFile Configs
- Thin Row Bulk Load: Test HBase client: Test Roll Over and using an implicit
call to bulk load
- Thin Row Bulk Load: Test multi family and multi column tests with one column
family with custom configs plus multi region
Run completed in 2 minutes, 31 seconds.
Total number of tests run: 79
Suites: completed 10, aborted 0
Tests: succeeded 79, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
[INFO]
[INFO] --- maven-surefire-plugin:2.19.1:test (secondPartTestsExecution) @
hbase-spark ---
---
T E S T S
---
Running org.apache.hadoop.hbase.spark.TestJavaHBaseContext
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.408 sec - in
org.apache.hadoop.hbase.spark.TestJavaHBaseContext
Results :
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 03:42 min
[INFO] Finished at: 2017-08-18T15:36:11-07:00
[INFO] Final Memory: 54M/964M
[INFO]
{code}
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
>
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133710#comment-16133710
]
Ted Yu commented on HBASE-18606:
The above addition seems to be the same as patch v1.
I used this command:
mvn test -PrunAllTests
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18471) The DeleteFamily cell is skipped when StoreScanner seeks to next column
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133707#comment-16133707
]
Hudson commented on HBASE-18471:
FAILURE: Integrated in Jenkins build HBase-1.4 #870 (See
[https://builds.apache.org/job/HBase-1.4/870/])
HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to
(chia7712: rev 73732fd9fa1c82a0b004a7efc5e61d1dc06868d0)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/ScanQueryMatcher.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
> The DeleteFamily cell is skipped when StoreScanner seeks to next column
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.branch-1.2.v0.patch, HBASE-18471.v0.patch,
> HBASE-18471.v1.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18572) Delete can't remove the cells which have no visibility label
[
https://issues.apache.org/jira/browse/HBASE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133706#comment-16133706
]
Hudson commented on HBASE-18572:
FAILURE: Integrated in Jenkins build HBase-1.4 #870 (See
[https://builds.apache.org/job/HBase-1.4/870/])
HBASE-18572 Delete can't remove the cells which have no visibility label
(chia7712: rev e116e8b03cf5c6fea4fa0f2dd5d71957acd5)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java
> Delete can't remove the cells which have no visibility label
>
>
> Key: HBASE-18572
> URL: https://issues.apache.org/jira/browse/HBASE-18572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18572.branch-1.v0.patch,
> HBASE-18572.branch-1.v0.patch, HBASE-18572.branch-1.v0.patch,
> HBASE-18572.ut.patch, HBASE-18572.v0.patch, HBASE-18572.v0.patch,
> HBASE-18572.v1.patch
>
>
> {code:title=DefaultVisibilityLabelServiceImpl.java}
> @Override
> public boolean matchVisibility(List putVisTags, Byte putTagsFormat,
> List deleteVisTags,
> Byte deleteTagsFormat) throws IOException {
> if ((deleteTagsFormat != null && deleteTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)
> && (putTagsFormat == null || putTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)) {
> if (putVisTags.isEmpty()) {
> // Early out if there are no tags in the cell
> return false;
> }
> if (putTagsFormat == null) {
> return matchUnSortedVisibilityTags(putVisTags, deleteVisTags);
> } else {
> return matchOrdinalSortedVisibilityTags(putVisTags, deleteVisTags);
> }
> }
> throw new IOException("Unexpected tag format passed for comparison,
> deleteTagsFormat : "
> + deleteTagsFormat + ", putTagsFormat : " + putTagsFormat);
> }
> {code}
> If the putVisTags is empty, it will return the false directly. Hence, the
> Delete#addColumn can't "see" the cell as well as deleting it. The
> Delete#addColumns, by contrast, doesn't invoke the
> cp#prePrepareTimeStampForDeleteVersion so it can remove the cell.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133705#comment-16133705 ] stack commented on HBASE-17442: --- Sounds good @appy. Thanks for doing research. Lets commit and open a new issue to investigate moving from hbase-server to hbase-replication module. > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.v1.patch, HBASE-17442.v2.patch, HBASE-17442.v2.patch, > HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133700#comment-16133700
]
stack commented on HBASE-18606:
---
Setting system property in java context after the process has started is too
late for shared library loadings.
I did this for scalatest a few days back to make scalatest work.
org.apache.hadoop.hbase.shaded.
What command are you running?
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133696#comment-16133696 ] Appy commented on HBASE-17442: -- The only two methods (peerAdded and listReplicationPeers) this patch is deleting were marked @VisibleForTesting listValidReplicationPeers() was added as private in HBASE-13057, made public + visibleForTesting in HBASE-14866, then renamed to listReplicationPeers later peerAdded() was added as visibleForTesting in HBASE-15507. So i think it's fine to commit this patch. What say [~zghaobac] [~busbey] [~stack]? > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.v1.patch, HBASE-17442.v2.patch, HBASE-17442.v2.patch, > HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133689#comment-16133689
]
Hudson commented on HBASE-14498:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #176 (See
[https://builds.apache.org/job/HBase-1.3-IT/176/])
HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are
(stack: rev 8b977d77851cb3b63b6eff32277bc182128b8201)
* (edit)
hbase-client/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperWatcher.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager
[jira] [Work started] (HBASE-18103) [AMv2] If Master gives OPEN to another, if original eventually succeeds, Master will kill it
[ https://issues.apache.org/jira/browse/HBASE-18103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18103 started by Umesh Agashe. > [AMv2] If Master gives OPEN to another, if original eventually succeeds, > Master will kill it > > > Key: HBASE-18103 > URL: https://issues.apache.org/jira/browse/HBASE-18103 > Project: HBase > Issue Type: Sub-task > Components: master, proc-v2 >Reporter: stack >Assignee: Umesh Agashe >Priority: Critical > Fix For: 2.0.0 > > > If a RS is slow to open a Region, the Master will give the Region to another > to open it (In this case, was a massive set of edits to process and a load of > StoreFiles to open...). Should the original RS succeed with its open > eventually, on reporting the master the successful open, the Master currently > kills the RS because the region is supposed to be elsewhere. > This is an easy fix. > The RS does not fully open a Region until Master gives it the go so just > close the region if master rejects the open > See '6.1.1 If Master gives Region to another to Open, old RS will be kill > itself on reject by Master; easy fix!' in > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.qtfojp9774h -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18628) ZKPermissionWatcher blocks all ZK notifications
[
https://issues.apache.org/jira/browse/HBASE-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133684#comment-16133684
]
Sean Busbey commented on HBASE-18628:
-
besides the above, v2 looks good.
> ZKPermissionWatcher blocks all ZK notifications
> ---
>
> Key: HBASE-18628
> URL: https://issues.apache.org/jira/browse/HBASE-18628
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Attachments: HBASE-18628.patch, HBASE-18628.v2.patch, jstack
>
>
> Buckle up folks, we're going for a ride here. I've seeing this on a branch-2
> based build, but I think the problem will affect branch-1 as well. I'm not
> able to easily reproduce the issue, but it will usually come up within an
> hour on a given cluster that I have, at which point the problem persists
> until an RS restart. I've been seeing the problem and paying attention for
> maybe two months, but I suspect it's been happening much longer than that.
> h3. Problem
> When running in a secure cluster, sometimes the ZK EventThread will get stuck
> on a permissions update and not be able to process new notifications. This
> happens to also block flush and snapshot, which is how we found it.
> h3. Analysis
> The main smoking gun is seeing this in repeated jstacks:
> {noformat}
> "main-EventThread" #43 daemon prio=5 os_prio=0 tid=0x7f0b92644000
> nid=0x6e69 waiting on condition [0x7f0b6730f000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.nodeChildrenChanged(ZKPermissionWatcher.java:191)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:503)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> {noformat}
> That sleep is a 20ms sleep in an {{AtomicReference.compareAndSet}} loop - but
> it never gets past the condition.
> {code}
> while (!nodes.compareAndSet(null, nodeList)) {
> try {
> Thread.sleep(20);
> } catch (InterruptedException e) {
> LOG.warn("Interrupted while setting node list", e);
> Thread.currentThread().interrupt();
> }
> }
> {code}
> The warning never shows up in the logs, it just keeps looping and looping.
> The last relevant line from the watcher in logs is:
> {noformat}
> 2017-08-17 21:25:12,379 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher:
> regionserver:22101-0x15df38884c80024, quorum=zk1:2181,zk2:2181,zk3:2181,
> baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged,
> state=SyncConnected, path=/hbase/acl
> {noformat}
> Which makes sense, because the code snippet is from permission watcher's
> {{nodeChildrenChanged}} handler.
> The separate thread introduced in HBASE-14370 is present, but not doing
> anything. And this event hasn't gotten to the part where it splits off into a
> thread:
> {noformat}
> "zk-permission-watcher4-thread-1" #160 daemon prio=5 os_prio=0
> tid=0x01750800 nid=0x6fd9 waiting on condition [0x7f0b5dce5000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007436ecea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> h3. Solutions
> There's a few approaches we can take to fix this, I think they are all
> complimentary. It might be useful to file subtasks or new issues for some of
> the solutions if they are longer term.
> # Move flush and snapshot to ProcedureV2. This makes my proximate problem go
> away, but it's only relevant to branch-2 and master, and doesn't fix anything
> on branch-1. Also, Permissions updates would still get stuck, preventing
> future permissions updates. I think this is important long term for the
> robustness of the system, but not a viable short term fix.
> # Add an Executor to ZookeeperWatcher and
[jira] [Commented] (HBASE-18628) ZKPermissionWatcher blocks all ZK notifications
[
https://issues.apache.org/jira/browse/HBASE-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133680#comment-16133680
]
Sean Busbey commented on HBASE-18628:
-
{code}
195 if (childrenChangedFuture.isDone()) {
196 // task finished between our check and attempted cancel,
this is fine.
197 } else {
198 LOG.warn("Could not cancel processing node children
changed event, " +
199 "please file a JIRA and attach logs if possible.");
200 }
{code}
Negate the check so there isn't an empty block.
> ZKPermissionWatcher blocks all ZK notifications
> ---
>
> Key: HBASE-18628
> URL: https://issues.apache.org/jira/browse/HBASE-18628
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Attachments: HBASE-18628.patch, HBASE-18628.v2.patch, jstack
>
>
> Buckle up folks, we're going for a ride here. I've seeing this on a branch-2
> based build, but I think the problem will affect branch-1 as well. I'm not
> able to easily reproduce the issue, but it will usually come up within an
> hour on a given cluster that I have, at which point the problem persists
> until an RS restart. I've been seeing the problem and paying attention for
> maybe two months, but I suspect it's been happening much longer than that.
> h3. Problem
> When running in a secure cluster, sometimes the ZK EventThread will get stuck
> on a permissions update and not be able to process new notifications. This
> happens to also block flush and snapshot, which is how we found it.
> h3. Analysis
> The main smoking gun is seeing this in repeated jstacks:
> {noformat}
> "main-EventThread" #43 daemon prio=5 os_prio=0 tid=0x7f0b92644000
> nid=0x6e69 waiting on condition [0x7f0b6730f000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.nodeChildrenChanged(ZKPermissionWatcher.java:191)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:503)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> {noformat}
> That sleep is a 20ms sleep in an {{AtomicReference.compareAndSet}} loop - but
> it never gets past the condition.
> {code}
> while (!nodes.compareAndSet(null, nodeList)) {
> try {
> Thread.sleep(20);
> } catch (InterruptedException e) {
> LOG.warn("Interrupted while setting node list", e);
> Thread.currentThread().interrupt();
> }
> }
> {code}
> The warning never shows up in the logs, it just keeps looping and looping.
> The last relevant line from the watcher in logs is:
> {noformat}
> 2017-08-17 21:25:12,379 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher:
> regionserver:22101-0x15df38884c80024, quorum=zk1:2181,zk2:2181,zk3:2181,
> baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged,
> state=SyncConnected, path=/hbase/acl
> {noformat}
> Which makes sense, because the code snippet is from permission watcher's
> {{nodeChildrenChanged}} handler.
> The separate thread introduced in HBASE-14370 is present, but not doing
> anything. And this event hasn't gotten to the part where it splits off into a
> thread:
> {noformat}
> "zk-permission-watcher4-thread-1" #160 daemon prio=5 os_prio=0
> tid=0x01750800 nid=0x6fd9 waiting on condition [0x7f0b5dce5000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007436ecea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> h3. Solutions
> There's a few approaches we can take to fix this, I think they are all
> complimentary. It might be useful to file subtasks or new issues for some of
> the solutions if they are longer term.
> # Move
[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133679#comment-16133679
]
Hudson commented on HBASE-14498:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #243 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/243/])
HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are
(stack: rev 8b977d77851cb3b63b6eff32277bc182128b8201)
* (edit)
hbase-client/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperWatcher.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO |
[jira] [Commented] (HBASE-18629) Enhance ChaosMonkeyRunner with interruptibility
[
https://issues.apache.org/jira/browse/HBASE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133677#comment-16133677
]
Hadoop QA commented on HBASE-18629:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 56s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
16s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18629 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882675/18629.v2.txt |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 3a83691c872c 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 56f9e1a |
| Default Java | 1.8.0_144 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8165/testReport/ |
| modules | C: hbase-it U: hbase-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8165/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Enhance ChaosMonkeyRunner with interruptibility
> ---
>
> Key: HBASE-18629
> URL: https://issues.apache.org/jira/browse/HBASE-18629
> Project: HBase
> Issue
[jira] [Updated] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18606:
---
Attachment: 18606.v2.txt
Tried this variant which doesn't work.
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18606.v1.txt, 18606.v2.txt
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18614) Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133661#comment-16133661
]
Hadoop QA commented on HBASE-18614:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
3s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 24m 28s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 76m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.hfile.TestBlockCacheReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18614 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882663/HBASE-18614.PATCH |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 1cbd0d67a167 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 56f9e1a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/8161/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8161/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-18629) Enhance ChaosMonkeyRunner with interruptibility
[
https://issues.apache.org/jira/browse/HBASE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133653#comment-16133653
]
Hadoop QA commented on HBASE-18629:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.13.1 Server=1.13.1 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18629 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882667/18629.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 2c63a9d8ab1e 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 56f9e1a |
| Default Java | 1.8.0_144 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8164/testReport/ |
| modules | C: hbase-it U: hbase-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8164/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Enhance ChaosMonkeyRunner with interruptibility
> ---
>
> Key: HBASE-18629
> URL: https://issues.apache.org/jira/browse/HBASE-18629
> Project: HBase
> Issue
[jira] [Commented] (HBASE-18572) Delete can't remove the cells which have no visibility label
[
https://issues.apache.org/jira/browse/HBASE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133646#comment-16133646
]
Hudson commented on HBASE-18572:
FAILURE: Integrated in Jenkins build HBase-1.5 #17 (See
[https://builds.apache.org/job/HBase-1.5/17/])
HBASE-18572 Delete can't remove the cells which have no visibility label
(chia7712: rev 10d65ee37f82c079ffe7f12cf7420d2ed7145f15)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
> Delete can't remove the cells which have no visibility label
>
>
> Key: HBASE-18572
> URL: https://issues.apache.org/jira/browse/HBASE-18572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18572.branch-1.v0.patch,
> HBASE-18572.branch-1.v0.patch, HBASE-18572.branch-1.v0.patch,
> HBASE-18572.ut.patch, HBASE-18572.v0.patch, HBASE-18572.v0.patch,
> HBASE-18572.v1.patch
>
>
> {code:title=DefaultVisibilityLabelServiceImpl.java}
> @Override
> public boolean matchVisibility(List putVisTags, Byte putTagsFormat,
> List deleteVisTags,
> Byte deleteTagsFormat) throws IOException {
> if ((deleteTagsFormat != null && deleteTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)
> && (putTagsFormat == null || putTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)) {
> if (putVisTags.isEmpty()) {
> // Early out if there are no tags in the cell
> return false;
> }
> if (putTagsFormat == null) {
> return matchUnSortedVisibilityTags(putVisTags, deleteVisTags);
> } else {
> return matchOrdinalSortedVisibilityTags(putVisTags, deleteVisTags);
> }
> }
> throw new IOException("Unexpected tag format passed for comparison,
> deleteTagsFormat : "
> + deleteTagsFormat + ", putTagsFormat : " + putTagsFormat);
> }
> {code}
> If the putVisTags is empty, it will return the false directly. Hence, the
> Delete#addColumn can't "see" the cell as well as deleting it. The
> Delete#addColumns, by contrast, doesn't invoke the
> cp#prePrepareTimeStampForDeleteVersion so it can remove the cell.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18471) The DeleteFamily cell is skipped when StoreScanner seeks to next column
[
https://issues.apache.org/jira/browse/HBASE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133647#comment-16133647
]
Hudson commented on HBASE-18471:
FAILURE: Integrated in Jenkins build HBase-1.5 #17 (See
[https://builds.apache.org/job/HBase-1.5/17/])
HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to
(chia7712: rev f61f09cdf27b34d3549ec450bbffba3487445d74)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/ScanQueryMatcher.java
> The DeleteFamily cell is skipped when StoreScanner seeks to next column
> ---
>
> Key: HBASE-18471
> URL: https://issues.apache.org/jira/browse/HBASE-18471
> Project: HBase
> Issue Type: Bug
> Components: Deletes, hbase, scan
>Affects Versions: 3.0.0, 1.3.0, 1.3.1, 2.0.0-alpha-1
>Reporter: Thomas Martens
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18471.branch-1.2.v0.patch, HBASE-18471.v0.patch,
> HBASE-18471.v1.patch, HBaseDmlTest.java
>
>
> The qualifier of a deleted row (with keep deleted cells true) re-appears
> after re-inserting the same row multiple times (with different timestamp)
> with an empty qualifier.
> Scenario:
> # Put row with family and qualifier (timestamp 1).
> # Delete entire row (timestamp 2).
> # Put same row again with family without qualifier (timestamp 3).
> A scan (latest version) returns the row with family without qualifier,
> version 3 (which is correct).
> # Put the same row again with family without qualifier (timestamp 4).
> A scan (latest version) returns multiple rows:
> * the row with family without qualifier, version 4 (which is correct).
> * the row with family with qualifier, version 1 (which is wrong).
> There is a test scenario attached.
> output:
> 13:42:53,952 [main] client.HBaseAdmin - Started disable of test_dml
> 13:42:55,801 [main] client.HBaseAdmin - Disabled test_dml
> 13:42:57,256 [main] client.HBaseAdmin - Deleted test_dml
> 13:42:58,592 [main] client.HBaseAdmin - Created test_dml
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'myQualifier' with
> timestamp: '1'
> Scan printout =>
> Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier: 'myQualifier',
> Value: 'myValue'
> Delete row: 'myRow'
> Scan printout =>
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '3'
> Scan printout =>
> Row: 'myRow', Timestamp: '3', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> Put row: 'myRow' with family: 'myFamily' with qualifier: 'null' with
> timestamp: '4'
> Scan printout =>
> Row: 'myRow', Timestamp: '4', Family: 'myFamily', Qualifier: '', Value:
> 'myValue'
> {color:red}Row: 'myRow', Timestamp: '1', Family: 'myFamily', Qualifier:
> 'myQualifier', Value: 'myValue'{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18631) Allow configuration of ChaosMonkey properties via hbase-site
Josh Elser created HBASE-18631: -- Summary: Allow configuration of ChaosMonkey properties via hbase-site Key: HBASE-18631 URL: https://issues.apache.org/jira/browse/HBASE-18631 Project: HBase Issue Type: Improvement Components: integration tests Reporter: Josh Elser Assignee: Josh Elser Priority: Minor Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7, 1.1.13 I noticed in some internal test automation that the code was attempting to configure some of the chaos-monkey actions via hbase-site.xml, but these weren't taking effect. After reading the code, I found that these can only be specified via a special properties file otherwise specified. It would be nice to also allow configuration via hbase-site. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated HBASE-18532: -- Attachment: HBASE-18532.PATCH Attaching patch for master. > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: COMBINED-STATS.PNG, HBASE-18532-1.1.2.PATCH, > HBASE-18532.PATCH, L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated HBASE-18532: -- Status: Patch Available (was: Open) > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: COMBINED-STATS.PNG, HBASE-18532-1.1.2.PATCH, > HBASE-18532.PATCH, L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18572) Delete can't remove the cells which have no visibility label
[
https://issues.apache.org/jira/browse/HBASE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133643#comment-16133643
]
Hudson commented on HBASE-18572:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #192 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/192/])
HBASE-18572 Delete can't remove the cells which have no visibility label
(chia7712: rev d810b8eb5e0bad174e80f861547066132058d521)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
> Delete can't remove the cells which have no visibility label
>
>
> Key: HBASE-18572
> URL: https://issues.apache.org/jira/browse/HBASE-18572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18572.branch-1.v0.patch,
> HBASE-18572.branch-1.v0.patch, HBASE-18572.branch-1.v0.patch,
> HBASE-18572.ut.patch, HBASE-18572.v0.patch, HBASE-18572.v0.patch,
> HBASE-18572.v1.patch
>
>
> {code:title=DefaultVisibilityLabelServiceImpl.java}
> @Override
> public boolean matchVisibility(List putVisTags, Byte putTagsFormat,
> List deleteVisTags,
> Byte deleteTagsFormat) throws IOException {
> if ((deleteTagsFormat != null && deleteTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)
> && (putTagsFormat == null || putTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)) {
> if (putVisTags.isEmpty()) {
> // Early out if there are no tags in the cell
> return false;
> }
> if (putTagsFormat == null) {
> return matchUnSortedVisibilityTags(putVisTags, deleteVisTags);
> } else {
> return matchOrdinalSortedVisibilityTags(putVisTags, deleteVisTags);
> }
> }
> throw new IOException("Unexpected tag format passed for comparison,
> deleteTagsFormat : "
> + deleteTagsFormat + ", putTagsFormat : " + putTagsFormat);
> }
> {code}
> If the putVisTags is empty, it will return the false directly. Hence, the
> Delete#addColumn can't "see" the cell as well as deleting it. The
> Delete#addColumns, by contrast, doesn't invoke the
> cp#prePrepareTimeStampForDeleteVersion so it can remove the cell.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14498:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to branch-1.3+ Tried cherry-picking to 1.2 but wouldn't go back so
punted on it. Thanks for the patches and perseverance [~pankaj_kumar] This one
was an oldie but a goodie.
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other
[jira] [Commented] (HBASE-18630) Prune dependencies; as is branch-2 has duplicates
[ https://issues.apache.org/jira/browse/HBASE-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133626#comment-16133626 ] Josh Elser commented on HBASE-18630: Thanks for the leg-work, Stack. Watching. > Prune dependencies; as is branch-2 has duplicates > - > > Key: HBASE-18630 > URL: https://issues.apache.org/jira/browse/HBASE-18630 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0-alpha-3 > > > Purge duplicate includes and try to prune back our dependencies (Suggestion > by [~elserj] up on the 2.0.0-alpha2 vote). Just looking at my current issue, > we have vestiges we include even though the root justification has passed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18572) Delete can't remove the cells which have no visibility label
[
https://issues.apache.org/jira/browse/HBASE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133622#comment-16133622
]
Hudson commented on HBASE-18572:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #253 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/253/])
HBASE-18572 Delete can't remove the cells which have no visibility label
(chia7712: rev 883b19363f7697e733e9a40b7771b602cbb9a110)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityScanDeleteTracker.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
> Delete can't remove the cells which have no visibility label
>
>
> Key: HBASE-18572
> URL: https://issues.apache.org/jira/browse/HBASE-18572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18572.branch-1.v0.patch,
> HBASE-18572.branch-1.v0.patch, HBASE-18572.branch-1.v0.patch,
> HBASE-18572.ut.patch, HBASE-18572.v0.patch, HBASE-18572.v0.patch,
> HBASE-18572.v1.patch
>
>
> {code:title=DefaultVisibilityLabelServiceImpl.java}
> @Override
> public boolean matchVisibility(List putVisTags, Byte putTagsFormat,
> List deleteVisTags,
> Byte deleteTagsFormat) throws IOException {
> if ((deleteTagsFormat != null && deleteTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)
> && (putTagsFormat == null || putTagsFormat ==
> SORTED_ORDINAL_SERIALIZATION_FORMAT)) {
> if (putVisTags.isEmpty()) {
> // Early out if there are no tags in the cell
> return false;
> }
> if (putTagsFormat == null) {
> return matchUnSortedVisibilityTags(putVisTags, deleteVisTags);
> } else {
> return matchOrdinalSortedVisibilityTags(putVisTags, deleteVisTags);
> }
> }
> throw new IOException("Unexpected tag format passed for comparison,
> deleteTagsFormat : "
> + deleteTagsFormat + ", putTagsFormat : " + putTagsFormat);
> }
> {code}
> If the putVisTags is empty, it will return the false directly. Hence, the
> Delete#addColumn can't "see" the cell as well as deleting it. The
> Delete#addColumns, by contrast, doesn't invoke the
> cp#prePrepareTimeStampForDeleteVersion so it can remove the cell.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18629) Enhance ChaosMonkeyRunner with interruptibility
[
https://issues.apache.org/jira/browse/HBASE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18629:
---
Attachment: 18629.v2.txt
> Enhance ChaosMonkeyRunner with interruptibility
> ---
>
> Key: HBASE-18629
> URL: https://issues.apache.org/jira/browse/HBASE-18629
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
> Attachments: 18629.v1.txt, 18629.v2.txt
>
>
> Currently ChaosMonkeyRunner performs looping unconditionally:
> {code}
> while (true) {// loop here until got killed
> Thread.sleep(1);
> }
> {code}
> When ChaosMonkeyRunner is invoked programmatically, it is desirable to add
> interruptibility to the runner so that the caller can manage its lifetime.
> Another enhancement is to allow passing the path to hbase-site.xml where
> chaos monkey parameters are specified.
> This is useful when the underlying hbase-site.xml is not on classpath.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18630) Prune dependencies; as is branch-2 has duplicates
stack created HBASE-18630: - Summary: Prune dependencies; as is branch-2 has duplicates Key: HBASE-18630 URL: https://issues.apache.org/jira/browse/HBASE-18630 Project: HBase Issue Type: Task Components: dependencies Reporter: stack Assignee: stack Priority: Blocker Fix For: 2.0.0-alpha-3 Purge duplicate includes and try to prune back our dependencies (Suggestion by [~elserj] up on the 2.0.0-alpha2 vote). Just looking at my current issue, we have vestiges we include even though the root justification has passed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18629) Enhance ChaosMonkeyRunner with interruptibility
[
https://issues.apache.org/jira/browse/HBASE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133613#comment-16133613
]
Ted Yu commented on HBASE-18629:
If you look at target/cached_classpath.txt , you would see that hbase-site.xml
is not in the classpath.
The patch gives more flexibility in passing config parameters.
bq. parsing the arguments manually instead of using an existing library
Configuration object is required argument for ToolRunner.run().
The new argument is parsed before ChaosMonkeyRunner is instantiated where
processOptions() is defined.
The next patch would address the other comments.
> Enhance ChaosMonkeyRunner with interruptibility
> ---
>
> Key: HBASE-18629
> URL: https://issues.apache.org/jira/browse/HBASE-18629
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
> Attachments: 18629.v1.txt
>
>
> Currently ChaosMonkeyRunner performs looping unconditionally:
> {code}
> while (true) {// loop here until got killed
> Thread.sleep(1);
> }
> {code}
> When ChaosMonkeyRunner is invoked programmatically, it is desirable to add
> interruptibility to the runner so that the caller can manage its lifetime.
> Another enhancement is to allow passing the path to hbase-site.xml where
> chaos monkey parameters are specified.
> This is useful when the underlying hbase-site.xml is not on classpath.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18629) Enhance ChaosMonkeyRunner with interruptibility
[
https://issues.apache.org/jira/browse/HBASE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133602#comment-16133602
]
Mike Drob commented on HBASE-18629:
---
Need to make stop boolean volatile if using it as a control flag across threads.
Why do we need to specify the hbase-site.xml? Isn't it easy enough to add it to
the classpath? Maybe it's not.
Instead of doing a manual copy of the args, please use System.arraycopy or
Arrays.copyOfRange
Please add usage message or documentation to the main method for the new
argument.
Why are we parsing the arguments manually instead of using an existing library?
Commons-cli is already in the class, even.
> Enhance ChaosMonkeyRunner with interruptibility
> ---
>
> Key: HBASE-18629
> URL: https://issues.apache.org/jira/browse/HBASE-18629
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
> Attachments: 18629.v1.txt
>
>
> Currently ChaosMonkeyRunner performs looping unconditionally:
> {code}
> while (true) {// loop here until got killed
> Thread.sleep(1);
> }
> {code}
> When ChaosMonkeyRunner is invoked programmatically, it is desirable to add
> interruptibility to the runner so that the caller can manage its lifetime.
> Another enhancement is to allow passing the path to hbase-site.xml where
> chaos monkey parameters are specified.
> This is useful when the underlying hbase-site.xml is not on classpath.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18624) Added support for clearing BlockCache
[ https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133601#comment-16133601 ] Ajay Jadhav commented on HBASE-18624: - [~anoop.hbase]: I think this makes sense even for non-replica clusters. It is a simple utility API to clear the blockcache based on tablename. We found this useful even in our performance testing instead of restarting the HBase cluster every time. Let me know if you have any concerns. [~ram_krish]: In the case of replica(s), all the clusters are pointing to same rootDir. So, whenever compaction runs, the archived files will be cleared by cleaner chore on the primary cluster itself. Having said that, we did provide a mechanism to disable the cleaner during HBase on S3 work to avoid performance overhead during peak times. [~gsbiju]: Even though this is part of the work done for the parent JIRA, it can be useful in other cases too. The umbrella Jira is where we are proposing the new design for read replica clusters i.e. primary HBase cluster which allows write/ read, and replica(s) (which are separate HBase clusters), allowing only reads. The only thing that is common is that they share a common root dir/ S3 bucket for data. > Added support for clearing BlockCache > - > > Key: HBASE-18624 > URL: https://issues.apache.org/jira/browse/HBASE-18624 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.0 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0 > > > Bulk loading the primary HBase cluster triggers a lot of compactions > resulting in archival/ creation > of multiple HFiles. This process will cause a lot of items to become stale in > replica’s BlockCache. > This patch will help users to clear the block cache for a given table by > either using shell or API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18518) Remove jersey1* dependencies from project and jersey1* jars from lib dir
[ https://issues.apache.org/jira/browse/HBASE-18518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133598#comment-16133598 ] Hudson commented on HBASE-18518: FAILURE: Integrated in Jenkins build HBase-2.0 #353 (See [https://builds.apache.org/job/HBase-2.0/353/]) HBASE-18518 Remove jersey1* dependencies form project and jersey1* jars (stack: rev 6f9af9a9245e3354c5859660c3224ed19d58498a) * (edit) pom.xml * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * (edit) hbase-rest/pom.xml > Remove jersey1* dependencies from project and jersey1* jars from lib dir > > > Key: HBASE-18518 > URL: https://issues.apache.org/jira/browse/HBASE-18518 > Project: HBase > Issue Type: Task > Components: dependencies, pom, REST >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Samir Ahmic >Assignee: Samir Ahmic > Labels: cleanup > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18518-master-01.patch, HBASE-18518-master-02.patch > > > Recently i have opened https://issues.apache.org/jira/browse/HBASE-18506 and > it is clear that is caused by mixing jersey1 and jersey2 jars in classpath. > With https://issues.apache.org/jira/browse/HBASE-12894 we have introduced > jersey2 to project, and we also have bunch of transitive dependencies > (mainly from hadoop) on jersey1 which is not happiest situation since jersey1 > and jersey2 under same classpath can case runtime issues as it was case with > rest. > This task will have following steps > * Clean code and replace jersey1 constructs with jersey2 versions(there > should not be much of this) > * Add exclusions for transitive jersey1 dependencies in pom.xml > * Add exclusions in hadoop-two-compat.xml to prevent jersey1 jars in lib dir -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18617) FuzzyRowKeyFilter should not modify the filter pairs
[ https://issues.apache.org/jira/browse/HBASE-18617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133597#comment-16133597 ] Hudson commented on HBASE-18617: FAILURE: Integrated in Jenkins build HBase-2.0 #353 (See [https://builds.apache.org/job/HBase-2.0/353/]) HBASE-18617 FuzzyRowKeyFilter should not modify the filter pairs (tedyu: rev 22ecb7a7b5da72e86951e88510f53efd19a503a0) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterEndToEnd.java > FuzzyRowKeyFilter should not modify the filter pairs > > > Key: HBASE-18617 > URL: https://issues.apache.org/jira/browse/HBASE-18617 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.2 >Reporter: vinisha >Assignee: vinisha >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: 18617.branch-1.patch, > FuzzyRowFilter_HBASE_18617_1.patch, FuzzyRowFilter_HBASE_18617-branch-1.patch > > > Current implementation of FuzzyRowKeyFilter modifies the provided filter > pairs: > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java#L89-L90 > Filter should not change the filter pair. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133594#comment-16133594 ] Amit Patel edited comment on HBASE-18508 at 8/18/17 8:38 PM: - Resubmitted same patch [^HBASE-18508.HBASE-14070.HLC.003.patch] after a fix to previously^^ failing test TestHeapSize#testSizes was pushed to the HBASE-14070.HLC branch. was (Author: amit.patel): Resubmitted same patch [^HBASE-18508.HBASE-14070.HLC.001.patch] after a fix to previously^^ failing test TestHeapSize#testSizes was pushed to the HBASE-14070.HLC branch. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133594#comment-16133594 ] Amit Patel commented on HBASE-18508: Resubmitted same patch [^HBASE-18508.HBASE-14070.HLC.001.patch] after a fix to previously^^ failing test TestHeapSize#testSizes was pushed to the HBASE-14070.HLC branch. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.003.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
