[jira] [Commented] (HBASE-17520) Implement isTableEnabled/Disabled/Available methods
[
https://issues.apache.org/jira/browse/HBASE-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852660#comment-15852660
]
Hadoop QA commented on HBASE-17520:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 15s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 79m 49s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 122m 37s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12850961/HBASE-17520-v3.patch |
| JIRA Issue | HBASE-17520 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux a45802cdbade 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4e77b18 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5573/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5573/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Implement isTableEnabled/Disabled/Available methods
[jira] [Updated] (HBASE-17508) Unify the implementation of small scan and regular scan for sync client
[ https://issues.apache.org/jira/browse/HBASE-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17508: -- Attachment: HBASE-17508-branch-1.patch > Unify the implementation of small scan and regular scan for sync client > --- > > Key: HBASE-17508 > URL: https://issues.apache.org/jira/browse/HBASE-17508 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17508-branch-1.patch, HBASE-17508.patch, > HBASE-17508-v1.patch, HBASE-17508-v2.patch, HBASE-17508-v3.patch, > HBASE-17508-v4.patch, HBASE-17508-v5.patch, HBASE-17508-v6.patch > > > Implement the same logic with HBASE-17045 for sync client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17511) Implement enable/disable table methods
[ https://issues.apache.org/jira/browse/HBASE-17511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17511: --- Status: Patch Available (was: Open) > Implement enable/disable table methods > -- > > Key: HBASE-17511 > URL: https://issues.apache.org/jira/browse/HBASE-17511 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17511-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17511) Implement enable/disable table methods
[ https://issues.apache.org/jira/browse/HBASE-17511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17511: --- Attachment: HBASE-17511-v1.patch > Implement enable/disable table methods > -- > > Key: HBASE-17511 > URL: https://issues.apache.org/jira/browse/HBASE-17511 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17511-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-17230) port HBASE-17112 to 1.3.1
[ https://issues.apache.org/jira/browse/HBASE-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang resolved HBASE-17230. --- Resolution: Fixed > port HBASE-17112 to 1.3.1 > - > > Key: HBASE-17230 > URL: https://issues.apache.org/jira/browse/HBASE-17230 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0 >Reporter: Phil Yang >Assignee: Phil Yang > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17112) Prevent setting timestamp of delta operations the same as previous value's
[ https://issues.apache.org/jira/browse/HBASE-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-17112: -- Fix Version/s: 1.3.1 > Prevent setting timestamp of delta operations the same as previous value's > -- > > Key: HBASE-17112 > URL: https://issues.apache.org/jira/browse/HBASE-17112 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.7, 0.98.23, 1.2.4 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 0.98.24, 1.1.8 > > Attachments: HBASE-17112-branch-1.1-v1.patch, > HBASE-17112-branch-1.1-v1.patch, HBASE-17112-branch-1-v1.patch, > HBASE-17112-branch-1-v1.patch, HBASE-17112-v1.patch, HBASE-17112-v2.patch, > HBASE-17112-v2.patch > > > In delta operations, Increment and Append. We will read current value first > and then write the new whole result into WAL as the type of Put with current > timestamp. If the previous ts is larger than current ts, we will use the > previous ts. > If we have two Puts with same TS, we will ignore the Put with lower sequence > id. It is not friendly with versioning. And for replication we will drop > sequence id while writing to peer cluster so in the slave we don't know what > the order they are being written. If the pushing is disordered, the result > will be wrong. > We can set the new ts to previous+1 if the previous is not less than now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17230) port HBASE-17112 to 1.3.1
[ https://issues.apache.org/jira/browse/HBASE-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-17230: -- Fix Version/s: (was: 1.3.1) > port HBASE-17112 to 1.3.1 > - > > Key: HBASE-17230 > URL: https://issues.apache.org/jira/browse/HBASE-17230 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0 >Reporter: Phil Yang >Assignee: Phil Yang > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16886) hbase-client: scanner with reversed=true and small=true gets no result
[ https://issues.apache.org/jira/browse/HBASE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16886: -- Fix Version/s: 1.3.1 > hbase-client: scanner with reversed=true and small=true gets no result > -- > > Key: HBASE-16886 > URL: https://issues.apache.org/jira/browse/HBASE-16886 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.3, 1.1.7, 0.98.23 >Reporter: huzheng >Assignee: huzheng > Labels: patch > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.5, 1.1.8 > > Attachments: 16886.addendum, 16886.v4.branch-1.patch, > HBASE-16886.v0.patch, HBASE-16886.v1.patch, HBASE-16886.v2.patch, > HBASE-16886.v3.patch, HBASE-16886.v4.0.98.patch, > HBASE-16886.v4.branch-1.patch, HBASE-16886.v4.branch-1.patch, > HBASE-16886.v4.branch-1.patch, HBASE-16886.v4.master.patch, > TestReversedSmallScan.java > > > Assume HBase have four regions (-oo, b), [b, c), [c, d), [d,+oo) , and all > rowKeys are located in region [d, +oo). using a Reversed Small Scanner will > get no result. > Attached file show this failed test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-17236) Port HBASE-16886 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-17236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang resolved HBASE-17236. --- Resolution: Fixed > Port HBASE-16886 to branch-1.3 > -- > > Key: HBASE-17236 > URL: https://issues.apache.org/jira/browse/HBASE-17236 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17389) Convert all internal usages from ReplicationAdmin to Admin
[
https://issues.apache.org/jira/browse/HBASE-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852619#comment-15852619
]
Hadoop QA commented on HBASE-17389:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 36s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 30s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 18s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 100m 10s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m 20s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 157m 50s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.6 Server=1.12.6 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12850957/HBASE-17389-v2.patch |
| JIRA Issue | HBASE-17389 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile rubocop ruby_lint |
| uname | Linux f848ec1dc66e 3.13.0-100-generic #147-Ubuntu SMP Tue Oct 18
16:48:51 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master /
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852621#comment-15852621
]
Hadoop QA commented on HBASE-14123:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 8s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 23 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 43s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
25s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 10s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 17s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
5s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 69 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 3s
{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 57s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 3m
33s {color} | {color:green} the patch passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 1s
{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 54s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m
[jira] [Commented] (HBASE-15995) Separate replication WAL reading from shipping
[ https://issues.apache.org/jira/browse/HBASE-15995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852622#comment-15852622 ] Phil Yang commented on HBASE-15995: --- Sorry for late, +1 on the latest patch. > Separate replication WAL reading from shipping > -- > > Key: HBASE-15995 > URL: https://issues.apache.org/jira/browse/HBASE-15995 > Project: HBase > Issue Type: Sub-task > Components: Replication >Affects Versions: 2.0.0 >Reporter: Vincent Poon >Assignee: Vincent Poon > Fix For: 2.0.0 > > Attachments: HBASE-15995.master.v1.patch, > HBASE-15995.master.v2.patch, HBASE-15995.master.v3.patch, > HBASE-15995.master.v4.patch, HBASE-15995.master.v6.patch, > replicationV1_100ms_delay.png, replicationV2_100ms_delay.png > > > Currently ReplicationSource reads edits from the WAL and ships them in the > same thread. > By breaking out the reading from the shipping, we can introduce greater > parallelism and lay the foundation for further refactoring to a pipelined, > streaming model. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17572) HMaster: Caught throwable while processing event C_M_MERGE_REGION (UndeclaredThrowableException)
[
https://issues.apache.org/jira/browse/HBASE-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852616#comment-15852616
]
Hudson commented on HBASE-17572:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #91 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/91/])
HBASE-17572 HMaster: Caught throwable while processing event (apurtell: rev
054a0bd9be9e646971a5036ac9ca14bfc05f545e)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
> HMaster: Caught throwable while processing event C_M_MERGE_REGION
> (UndeclaredThrowableException)
>
>
> Key: HBASE-17572
> URL: https://issues.apache.org/jira/browse/HBASE-17572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.3.1
>
> Attachments: HBASE-17572-branch-1.3.patch
>
>
> Running ITBLL 1B rows against branch-1.3 compiled against Hadoop 2.7.3 with
> the noKill monkey policy, I see both masters go down with
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> In ServerManager#sendRegionsMerge we call ProtobufUtil#mergeRegions, which
> does a doAs, and the code within that block invokes
> RSRpcServices#mergeRegions, but is not resilient against
> RegionOpeningException ("region is opening")
> An UndeclaredThrowableException is "thrown by a method invocation on a proxy
> instance if its invocation handler's invoke method throws a checked exception
> (a Throwable that is not assignable to RuntimeException or Error) that is not
> assignable to any of the exception types declared in the throws clause of the
> method that was invoked on the proxy instance and dispatched to the
> invocation handler."
> (http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/UndeclaredThrowableException.html)
>
> {noformat}
> 2017-01-31 07:21:17,495 FATAL [MASTER_TABLE_OPERATIONS-node-1:16000-0]
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.mergeRegions(ProtobufUtil.java:1990)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionsMerge(ServerManager.java:925)
> at
> org.apache.hadoop.hbase.master.handler.DispatchMergingRegionHandler.process(DispatchMergingRegionHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.google.protobuf.ServiceException:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.RegionOpeningException):
> org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region
> IntegrationTestBigLinkedList,|\xFFnk\x1C\x85<[\x1Ef\xFDE\xF9\xAA\xAC\x08,1485846598043.f56ad22121e872777468020c4452a7c7.
> is opening on node-2.cluster,16020,1485822382322
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2964)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1139)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mergeRegions(RSRpcServices.java:1497)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22749)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2355)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:244)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:340)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.mergeRegions(AdminProtos.java:23695)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1993)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1990)
> at java.security.AccessController.doPrivileged(Native Method)
> at
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852609#comment-15852609
]
Guanghao Zhang commented on HBASE-17565:
We can add a method like isNeeded() for CostFunction and implement whether to
ignore in CostFunction's method.
{code}
for (CostFunction c : costFunctions) {
float multiplier = c.getMultiplier();
if (multiplier <= 0 || !c.isNeeded()) {
LOG.debug("ignoring " + c.getClass() + " with multiplier " +
multiplier);
continue;
}
sumMultiplier += multiplier;
total += c.cost() * multiplier;
}
{code}
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852601#comment-15852601
]
Hudson commented on HBASE-17592:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2440 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2440/])
HBASE-17592 Fix rpcTimeout typo (Attila Sasvari) (elserj: rev
4e77b18da2515a14772a456f408ee34376a3c71f)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17592-1.patch
>
>
> Fix typos in timeout related client exception messages in {{IPCUtil.java}}
> and {{RpcConnection.java}} - "rpcTimetout".
> {code}
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> 179:+ (EnvironmentEdgeManager.currentTime() - call.getStartTime()) +
> ", rpcTimetout="
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
> 196: + (EnvironmentEdgeManager.currentTime() -
> call.getStartTime()) + ", rpcTimetout="
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852581#comment-15852581
]
Ted Yu commented on HBASE-17565:
bq. This only effect when there are a very very big multiplier..
Big multiplier does exist. We need to consider the aggregate effect.
bq. Why we need a so big default multiplier for read replica?
The background is that when any two replicas among primary, secondary and
tertiary replicas are on the same server, we lose benefit of read replica when
this server goes down. See the javadoc for RegionReplicaHostCostFunction :
{code}
* A cost function for region replicas. We give a very high cost to hosting
* replicas of the same region in the same host. We do not prevent the case
* though, since if numReplicas > numRegionServers, we still want to keep the
* replica open.
{code}
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17520) Implement isTableEnabled/Disabled/Available methods
[ https://issues.apache.org/jira/browse/HBASE-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17520: --- Attachment: HBASE-17520-v3.patch > Implement isTableEnabled/Disabled/Available methods > --- > > Key: HBASE-17520 > URL: https://issues.apache.org/jira/browse/HBASE-17520 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17520-v1.patch, HBASE-17520-v2.patch, > HBASE-17520-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852551#comment-15852551
]
Guanghao Zhang commented on HBASE-17565:
Now EPSILON = 0.001D, do we need consider the aggregate effect of
cost multiplied by multiplier? This only effect when there are a very very big
multiplier..
bq. The large multiplier for read replica was obtained through trial and error
when developing read replica feature.
As the javadoc of StochasticLoadBalancer said:
{code}
* Every cost function returns a number between 0 and 1 inclusive; where 0
is the lowest cost
* best solution, and 1 is the highest possible cost and the worst solution.
The computed costs are
* scaled by their respective multipliers:
{code}
The bigger multiplier means that the respective cost function have the bigger
weight. Why we need a so big default multiplier for read replica? It means read
replica has biggest weight of all cost function. In our use case, we always
config the sum of all cost function's respective multipliers to 100.
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17530) Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786
[ https://issues.apache.org/jira/browse/HBASE-17530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17530: -- Attachment: HBASE-17530.master.001.patch > Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786 > -- > > Key: HBASE-17530 > URL: https://issues.apache.org/jira/browse/HBASE-17530 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-17530-Readd-TestMergeTableRegionsProcedure-mis.patch, > HBASE-17530.master.001.patch > > > HBASE-16786 mistakenly removed the test TestMergeTableRegionsProcedure. Need > to restore. Filing issue here as critical. Test does basic exercise of merge > functionality. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17350) Fixup of regionserver group-based assignment
[ https://issues.apache.org/jira/browse/HBASE-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852552#comment-15852552 ] stack commented on HBASE-17350: --- Fix whitespace in .005. The test failures can't be related. > Fixup of regionserver group-based assignment > > > Key: HBASE-17350 > URL: https://issues.apache.org/jira/browse/HBASE-17350 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-17350.master.001.patch, > HBASE-17350.master.002.patch, HBASE-17350.master.003.patch, > HBASE-17350.master.004.patch > > > Can we do some fixup on the regionserver group-based assignement before it > makes it into a release? Here are a few items after trying to use it last > night: > + The commands are named inconsistently. Usually it is verb with a rsgroup > suffix but we have get_table_rsgroups and then move_rsgoup_tables. Ditto for > servers. > + In local mode, the regionserver doesn't belong to a group. Shouldn't it? > + Adding a server to a group with the move_rsgroup_tables is non-intuitive, > to me at least (especially given #2 above). > + The error message you get when you run one of these rsgroup commands should > tell you how to set up rsgroups rather than dump out a cryptic exception. > Thats all for now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17350) Fixup of regionserver group-based assignment
[
https://issues.apache.org/jira/browse/HBASE-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852546#comment-15852546
]
Hadoop QA commented on HBASE-17350:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 1s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 1s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 50s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 39s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 22s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 54s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 140m 49s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAsyncAdmin |
| | hadoop.hbase.client.TestBlockEvictionFromClient |
| Timed out junit tests | org.apache.hadoop.hbase.mapreduce.TestRowCounter |
| | org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852542#comment-15852542
]
Ted Yu commented on HBASE-17565:
bq. This is not need to change to 1.01f .
With certain cluster state, total == sumMultiplier, leading to
testNeedBalance() failure.
Since the underlying bug in needsBalance() is fixed, the above change is needed
for test to pass.
bq. when the cost is zero, then we don't need consider the multiplier
As I commented above, we should consider the aggregate effect of cost
multiplied by multiplier, not just the cost itself.
bq. we should make the default multiplier as a small value?
The large multiplier for read replica was obtained through trial and error when
developing read replica feature.
I think we should leave it as is.
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17389) Convert all internal usages from ReplicationAdmin to Admin
[ https://issues.apache.org/jira/browse/HBASE-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17389: --- Attachment: HBASE-17389-v2.patch > Convert all internal usages from ReplicationAdmin to Admin > -- > > Key: HBASE-17389 > URL: https://issues.apache.org/jira/browse/HBASE-17389 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17389-v1.patch, HBASE-17389-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17389) Convert all internal usages from ReplicationAdmin to Admin
[ https://issues.apache.org/jira/browse/HBASE-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17389: --- Status: Patch Available (was: Open) > Convert all internal usages from ReplicationAdmin to Admin > -- > > Key: HBASE-17389 > URL: https://issues.apache.org/jira/browse/HBASE-17389 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17389-v1.patch, HBASE-17389-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852540#comment-15852540
]
Hadoop QA commented on HBASE-17187:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m 52s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 17s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 79m 52s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 134m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12850943/hbase-17187_v2.patch |
| JIRA Issue | HBASE-17187 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 775e877fb312 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4e77b18 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5570/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5570/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> DoNotRetryExceptions from coprocessors should bubble
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852537#comment-15852537
]
Guanghao Zhang commented on HBASE-17565:
I thought the right solution is when the cost is zero, then we don't need
consider the multiplier. And maybe we should make the default multiplier as a
small value?
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852535#comment-15852535
]
Guanghao Zhang commented on HBASE-17565:
bq. conf.setFloat("hbase.master.balancer.stochastic.minCostNeedBalance", 1.0f);
This is not need to change to 1.01f.
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17261) Balancer makes no sense on tip of branch-1: says balanced when not
[
https://issues.apache.org/jira/browse/HBASE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852534#comment-15852534
]
Guanghao Zhang commented on HBASE-17261:
bq. We can first make the default behavior same with before HBASE-15529.
This can be tuned along with other balancer configs. So we can resolve this by
HBASE-17565.
> Balancer makes no sense on tip of branch-1: says balanced when not
> --
>
> Key: HBASE-17261
> URL: https://issues.apache.org/jira/browse/HBASE-17261
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Guanghao Zhang
> Attachments: HBASE-17261.patch
>
>
> Running ITBLL on tip of branch-1, I see this in log when I try to balance:
> {code}
> 2016-12-05 16:42:21,031 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=46,queue=1,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 525.2547686174673|
> , sum multiplier is 111087.0 min cost which need balance is 0.05
> {code}
> Its some old nonsense.
> Does this every time I balance. Can't even force a balance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17261) Balancer makes no sense on tip of branch-1: says balanced when not
[
https://issues.apache.org/jira/browse/HBASE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-17261:
---
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Duplicate with HBASE-17565. Let's focus on HBASE-17565.
> Balancer makes no sense on tip of branch-1: says balanced when not
> --
>
> Key: HBASE-17261
> URL: https://issues.apache.org/jira/browse/HBASE-17261
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Guanghao Zhang
> Attachments: HBASE-17261.patch
>
>
> Running ITBLL on tip of branch-1, I see this in log when I try to balance:
> {code}
> 2016-12-05 16:42:21,031 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=46,queue=1,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 525.2547686174673|
> , sum multiplier is 111087.0 min cost which need balance is 0.05
> {code}
> Its some old nonsense.
> Does this every time I balance. Can't even force a balance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17572) HMaster: Caught throwable while processing event C_M_MERGE_REGION (UndeclaredThrowableException)
[
https://issues.apache.org/jira/browse/HBASE-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852529#comment-15852529
]
Hudson commented on HBASE-17572:
FAILURE: Integrated in Jenkins build HBase-1.4 #611 (See
[https://builds.apache.org/job/HBase-1.4/611/])
HBASE-17572 HMaster: Caught throwable while processing event (apurtell: rev
fb1239766d08ef7b888261d25ecfd42e95566081)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
> HMaster: Caught throwable while processing event C_M_MERGE_REGION
> (UndeclaredThrowableException)
>
>
> Key: HBASE-17572
> URL: https://issues.apache.org/jira/browse/HBASE-17572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.3.1
>
> Attachments: HBASE-17572-branch-1.3.patch
>
>
> Running ITBLL 1B rows against branch-1.3 compiled against Hadoop 2.7.3 with
> the noKill monkey policy, I see both masters go down with
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> In ServerManager#sendRegionsMerge we call ProtobufUtil#mergeRegions, which
> does a doAs, and the code within that block invokes
> RSRpcServices#mergeRegions, but is not resilient against
> RegionOpeningException ("region is opening")
> An UndeclaredThrowableException is "thrown by a method invocation on a proxy
> instance if its invocation handler's invoke method throws a checked exception
> (a Throwable that is not assignable to RuntimeException or Error) that is not
> assignable to any of the exception types declared in the throws clause of the
> method that was invoked on the proxy instance and dispatched to the
> invocation handler."
> (http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/UndeclaredThrowableException.html)
>
> {noformat}
> 2017-01-31 07:21:17,495 FATAL [MASTER_TABLE_OPERATIONS-node-1:16000-0]
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.mergeRegions(ProtobufUtil.java:1990)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionsMerge(ServerManager.java:925)
> at
> org.apache.hadoop.hbase.master.handler.DispatchMergingRegionHandler.process(DispatchMergingRegionHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.google.protobuf.ServiceException:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.RegionOpeningException):
> org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region
> IntegrationTestBigLinkedList,|\xFFnk\x1C\x85<[\x1Ef\xFDE\xF9\xAA\xAC\x08,1485846598043.f56ad22121e872777468020c4452a7c7.
> is opening on node-2.cluster,16020,1485822382322
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2964)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1139)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mergeRegions(RSRpcServices.java:1497)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22749)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2355)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:244)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:340)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.mergeRegions(AdminProtos.java:23695)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1993)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1990)
> at java.security.AccessController.doPrivileged(Native Method)
> at
[jira] [Commented] (HBASE-17508) Unify the implementation of small scan and regular scan for sync client
[ https://issues.apache.org/jira/browse/HBASE-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852520#comment-15852520 ] Duo Zhang commented on HBASE-17508: --- Filed HBASE-17595 to track it. [~stack]. Let me prepare patch for branch-1. > Unify the implementation of small scan and regular scan for sync client > --- > > Key: HBASE-17508 > URL: https://issues.apache.org/jira/browse/HBASE-17508 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17508.patch, HBASE-17508-v1.patch, > HBASE-17508-v2.patch, HBASE-17508-v3.patch, HBASE-17508-v4.patch, > HBASE-17508-v5.patch, HBASE-17508-v6.patch > > > Implement the same logic with HBASE-17045 for sync client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17595) Add partial result for small/limited scan
Duo Zhang created HBASE-17595: - Summary: Add partial result for small/limited scan Key: HBASE-17595 URL: https://issues.apache.org/jira/browse/HBASE-17595 Project: HBase Issue Type: Sub-task Components: asyncclient, Client, scan Affects Versions: 2.0.0, 1.4.0 Reporter: Duo Zhang Priority: Blocker Fix For: 2.0.0, 1.4.0 The partial result support is marked as a 'TODO' when implementing HBASE-17045. And when implementing HBASE-17508, we found that if we make small scan share the same logic with general scan, the scan request other than open scanner will not have the small flag so the server may return partial result to the client and cause some strange behavior. It is solved by modifying the logic at server side, but this means the 1.4.x client is not safe to contact with earlier 1.x server. So we'd better address the problem at client side. Marked as blocker as this issue should be finished before any 2.x and 1.4.x releases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17508) Unify the implementation of small scan and regular scan for sync client
[
https://issues.apache.org/jira/browse/HBASE-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852515#comment-15852515
]
Duo Zhang commented on HBASE-17508:
---
{quote}
Only concern is the follow-on JIRA that will ensure we maintain compatibility.
{quote}
You mean the 'allowPartial' for small and limited scan? Let me file a issue to
track it.
> Unify the implementation of small scan and regular scan for sync client
> ---
>
> Key: HBASE-17508
> URL: https://issues.apache.org/jira/browse/HBASE-17508
> Project: HBase
> Issue Type: Sub-task
> Components: Client, scan
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-17508.patch, HBASE-17508-v1.patch,
> HBASE-17508-v2.patch, HBASE-17508-v3.patch, HBASE-17508-v4.patch,
> HBASE-17508-v5.patch, HBASE-17508-v6.patch
>
>
> Implement the same logic with HBASE-17045 for sync client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17278) [C++] Cell Scanner and KeyValueCodec for encoding cells in RPC
[ https://issues.apache.org/jira/browse/HBASE-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-17278: -- Summary: [C++] Cell Scanner and KeyValueCodec for encoding cells in RPC (was: [C++] Cell Scanner Implementation) > [C++] Cell Scanner and KeyValueCodec for encoding cells in RPC > -- > > Key: HBASE-17278 > URL: https://issues.apache.org/jira/browse/HBASE-17278 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17278.HBASE-14850.v1.patch, > HBASE-17278.HBASE-14850.v2.patch, HBASE-17278.HBASE-14850.v3.patch, > HBASE-17278.HBASE-14850.v4.patch, hbase-17278_v5.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17278) [C++] Cell Scanner Implementation
[ https://issues.apache.org/jira/browse/HBASE-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-17278: -- Summary: [C++] Cell Scanner Implementation (was: [C++] Cell Scanner Implementation to be used by ResultScanner) > [C++] Cell Scanner Implementation > - > > Key: HBASE-17278 > URL: https://issues.apache.org/jira/browse/HBASE-17278 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17278.HBASE-14850.v1.patch, > HBASE-17278.HBASE-14850.v2.patch, HBASE-17278.HBASE-14850.v3.patch, > HBASE-17278.HBASE-14850.v4.patch, hbase-17278_v5.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17572) HMaster: Caught throwable while processing event C_M_MERGE_REGION (UndeclaredThrowableException)
[
https://issues.apache.org/jira/browse/HBASE-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852490#comment-15852490
]
Hudson commented on HBASE-17572:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #104 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/104/])
HBASE-17572 HMaster: Caught throwable while processing event (apurtell: rev
054a0bd9be9e646971a5036ac9ca14bfc05f545e)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
> HMaster: Caught throwable while processing event C_M_MERGE_REGION
> (UndeclaredThrowableException)
>
>
> Key: HBASE-17572
> URL: https://issues.apache.org/jira/browse/HBASE-17572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.3.1
>
> Attachments: HBASE-17572-branch-1.3.patch
>
>
> Running ITBLL 1B rows against branch-1.3 compiled against Hadoop 2.7.3 with
> the noKill monkey policy, I see both masters go down with
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> In ServerManager#sendRegionsMerge we call ProtobufUtil#mergeRegions, which
> does a doAs, and the code within that block invokes
> RSRpcServices#mergeRegions, but is not resilient against
> RegionOpeningException ("region is opening")
> An UndeclaredThrowableException is "thrown by a method invocation on a proxy
> instance if its invocation handler's invoke method throws a checked exception
> (a Throwable that is not assignable to RuntimeException or Error) that is not
> assignable to any of the exception types declared in the throws clause of the
> method that was invoked on the proxy instance and dispatched to the
> invocation handler."
> (http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/UndeclaredThrowableException.html)
>
> {noformat}
> 2017-01-31 07:21:17,495 FATAL [MASTER_TABLE_OPERATIONS-node-1:16000-0]
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.mergeRegions(ProtobufUtil.java:1990)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionsMerge(ServerManager.java:925)
> at
> org.apache.hadoop.hbase.master.handler.DispatchMergingRegionHandler.process(DispatchMergingRegionHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.google.protobuf.ServiceException:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.RegionOpeningException):
> org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region
> IntegrationTestBigLinkedList,|\xFFnk\x1C\x85<[\x1Ef\xFDE\xF9\xAA\xAC\x08,1485846598043.f56ad22121e872777468020c4452a7c7.
> is opening on node-2.cluster,16020,1485822382322
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2964)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1139)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mergeRegions(RSRpcServices.java:1497)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22749)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2355)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:244)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:340)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.mergeRegions(AdminProtos.java:23695)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1993)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1990)
> at java.security.AccessController.doPrivileged(Native Method)
> at
[jira] [Commented] (HBASE-17261) Balancer makes no sense on tip of branch-1: says balanced when not
[
https://issues.apache.org/jira/browse/HBASE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852488#comment-15852488
]
Guanghao Zhang commented on HBASE-17261:
bq. I looked at HBASE-15529 but didn't see release note on which multiplier
config to change when minCostNeedBalance is set to a non-zero value.
The problem is default multiplier is too big. So we need change the multiplier
config when use minCostNeedBalance. I thought this config should only use for
power user. We can first make the default behavior same with before
HBASE-15529. Then focus the probelm in HBASE-17565. Thanks.
> Balancer makes no sense on tip of branch-1: says balanced when not
> --
>
> Key: HBASE-17261
> URL: https://issues.apache.org/jira/browse/HBASE-17261
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Guanghao Zhang
> Attachments: HBASE-17261.patch
>
>
> Running ITBLL on tip of branch-1, I see this in log when I try to balance:
> {code}
> 2016-12-05 16:42:21,031 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=46,queue=1,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 525.2547686174673|
> , sum multiplier is 111087.0 min cost which need balance is 0.05
> {code}
> Its some old nonsense.
> Does this every time I balance. Can't even force a balance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17275) Assign timeout cause region unassign forever
[
https://issues.apache.org/jira/browse/HBASE-17275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852477#comment-15852477
]
Ted Yu commented on HBASE-17275:
The patch looks good to me.
[~syuanjiang]:
What do you think ?
> Assign timeout cause region unassign forever
>
>
> Key: HBASE-17275
> URL: https://issues.apache.org/jira/browse/HBASE-17275
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 1.2.3, 1.1.7
>Reporter: Allan Yang
>Assignee: Allan Yang
> Attachments: HBASE-17275-branch-1.patch
>
>
> This is a real cased happened in my test cluster.
> I have more 8000 regions to assign when I restart a cluster, but I only
> started one regionserver. That means master need to assign these 8000 regions
> to a single server(I know it is not right, but just for testing).
> The rs recevied the open region rpc and began to open regions. But the due to
> the hugh number of regions, , master timeout the rpc call(but actually some
> region had already opened) after 1 mins, as you can see from log 1.
> {noformat}
> 1. 2016-11-22 10:17:32,285 INFO [example.org:30001.activeMasterManager]
> master.AssignmentManager: Unable to communicate with
> example.org,30003,1479780976834 in order to assign regions,
> java.io.IOException: Call to /example.org:30003 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1, waitTime=60001,
> operationTimeout=6 expired.
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1338)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:290)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.openRegion(AdminProtos.java:30177)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:1000)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1719)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2828)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2775)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2876)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:646)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:493)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:796)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:188)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1711)
> at java.lang.Thread.run(Thread.java:756)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1,
> waitTime=60001, operationTimeout=6 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:81)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1246)
> ... 14 more
> {noformat}
> for the region 7e9aee32eb98a6fc9d503b99fc5f9615(like many others), after
> timeout, master use a pool to re-assign them, as in 2
> {noformat}
> 2. 2016-11-22 10:17:32,303 DEBUG [AM.-pool1-t26] master.AssignmentManager:
> Force region state offline {7e9aee32eb98a6fc9d503b99fc5f9615
> state=PENDING_OPEN, ts=1479780992078, server=example.org,30003,1479780976834}
>
> {noformat}
> But, this region was actually opened on the rs, but (maybe) due to the hugh
> pressure, the OPENED zk event recevied by master , as you can tell from 3,
> "which is more than 15 seconds late"
> {noformat}
> 3. 2016-11-22 10:17:32,304 DEBUG [AM.ZK.Worker-pool2-t3]
> master.AssignmentManager: Handling RS_ZK_REGION_OPENED,
> server=example.org,30003,1479780976834,
> region=7e9aee32eb98a6fc9d503b99fc5f9615, which is more than 15 seconds late,
> current_state={7e9aee32eb98a6fc9d503b99fc5f9615 state=PENDING_OPEN,
> ts=1479780992078, server=example.org,30003,1479780976834}
> {noformat}
> In the meantime, master still try to re-assign this region in another thread.
> Master first close this region in case of multi assign, then change the state
> of this region change from PENDING_OPEN >OFFLINE>PENDING_OPEN. Its RIT node
> in zk was also transitioned to OFFLINE, as in 4,5,6,7
> {noformat}
> 4. 2016-11-22 10:17:32,321 DEBUG [AM.-pool1-t26] master.AssignmentManager:
>
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852476#comment-15852476
]
Guanghao Zhang commented on HBASE-17565:
bq. The constant, 1.0f in testNeedBalance(), was introduced by:
It was used to test needBalance() method. When set it to 1.0f, then it doesn't
need balance. So assertNull(plans).
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17275) Assign timeout cause region unassign forever
[
https://issues.apache.org/jira/browse/HBASE-17275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852451#comment-15852451
]
Allan Yang commented on HBASE-17275:
{quote}
Isn't there something that needs to be done for "which is more than 15 seconds
late" log ?
{quote}
Since I assign more than 8,000 regions to a single RS. The delay may due to the
huge pressure on Zookeeper server, or the serialization of event thread in the
client side. Yes, it definitely need some investigation.
{quote}
i.e. we need to know whether the late event is for the first or second region
assignment.
{quote}
It is from the first region assignment, as you can tell from the log, when
master recv the zk event, the second region assignment still not send to the RS.
> Assign timeout cause region unassign forever
>
>
> Key: HBASE-17275
> URL: https://issues.apache.org/jira/browse/HBASE-17275
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 1.2.3, 1.1.7
>Reporter: Allan Yang
>Assignee: Allan Yang
> Attachments: HBASE-17275-branch-1.patch
>
>
> This is a real cased happened in my test cluster.
> I have more 8000 regions to assign when I restart a cluster, but I only
> started one regionserver. That means master need to assign these 8000 regions
> to a single server(I know it is not right, but just for testing).
> The rs recevied the open region rpc and began to open regions. But the due to
> the hugh number of regions, , master timeout the rpc call(but actually some
> region had already opened) after 1 mins, as you can see from log 1.
> {noformat}
> 1. 2016-11-22 10:17:32,285 INFO [example.org:30001.activeMasterManager]
> master.AssignmentManager: Unable to communicate with
> example.org,30003,1479780976834 in order to assign regions,
> java.io.IOException: Call to /example.org:30003 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1, waitTime=60001,
> operationTimeout=6 expired.
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1338)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:290)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.openRegion(AdminProtos.java:30177)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:1000)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1719)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2828)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2775)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2876)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:646)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:493)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:796)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:188)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1711)
> at java.lang.Thread.run(Thread.java:756)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1,
> waitTime=60001, operationTimeout=6 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:81)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1246)
> ... 14 more
> {noformat}
> for the region 7e9aee32eb98a6fc9d503b99fc5f9615(like many others), after
> timeout, master use a pool to re-assign them, as in 2
> {noformat}
> 2. 2016-11-22 10:17:32,303 DEBUG [AM.-pool1-t26] master.AssignmentManager:
> Force region state offline {7e9aee32eb98a6fc9d503b99fc5f9615
> state=PENDING_OPEN, ts=1479780992078, server=example.org,30003,1479780976834}
>
> {noformat}
> But, this region was actually opened on the rs, but (maybe) due to the hugh
> pressure, the OPENED zk event recevied by master , as you can tell from 3,
> "which is more than 15 seconds late"
> {noformat}
> 3. 2016-11-22 10:17:32,304 DEBUG [AM.ZK.Worker-pool2-t3]
> master.AssignmentManager: Handling RS_ZK_REGION_OPENED,
> server=example.org,30003,1479780976834,
> region=7e9aee32eb98a6fc9d503b99fc5f9615, which is more than 15 seconds late,
[jira] [Commented] (HBASE-16999) [Master] Inform RegionServers on size quota violations
[
https://issues.apache.org/jira/browse/HBASE-16999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852448#comment-15852448
]
Hadoop QA commented on HBASE-16999:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 49s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
26s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 26s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 44s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 0s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 14s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 31s
{color} | {color:red} The patch causes 306 errors with Hadoop v2.5.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 30s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 0s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 94m 59s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.10.1 Server=1.10.1 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12850929/HBASE-16999.003.HBASE-16961.patch
|
| JIRA Issue | HBASE-16999 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux a5f46aa80f4f
[jira] [Commented] (HBASE-17593) Fix build with hadoop 3 profile
[
https://issues.apache.org/jira/browse/HBASE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852447#comment-15852447
]
Hadoop QA commented on HBASE-17593:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 6s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 55s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
5s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 54s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 27s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 136m 7s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12850907/HBASE-17593.master.001.patch
|
| JIRA Issue | HBASE-17593 |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
xml compile |
| uname | Linux a80fa90bdd94 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4e77b18 |
| Default Java | 1.8.0_121 |
| shellcheck | v0.4.5 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5567/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5567/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5567/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Fix build with hadoop 3 profile
> ---
>
> Key: HBASE-17593
> URL: https://issues.apache.org/jira/browse/HBASE-17593
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17593.master.001.patch
>
>
> {{mvn clean install -DskipTests -Dhadoop-three.version=3.0.0-alpha1
> -Dhadoop.profile=3.0}} fails for me, but passes with
> {{-Dhadoop-three.version=3.0.0-alpha2}}. The failure with alpha1 is
[jira] [Updated] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-17187:
--
Fix Version/s: 1.3.1
1.4.0
2.0.0
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: hbase-17187_v1.patch, hbase-17187_v2.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-17187:
--
Attachment: hbase-17187_v2.patch
Here is v2. Will commit if qa passes.
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch, hbase-17187_v2.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14123: -- Attachment: 14123.master.v56.patch v56. Fixed integration test > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0 > > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt, > 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, > 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt, > 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt, > 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, > 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt, > 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch, > 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch, > 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch, > 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch, > 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v56.patch, > 14123-master.v5.txt, 14123-master.v6.txt, 14123-master.v7.txt, > 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, > HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, > HBASE-14123-v10.patch, HBASE-14123-v11.patch, HBASE-14123-v12.patch, > HBASE-14123-v13.patch, HBASE-14123-v15.patch, HBASE-14123-v16.patch, > HBASE-14123-v1.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, > HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, > HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14123: -- Attachment: (was: 14123.master.v55.patch) > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0 > > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt, > 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, > 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt, > 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt, > 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, > 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt, > 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch, > 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch, > 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch, > 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch, > 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v56.patch, > 14123-master.v5.txt, 14123-master.v6.txt, 14123-master.v7.txt, > 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, > HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, > HBASE-14123-v10.patch, HBASE-14123-v11.patch, HBASE-14123-v12.patch, > HBASE-14123-v13.patch, HBASE-14123-v15.patch, HBASE-14123-v16.patch, > HBASE-14123-v1.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, > HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, > HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17350) Fixup of regionserver group-based assignment
[ https://issues.apache.org/jira/browse/HBASE-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852414#comment-15852414 ] stack commented on HBASE-17350: --- .004 to address Josh Elser review up on rb. > Fixup of regionserver group-based assignment > > > Key: HBASE-17350 > URL: https://issues.apache.org/jira/browse/HBASE-17350 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-17350.master.001.patch, > HBASE-17350.master.002.patch, HBASE-17350.master.003.patch, > HBASE-17350.master.004.patch > > > Can we do some fixup on the regionserver group-based assignement before it > makes it into a release? Here are a few items after trying to use it last > night: > + The commands are named inconsistently. Usually it is verb with a rsgroup > suffix but we have get_table_rsgroups and then move_rsgoup_tables. Ditto for > servers. > + In local mode, the regionserver doesn't belong to a group. Shouldn't it? > + Adding a server to a group with the move_rsgroup_tables is non-intuitive, > to me at least (especially given #2 above). > + The error message you get when you run one of these rsgroup commands should > tell you how to set up rsgroups rather than dump out a cryptic exception. > Thats all for now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17350) Fixup of regionserver group-based assignment
[ https://issues.apache.org/jira/browse/HBASE-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17350: -- Attachment: HBASE-17350.master.004.patch > Fixup of regionserver group-based assignment > > > Key: HBASE-17350 > URL: https://issues.apache.org/jira/browse/HBASE-17350 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-17350.master.001.patch, > HBASE-17350.master.002.patch, HBASE-17350.master.003.patch, > HBASE-17350.master.004.patch > > > Can we do some fixup on the regionserver group-based assignement before it > makes it into a release? Here are a few items after trying to use it last > night: > + The commands are named inconsistently. Usually it is verb with a rsgroup > suffix but we have get_table_rsgroups and then move_rsgoup_tables. Ditto for > servers. > + In local mode, the regionserver doesn't belong to a group. Shouldn't it? > + Adding a server to a group with the move_rsgroup_tables is non-intuitive, > to me at least (especially given #2 above). > + The error message you get when you run one of these rsgroup commands should > tell you how to set up rsgroups rather than dump out a cryptic exception. > Thats all for now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852406#comment-15852406 ] stack commented on HBASE-14123: --- bq. Vladimir, can you please link those issues here so that we can keep track of them. I suggest to also do a little write up for those loosely following what is going on. That'd be great. > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0 > > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt, > 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, > 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt, > 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt, > 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, > 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt, > 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch, > 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch, > 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch, > 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch, > 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v55.patch, > 14123-master.v5.txt, 14123-master.v6.txt, 14123-master.v7.txt, > 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, > HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, > HBASE-14123-v10.patch, HBASE-14123-v11.patch, HBASE-14123-v12.patch, > HBASE-14123-v13.patch, HBASE-14123-v15.patch, HBASE-14123-v16.patch, > HBASE-14123-v1.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, > HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, > HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14123: -- Attachment: 14123.master.v55.patch v55, some small typo fixes > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0 > > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt, > 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, > 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt, > 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt, > 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, > 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt, > 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch, > 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch, > 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch, > 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch, > 14123.master.v52.patch, 14123.master.v54.patch, 14123.master.v55.patch, > 14123-master.v5.txt, 14123-master.v6.txt, 14123-master.v7.txt, > 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, > HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, > HBASE-14123-v10.patch, HBASE-14123-v11.patch, HBASE-14123-v12.patch, > HBASE-14123-v13.patch, HBASE-14123-v15.patch, HBASE-14123-v16.patch, > HBASE-14123-v1.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, > HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, > HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852396#comment-15852396 ] Enis Soztutar commented on HBASE-14123: --- Thanks. I've checked v54 patch. Left two comments. You also have to address the hadoopqa comments above. Some maybe superfluous, but you should still check and leave a comment if so. I'll be +1 for the merge once the RB issues and hadoopqa issues are addressed given that there are issues tracking remaining review comments. Vladimir, can you please link those issues here so that we can keep track of them. I suggest to also do a little write up for those loosely following what is going on. [~stack] / [~tedyu] / others do you want to take a look at the recent patch? > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0 > > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt, > 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, > 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt, > 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt, > 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, > 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt, > 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch, > 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch, > 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch, > 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch, > 14123.master.v52.patch, 14123.master.v54.patch, 14123-master.v5.txt, > 14123-master.v6.txt, 14123-master.v7.txt, 14123-master.v8.txt, > 14123-master.v9.txt, 14123-v14.txt, HBASE-14123-for-7912-v1.patch, > HBASE-14123-for-7912-v6.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch, > HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch, > HBASE-14123-v16.patch, HBASE-14123-v1.patch, HBASE-14123-v2.patch, > HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, > HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17593) Fix build with hadoop 3 profile
[
https://issues.apache.org/jira/browse/HBASE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852387#comment-15852387
]
Ted Yu commented on HBASE-17593:
Ran "mvn clean install -DskipTests -Dhadoop.profile=3.0" with patch which
passed.
+1
> Fix build with hadoop 3 profile
> ---
>
> Key: HBASE-17593
> URL: https://issues.apache.org/jira/browse/HBASE-17593
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17593.master.001.patch
>
>
> {{mvn clean install -DskipTests -Dhadoop-three.version=3.0.0-alpha1
> -Dhadoop.profile=3.0}} fails for me, but passes with
> {{-Dhadoop-three.version=3.0.0-alpha2}}. The failure with alpha1 is given
> below.
> Haven't investigated the failure, maybe we can simply update the hadoop three
> version to alpha2?
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.8:run (generate) on project
> hbase-server: An Ant BuildException has occured: java.lang.NoSuchMethodError:
> javax.servlet.ServletContext.getJspConfigDescriptor()Ljavax/servlet/descriptor/JspConfigDescriptor;
> [ERROR] around Ant part ... outputdir="${generated.sources}/java"
> package="org.apache.hadoop.hbase.generated.master"
> webxml="${build.webapps}/master/WEB-INF/web.xml"/>... @ 17:187 in
> /Users/appy/apache/hbase/hbase-server/target/antrun/build-main.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16999) [Master] Inform RegionServers on size quota violations
[ https://issues.apache.org/jira/browse/HBASE-16999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852383#comment-15852383 ] Ted Yu commented on HBASE-16999: +1, pending QA result. > [Master] Inform RegionServers on size quota violations > -- > > Key: HBASE-16999 > URL: https://issues.apache.org/jira/browse/HBASE-16999 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-16999.002.HBASE-16961.patch, > HBASE-16999.003.HBASE-16961.patch, HBASE-16999-HBASE-16961.001.patch > > > The Master, after computing the total usage across all regionservers, needs > to inform the RegionServers about tables that need to change violation policy > enforcement (enable or disable enforcement). > RPC, ZK, or ProcV2's notification bus are some examples of ways this could be > implemented. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17594) Add supplemental model for jsr311-api
[
https://issues.apache.org/jira/browse/HBASE-17594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852367#comment-15852367
]
Appy commented on HBASE-17594:
--
Hmm, something's not right. If i build with alpha2, {{mvn clean install
-DskipTests -Dhadoop-three.version=3.0.0-alpha2 -Dhadoop.profile=3.0}}, it
doesn't fail. Even though deps tree has jsr311-api, and no license info in
supplemental models.
Am i getting it wrong or is the build wrong? :)
[~busbey]
{noformat}
[INFO] +- org.apache.hadoop:hadoop-common:jar:3.0.0-alpha2:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:3.0.0-alpha2:compile
[INFO] | +- commons-cli:commons-cli:jar:1.3.1:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] | +- xmlenc:xmlenc:jar:0.52:compile
[INFO] | +- commons-net:commons-net:jar:3.1:compile
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] | +- org.eclipse.jetty:jetty-server:jar:9.3.8.v20160314:compile
[INFO] | | +- org.eclipse.jetty:jetty-http:jar:9.3.8.v20160314:compile
[INFO] | | \- org.eclipse.jetty:jetty-io:jar:9.3.8.v20160314:compile
[INFO] | +- org.eclipse.jetty:jetty-util:jar:9.3.8.v20160314:compile
[INFO] | +- org.eclipse.jetty:jetty-servlet:jar:9.3.8.v20160314:compile
[INFO] | | \- org.eclipse.jetty:jetty-security:jar:9.3.8.v20160314:compile
[INFO] | +- org.eclipse.jetty:jetty-webapp:jar:9.3.8.v20160314:compile
[INFO] | | \- org.eclipse.jetty:jetty-xml:jar:9.3.8.v20160314:compile
[INFO] | +- com.sun.jersey:jersey-core:jar:1.19:compile
[INFO] | | \- javax.ws.rs:jsr311-api:jar:1.1.1:compile
{noformat}
> Add supplemental model for jsr311-api
> -
>
> Key: HBASE-17594
> URL: https://issues.apache.org/jira/browse/HBASE-17594
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>
> It's transitive dependency from hadoop 3.
> We don't see any failure for that because for some reason our hadoop3 profile
> check isn't working, I say so because I wasn't able to do mvn install. (see
> HBASE-17593)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: (was: HBASE-17465-HBASE-14850.007.patch) > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: HBASE-17465-HBASE-14850.007.patch > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17465) [C++] implement request retry mechanism over RPC
[
https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852356#comment-15852356
]
Xiaobing Zhou commented on HBASE-17465:
---
posted v7.
# added unit tests
# simplified generic parameters.
# implemented a bunch of mockup class, .e.g. MockRawAsyncTableImpl, which can
be used as a base for RawAsyncTableImpl implementation.
Still need to refine tests by completing mockup functions, e.g.
MockAsyncConnection::GetRegionServerStub, also need to add various
configuration setting to hit retry implementation for verification.
Usage example in tests:
{code}
/* create hbase client connection */
auto conn = std::make_shared();
/* create retry caller factory */
auto tableImpl =
std::make_shared(conn);
AsyncRpcRetryingCallerFactory caller_factory(conn);
/* create request caller builder */
auto p = caller_factory.Single();
/* call with retry to get result */
hbase::Result result =
p->table(std::make_shared(tn))
->row(row)
->rpc_timeout(conn->get_conn_conf()->GetReadRpcTimeoutNs())
->operation_timeout(conn->get_conn_conf()->GetOperationTimeoutNs())
->action(
[=, ](auto& controller, auto& loc, auto& stub) ->
folly::Future {
return tableImpl->GetCall(stub, controller, loc, get);})
->Build()->Call().get();
/* verify result */
ASSERT_TRUE(!result->IsEmpty()) << "Result shouldn't be empty.";
EXPECT_EQ("test2", result->Row());
EXPECT_EQ("value2", *(result->Value("d", "2")));
EXPECT_EQ("value for extra", *(result->Value("d", "extra")));
/* clean */
delete test_util;
{code}
> [C++] implement request retry mechanism over RPC
>
>
> Key: HBASE-17465
> URL: https://issues.apache.org/jira/browse/HBASE-17465
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-17465-HBASE-14850.000.patch,
> HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch,
> HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch,
> HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch,
> HBASE-17465-HBASE-14850.007.patch
>
>
> HBASE-17051 implemented RPC layer. Requests retries will make system
> reliable. This JIRA proposes adding it, which corresponds to similar
> implementation in SingleRequestCallerBuilder (or BatchCallerBuilder,
> ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and
> AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more
> generic, decoupled with HRegionLocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852349#comment-15852349
]
Andrew Purtell commented on HBASE-17187:
Thanks [~enis]. Missed this one while looking at HBASE-17587. Glad we can do a
rebase.
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-16999) [Master] Inform RegionServers on size quota violations
[ https://issues.apache.org/jira/browse/HBASE-16999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-16999: --- Attachment: HBASE-16999.003.HBASE-16961.patch .003 Initial review from Ted. > [Master] Inform RegionServers on size quota violations > -- > > Key: HBASE-16999 > URL: https://issues.apache.org/jira/browse/HBASE-16999 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-16999.002.HBASE-16961.patch, > HBASE-16999.003.HBASE-16961.patch, HBASE-16999-HBASE-16961.001.patch > > > The Master, after computing the total usage across all regionservers, needs > to inform the RegionServers about tables that need to change violation policy > enforcement (enable or disable enforcement). > RPC, ZK, or ProcV2's notification bus are some examples of ways this could be > implemented. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: (was: HBASE-17465-HBASE-14850.007.patch) > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: HBASE-17465-HBASE-14850.007.patch > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17594) Add supplemental model for jsr311-api
Appy created HBASE-17594: Summary: Add supplemental model for jsr311-api Key: HBASE-17594 URL: https://issues.apache.org/jira/browse/HBASE-17594 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy It's transitive dependency from hadoop 3. We don't see any failure for that because for some reason our hadoop3 profile check isn't working, I say so because I wasn't able to do mvn install. (see HBASE-17593) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17568) Expire region reports in the HMaster
[ https://issues.apache.org/jira/browse/HBASE-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17568: --- Attachment: HBASE-17568.001.patch .001 Parking an initial impl for the reviews to catch up.. > Expire region reports in the HMaster > > > Key: HBASE-17568 > URL: https://issues.apache.org/jira/browse/HBASE-17568 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Attachments: HBASE-17568.001.patch > > > (From a TODO) > The RegionServers send reports of sizes from to the Master so the Master can > compute the "size" of each Table. > Presently, the Master has no way to expire these reports. Thus, reports for > tables that are deleted would be retained in memory. Retain the time at which > the report was received by the master, and then use that to determine when to > age it off. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17465) [C++] implement request retry mechanism over RPC
[ https://issues.apache.org/jira/browse/HBASE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-17465: -- Attachment: HBASE-17465-HBASE-14850.007.patch > [C++] implement request retry mechanism over RPC > > > Key: HBASE-17465 > URL: https://issues.apache.org/jira/browse/HBASE-17465 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-17465-HBASE-14850.000.patch, > HBASE-17465-HBASE-14850.001.patch, HBASE-17465-HBASE-14850.002.patch, > HBASE-17465-HBASE-14850.003.patch, HBASE-17465-HBASE-14850.004.patch, > HBASE-17465-HBASE-14850.005.patch, HBASE-17465-HBASE-14850.006.patch, > HBASE-17465-HBASE-14850.007.patch > > > HBASE-17051 implemented RPC layer. Requests retries will make system > reliable. This JIRA proposes adding it, which corresponds to similar > implementation in SingleRequestCallerBuilder (or BatchCallerBuilder, > ScanSingleRegionCallerBuilder, SmallScanCallerBuilder, etc.) and > AsyncSingleRequestRpcRetryingCaller. As a bonus, retry should be more > generic, decoupled with HRegionLocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17593) Fix build with hadoop 3 profile
[
https://issues.apache.org/jira/browse/HBASE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852292#comment-15852292
]
Hadoop QA commented on HBASE-17593:
---
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at
https://builds.apache.org/job/PreCommit-HBASE-Build/5567/console in case of
problems.
> Fix build with hadoop 3 profile
> ---
>
> Key: HBASE-17593
> URL: https://issues.apache.org/jira/browse/HBASE-17593
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17593.master.001.patch
>
>
> {{mvn clean install -DskipTests -Dhadoop-three.version=3.0.0-alpha1
> -Dhadoop.profile=3.0}} fails for me, but passes with
> {{-Dhadoop-three.version=3.0.0-alpha2}}. The failure with alpha1 is given
> below.
> Haven't investigated the failure, maybe we can simply update the hadoop three
> version to alpha2?
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.8:run (generate) on project
> hbase-server: An Ant BuildException has occured: java.lang.NoSuchMethodError:
> javax.servlet.ServletContext.getJspConfigDescriptor()Ljavax/servlet/descriptor/JspConfigDescriptor;
> [ERROR] around Ant part ... outputdir="${generated.sources}/java"
> package="org.apache.hadoop.hbase.generated.master"
> webxml="${build.webapps}/master/WEB-INF/web.xml"/>... @ 17:187 in
> /Users/appy/apache/hbase/hbase-server/target/antrun/build-main.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17587) Do not Rethrow DoNotRetryIOException as UnknownScannerException
[ https://issues.apache.org/jira/browse/HBASE-17587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852286#comment-15852286 ] Zach York commented on HBASE-17587: --- Thanks [~enis]. Thanks for the unit testing as well. I spent a little bit looking into unit testing it, but didn't see an easy way to add it. > Do not Rethrow DoNotRetryIOException as UnknownScannerException > --- > > Key: HBASE-17587 > URL: https://issues.apache.org/jira/browse/HBASE-17587 > Project: HBase > Issue Type: Bug > Components: Coprocessors, regionserver, rpc >Affects Versions: 2.0.0, 1.3.0 >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17587.branch-1.001.patch, > HBASE-17587.branch-1.002.patch, HBASE-17587.branch-1.003.patch, > HBASE-17587.branch-1.3.001.patch, HBASE-17587.branch-1.3.002.patch, > HBASE-17587.branch-1.3.003.patch, HBASE-17587.master.001.patch, > HBASE-17587.master.002.patch, HBASE-17587.master.003.patch > > > HBase commit > https://github.com/apache/hbase/commit/94ade6a514e9935a8c283befb31f29cd8d3a2045 > broke co-processors (such as Phoenix) throwing DoNotRetryIOExceptions when > scanning a table. This change rethrows them as UnknownScannerExceptions which > the HBase client will retry on. This is unintended behavior since > co-processors should be able to throw DoNotRetryIOExceptions back to the > client. This came up through a phoenix IT when trying to upgrade to HBase > 1.3.0 https://github.com/apache/phoenix/pull/230 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser reassigned HBASE-17592:
--
Assignee: Attila Sasvari
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17592-1.patch
>
>
> Fix typos in timeout related client exception messages in {{IPCUtil.java}}
> and {{RpcConnection.java}} - "rpcTimetout".
> {code}
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> 179:+ (EnvironmentEdgeManager.currentTime() - call.getStartTime()) +
> ", rpcTimetout="
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
> 196: + (EnvironmentEdgeManager.currentTime() -
> call.getStartTime()) + ", rpcTimetout="
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17587) Do not Rethrow DoNotRetryIOException as UnknownScannerException
[ https://issues.apache.org/jira/browse/HBASE-17587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852270#comment-15852270 ] Enis Soztutar commented on HBASE-17587: --- Linking HBASE-17187, which has a slightly more comprehensive fix. I'll commit that on top of this jira. > Do not Rethrow DoNotRetryIOException as UnknownScannerException > --- > > Key: HBASE-17587 > URL: https://issues.apache.org/jira/browse/HBASE-17587 > Project: HBase > Issue Type: Bug > Components: Coprocessors, regionserver, rpc >Affects Versions: 2.0.0, 1.3.0 >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: HBASE-17587.branch-1.001.patch, > HBASE-17587.branch-1.002.patch, HBASE-17587.branch-1.003.patch, > HBASE-17587.branch-1.3.001.patch, HBASE-17587.branch-1.3.002.patch, > HBASE-17587.branch-1.3.003.patch, HBASE-17587.master.001.patch, > HBASE-17587.master.002.patch, HBASE-17587.master.003.patch > > > HBase commit > https://github.com/apache/hbase/commit/94ade6a514e9935a8c283befb31f29cd8d3a2045 > broke co-processors (such as Phoenix) throwing DoNotRetryIOExceptions when > scanning a table. This change rethrows them as UnknownScannerExceptions which > the HBase client will retry on. This is unintended behavior since > co-processors should be able to throw DoNotRetryIOExceptions back to the > client. This came up through a phoenix IT when trying to upgrade to HBase > 1.3.0 https://github.com/apache/phoenix/pull/230 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852265#comment-15852265
]
Enis Soztutar commented on HBASE-17187:
---
Thanks [~apurtell]. Indeed looks like it maybe for the same issue. My bad that
this fell from my radar. The fix here is though more complete, and comes with a
unit test. Let me rebase this patch on top of HBASE-17587. No need to revert
that I think.
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17593) Fix build with hadoop 3 profile
[
https://issues.apache.org/jira/browse/HBASE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17593:
-
Attachment: HBASE-17593.master.001.patch
> Fix build with hadoop 3 profile
> ---
>
> Key: HBASE-17593
> URL: https://issues.apache.org/jira/browse/HBASE-17593
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17593.master.001.patch
>
>
> {{mvn clean install -DskipTests -Dhadoop-three.version=3.0.0-alpha1
> -Dhadoop.profile=3.0}} fails for me, but passes with
> {{-Dhadoop-three.version=3.0.0-alpha2}}. The failure with alpha1 is given
> below.
> Haven't investigated the failure, maybe we can simply update the hadoop three
> version to alpha2?
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.8:run (generate) on project
> hbase-server: An Ant BuildException has occured: java.lang.NoSuchMethodError:
> javax.servlet.ServletContext.getJspConfigDescriptor()Ljavax/servlet/descriptor/JspConfigDescriptor;
> [ERROR] around Ant part ... outputdir="${generated.sources}/java"
> package="org.apache.hadoop.hbase.generated.master"
> webxml="${build.webapps}/master/WEB-INF/web.xml"/>... @ 17:187 in
> /Users/appy/apache/hbase/hbase-server/target/antrun/build-main.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17593) Fix build with hadoop 3 profile
[
https://issues.apache.org/jira/browse/HBASE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17593:
-
Status: Patch Available (was: Open)
> Fix build with hadoop 3 profile
> ---
>
> Key: HBASE-17593
> URL: https://issues.apache.org/jira/browse/HBASE-17593
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17593.master.001.patch
>
>
> {{mvn clean install -DskipTests -Dhadoop-three.version=3.0.0-alpha1
> -Dhadoop.profile=3.0}} fails for me, but passes with
> {{-Dhadoop-three.version=3.0.0-alpha2}}. The failure with alpha1 is given
> below.
> Haven't investigated the failure, maybe we can simply update the hadoop three
> version to alpha2?
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.8:run (generate) on project
> hbase-server: An Ant BuildException has occured: java.lang.NoSuchMethodError:
> javax.servlet.ServletContext.getJspConfigDescriptor()Ljavax/servlet/descriptor/JspConfigDescriptor;
> [ERROR] around Ant part ... outputdir="${generated.sources}/java"
> package="org.apache.hadoop.hbase.generated.master"
> webxml="${build.webapps}/master/WEB-INF/web.xml"/>... @ 17:187 in
> /Users/appy/apache/hbase/hbase-server/target/antrun/build-main.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17593) Fix build with hadoop 3 profile
Appy created HBASE-17593:
Summary: Fix build with hadoop 3 profile
Key: HBASE-17593
URL: https://issues.apache.org/jira/browse/HBASE-17593
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
{{mvn clean install -DskipTests -Dhadoop-three.version=3.0.0-alpha1
-Dhadoop.profile=3.0}} fails for me, but passes with
{{-Dhadoop-three.version=3.0.0-alpha2}}. The failure with alpha1 is given below.
Haven't investigated the failure, maybe we can simply update the hadoop three
version to alpha2?
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.8:run (generate) on project
hbase-server: An Ant BuildException has occured: java.lang.NoSuchMethodError:
javax.servlet.ServletContext.getJspConfigDescriptor()Ljavax/servlet/descriptor/JspConfigDescriptor;
[ERROR] around Ant part .. @ 17:187 in
/Users/appy/apache/hbase/hbase-server/target/antrun/build-main.xml
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17572) HMaster: Caught throwable while processing event C_M_MERGE_REGION (UndeclaredThrowableException)
[
https://issues.apache.org/jira/browse/HBASE-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17572:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.3.1
1.4.0
Status: Resolved (was: Patch Available)
I make a quick pass over all of ProtobufUtil and the other handlers and this is
the only one affected.
Change only applies to branch-1.3 and branch-1. branch-1.2 and below don't have
the doAs addition where the ServiceException could bubble out. Master has a
different handler implementation.
> HMaster: Caught throwable while processing event C_M_MERGE_REGION
> (UndeclaredThrowableException)
>
>
> Key: HBASE-17572
> URL: https://issues.apache.org/jira/browse/HBASE-17572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.3.1
>
> Attachments: HBASE-17572-branch-1.3.patch
>
>
> Running ITBLL 1B rows against branch-1.3 compiled against Hadoop 2.7.3 with
> the noKill monkey policy, I see both masters go down with
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> In ServerManager#sendRegionsMerge we call ProtobufUtil#mergeRegions, which
> does a doAs, and the code within that block invokes
> RSRpcServices#mergeRegions, but is not resilient against
> RegionOpeningException ("region is opening")
> An UndeclaredThrowableException is "thrown by a method invocation on a proxy
> instance if its invocation handler's invoke method throws a checked exception
> (a Throwable that is not assignable to RuntimeException or Error) that is not
> assignable to any of the exception types declared in the throws clause of the
> method that was invoked on the proxy instance and dispatched to the
> invocation handler."
> (http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/UndeclaredThrowableException.html)
>
> {noformat}
> 2017-01-31 07:21:17,495 FATAL [MASTER_TABLE_OPERATIONS-node-1:16000-0]
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.mergeRegions(ProtobufUtil.java:1990)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionsMerge(ServerManager.java:925)
> at
> org.apache.hadoop.hbase.master.handler.DispatchMergingRegionHandler.process(DispatchMergingRegionHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.google.protobuf.ServiceException:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.RegionOpeningException):
> org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region
> IntegrationTestBigLinkedList,|\xFFnk\x1C\x85<[\x1Ef\xFDE\xF9\xAA\xAC\x08,1485846598043.f56ad22121e872777468020c4452a7c7.
> is opening on node-2.cluster,16020,1485822382322
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2964)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1139)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mergeRegions(RSRpcServices.java:1497)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22749)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2355)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:244)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:340)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.mergeRegions(AdminProtos.java:23695)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1993)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1990)
>
[jira] [Commented] (HBASE-17572) HMaster: Caught throwable while processing event C_M_MERGE_REGION (UndeclaredThrowableException)
[
https://issues.apache.org/jira/browse/HBASE-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852205#comment-15852205
]
Andrew Purtell commented on HBASE-17572:
Fix confirmed, applying to branch-1.3 and up shortly. Thanks [~stack]
> HMaster: Caught throwable while processing event C_M_MERGE_REGION
> (UndeclaredThrowableException)
>
>
> Key: HBASE-17572
> URL: https://issues.apache.org/jira/browse/HBASE-17572
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Attachments: HBASE-17572-branch-1.3.patch
>
>
> Running ITBLL 1B rows against branch-1.3 compiled against Hadoop 2.7.3 with
> the noKill monkey policy, I see both masters go down with
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> In ServerManager#sendRegionsMerge we call ProtobufUtil#mergeRegions, which
> does a doAs, and the code within that block invokes
> RSRpcServices#mergeRegions, but is not resilient against
> RegionOpeningException ("region is opening")
> An UndeclaredThrowableException is "thrown by a method invocation on a proxy
> instance if its invocation handler's invoke method throws a checked exception
> (a Throwable that is not assignable to RuntimeException or Error) that is not
> assignable to any of the exception types declared in the throws clause of the
> method that was invoked on the proxy instance and dispatched to the
> invocation handler."
> (http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/UndeclaredThrowableException.html)
>
> {noformat}
> 2017-01-31 07:21:17,495 FATAL [MASTER_TABLE_OPERATIONS-node-1:16000-0]
> master.HMaster: Caught throwable while processing event C_M_MERGE_REGION
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.mergeRegions(ProtobufUtil.java:1990)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionsMerge(ServerManager.java:925)
> at
> org.apache.hadoop.hbase.master.handler.DispatchMergingRegionHandler.process(DispatchMergingRegionHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.google.protobuf.ServiceException:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.RegionOpeningException):
> org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region
> IntegrationTestBigLinkedList,|\xFFnk\x1C\x85<[\x1Ef\xFDE\xF9\xAA\xAC\x08,1485846598043.f56ad22121e872777468020c4452a7c7.
> is opening on node-2.cluster,16020,1485822382322
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2964)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1139)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mergeRegions(RSRpcServices.java:1497)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22749)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2355)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:244)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:340)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.mergeRegions(AdminProtos.java:23695)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1993)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil$1.run(ProtobufUtil.java:1990)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1719)
> ... 7 more
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.RegionOpeningException):
>
[jira] [Commented] (HBASE-17187) DoNotRetryExceptions from coprocessors should bubble up to the application
[
https://issues.apache.org/jira/browse/HBASE-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852197#comment-15852197
]
Andrew Purtell commented on HBASE-17187:
Is this a dup now that we've committed HBASE-17587 ?
> DoNotRetryExceptions from coprocessors should bubble up to the application
> --
>
> Key: HBASE-17187
> URL: https://issues.apache.org/jira/browse/HBASE-17187
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-17187_v1.patch
>
>
> In HBASE-16604, we fixed a case where scanner retries was causing the scan to
> miss some data in case the scanner is left with a dirty state (like a
> half-seeked KVHeap).
> The patch introduced a minor compatibility issue, because now if a
> coprocessor throws DNRIOE, we still retry the ClientScanner indefinitely.
> The test {{ServerExceptionIT}} in Phoenix is failing because of this with
> HBASE-16604.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-17592:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks, Attila!
Working on getting you added to the contributor's group as well so you can be
marked as the assignee.
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17592-1.patch
>
>
> Fix typos in timeout related client exception messages in {{IPCUtil.java}}
> and {{RpcConnection.java}} - "rpcTimetout".
> {code}
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> 179:+ (EnvironmentEdgeManager.currentTime() - call.getStartTime()) +
> ", rpcTimetout="
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
> 196: + (EnvironmentEdgeManager.currentTime() -
> call.getStartTime()) + ", rpcTimetout="
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-17592:
---
Fix Version/s: 2.0.0
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17592-1.patch
>
>
> Fix typos in timeout related client exception messages in {{IPCUtil.java}}
> and {{RpcConnection.java}} - "rpcTimetout".
> {code}
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> 179:+ (EnvironmentEdgeManager.currentTime() - call.getStartTime()) +
> ", rpcTimetout="
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
> 196: + (EnvironmentEdgeManager.currentTime() -
> call.getStartTime()) + ", rpcTimetout="
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852188#comment-15852188
]
Josh Elser commented on HBASE-17592:
bq. merging this into master and branch-1.
Just kidding, only applies to master. Not landing on branch-1 or earlier.
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Priority: Trivial
> Attachments: HBASE-17592-1.patch
>
>
> Fix typos in timeout related client exception messages in {{IPCUtil.java}}
> and {{RpcConnection.java}} - "rpcTimetout".
> {code}
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> 179:+ (EnvironmentEdgeManager.currentTime() - call.getStartTime()) +
> ", rpcTimetout="
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
> 196: + (EnvironmentEdgeManager.currentTime() -
> call.getStartTime()) + ", rpcTimetout="
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852179#comment-15852179
]
Josh Elser commented on HBASE-17592:
+1 merging this into master and branch-1.
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Priority: Trivial
> Attachments: HBASE-17592-1.patch
>
>
> Fix typos in timeout related client exception messages in {{IPCUtil.java}}
> and {{RpcConnection.java}} - "rpcTimetout".
> {code}
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> 179:+ (EnvironmentEdgeManager.currentTime() - call.getStartTime()) +
> ", rpcTimetout="
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
> 196: + (EnvironmentEdgeManager.currentTime() -
> call.getStartTime()) + ", rpcTimetout="
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17172) Optimize mob compaction with _del files
[
https://issues.apache.org/jira/browse/HBASE-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852158#comment-15852158
]
Hadoop QA commented on HBASE-17172:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m 25s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 38s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 0s
{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 122m 19s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 174m 47s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| |
org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactionRequest$CompactionDelPartitionId
defines equals and uses Object.hashCode() At
PartitionedMobCompactionRequest.java:Object.hashCode() At
PartitionedMobCompactionRequest.java:[lines 315-323] |
| | Should
org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor$DelPartitionComparator
be a _static_ inner class? At PartitionedMobCompactor.java:inner class? At
PartitionedMobCompactor.java:[lines 377-398] |
| Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient |
| Timed out junit tests | org.apache.hadoop.hbase.client.TestAsyncTableBatch |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.1 Server=1.12.1 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12850856/HBASE-17172.master.001.patch
|
| JIRA Issue | HBASE-17172 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 461fbc4738cd 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13
00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1159296 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| findbugs |
[jira] [Commented] (HBASE-17591) Update documentation to say distributed log replay defaults to 'false'
[
https://issues.apache.org/jira/browse/HBASE-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852035#comment-15852035
]
Sean Busbey commented on HBASE-17591:
-
I'd say only if we're also purging the related code (or atleast the code that
lets you turn it on). Otherwise folks will treat it as one of our many "power
user" features and presume that it's "safe" to use.
> Update documentation to say distributed log replay defaults to 'false'
> --
>
> Key: HBASE-17591
> URL: https://issues.apache.org/jira/browse/HBASE-17591
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Aki Ariga
>Priority: Trivial
>
> As consequence of HBASE-12577, `hbase.master.distributed.log.replay` is no
> longer default true. But in the documentation, it is still noted as default
> true as follows:
> {quote}
> To enable distributed log replay, set hbase.master.distributed.log.replay to
> true. This will be the default for HBase 0.99 (HBASE-10888).
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-14199) maven-remote-resources-plugin failure processing NOTICE.vm in hbase-assembly
[
https://issues.apache.org/jira/browse/HBASE-14199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-14199:
-
Description:
Only seen when building 0.98 with -Dhadoop.profile=1.1. Happens with both JDK 6
and 7.
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(default) on project hbase-assembly: Error rendering velocity resource. Error
invoking method
'get(java.lang.Integer)' in java.util.ArrayList at META-INF/NOTICE.vm[line 275,
column 22]:
InvocationTargetException: Index: 0, Size: 0 -> [Help 1]
{noformat}
>From later comment by busbey:
"Use -Dlicense.debug.print.included=true and then to examine the generated
LICENSE file to see what transitive dependency doesn't have license
information.""
was:
Only seen when building 0.98 with -Dhadoop.profile=1.1. Happens with both JDK 6
and 7.
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(default) on project hbase-assembly: Error rendering velocity resource. Error
invoking method
'get(java.lang.Integer)' in java.util.ArrayList at META-INF/NOTICE.vm[line 275,
column 22]:
InvocationTargetException: Index: 0, Size: 0 -> [Help 1]
{noformat}
> maven-remote-resources-plugin failure processing NOTICE.vm in hbase-assembly
>
>
> Key: HBASE-14199
> URL: https://issues.apache.org/jira/browse/HBASE-14199
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 0.98.14
>
> Attachments: HBASE-14199-0.98.patch, HBASE-14199.patch,
> HBASE-14199.patch
>
>
> Only seen when building 0.98 with -Dhadoop.profile=1.1. Happens with both JDK
> 6 and 7.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
> (default) on project hbase-assembly: Error rendering velocity resource. Error
> invoking method
> 'get(java.lang.Integer)' in java.util.ArrayList at META-INF/NOTICE.vm[line
> 275, column 22]:
> InvocationTargetException: Index: 0, Size: 0 -> [Help 1]
> {noformat}
> From later comment by busbey:
> "Use -Dlicense.debug.print.included=true and then to examine the generated
> LICENSE file to see what transitive dependency doesn't have license
> information.""
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17591) Update documentation to say distributed log replay defaults to 'false'
[
https://issues.apache.org/jira/browse/HBASE-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852018#comment-15852018
]
Enis Soztutar commented on HBASE-17591:
---
Probably we should purge all documentation related to dist log replay. We are
not supporting that feature anymore.
> Update documentation to say distributed log replay defaults to 'false'
> --
>
> Key: HBASE-17591
> URL: https://issues.apache.org/jira/browse/HBASE-17591
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Aki Ariga
>Priority: Trivial
>
> As consequence of HBASE-12577, `hbase.master.distributed.log.replay` is no
> longer default true. But in the documentation, it is still noted as default
> true as follows:
> {quote}
> To enable distributed log replay, set hbase.master.distributed.log.replay to
> true. This will be the default for HBase 0.99 (HBASE-10888).
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17582) Drop page cache hint is broken
[ https://issues.apache.org/jira/browse/HBASE-17582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852011#comment-15852011 ] Appy commented on HBASE-17582: -- Was thinking same. Created HBASE-17590. > Drop page cache hint is broken > -- > > Key: HBASE-17582 > URL: https://issues.apache.org/jira/browse/HBASE-17582 > Project: HBase > Issue Type: Bug > Components: Compaction, io >Affects Versions: 2.0.0 >Reporter: Ashu Pachauri >Assignee: Appy >Priority: Critical > Attachments: HBASE-17582.master.001.patch > > > We pass a boolean for pass page cache drop hint while creating store file > scanners and writers. > The hint is not passed on down the stack by StoreFileWriter and > StoreFileScanner in the master branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17437) Support specifying a WAL directory outside of the root directory
[ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852002#comment-15852002 ] Zach York commented on HBASE-17437: --- [~tedyu] Yes currently you need a clean shutdown to change these particular settings (similar to changing the rootdir I'm assuming). [~enis] Yes I am incorporating HBASE-17588 into my patch for branch-1. I am running into some issues debugging why the tests are failing (For the WALReplay tests) but hopefully I will be able to fix that soon and we can get this patch contributed. Thanks for the interest! > Support specifying a WAL directory outside of the root directory > > > Key: HBASE-17437 > URL: https://issues.apache.org/jira/browse/HBASE-17437 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration, wal >Affects Versions: 1.2.4 >Reporter: Yishan Yang >Assignee: Zach York > Labels: patch > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17437.branch-1.001.patch, > hbase-17437-branch-1.2.patch, HBASE-17437.master.001.patch, > HBASE-17437.master.002.patch, HBASE-17437.master.003.patch, > HBASE-17437.master.004.patch, HBASE-17437.master.005.patch, > HBASE-17437.master.006.patch, HBASE-17437.master.007.patch, > HBASE-17437.master.008.patch, HBASE-17437.master.009.patch, > HBASE-17437.master.010.patch, HBASE-17437.master.011.patch, > HBASE-17437.master.012.patch, hbase-17437-master.patch > > > Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some > FileSystems (such as Amazon S3) don’t support append or consistent writes. > These two properties are imperative for the WAL in order to avoid loss of > writes. However, StoreFiles don’t necessarily need the same consistency > guarantees (since writes are cached locally and if writes fail, they can > always be replayed from the WAL). > > This JIRA aims to allow users to configure a log directory (for WALs) that is > outside of the root directory or even in a different FileSystem. The default > value will still put the log directory under the root directory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-17565:
---
Attachment: 17565.v3.txt
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851993#comment-15851993
]
Ted Yu commented on HBASE-17565:
TestStochasticLoadBalancer2 is even longer:
{code}
Running org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 422.557 sec -
in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
{code}
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt, 17565.v3.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851983#comment-15851983
]
Ted Yu commented on HBASE-17565:
Patch v3 coming up with two additional mocked clusters.
TestStochasticLoadBalancer took 300 seconds on a node with SSD:
{code}
Running org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer
Tests run: 23, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 303.671 sec -
in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer
{code}
I don't want to make the test too long since the Apache node running tests is
much slower.
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17437) Support specifying a WAL directory outside of the root directory
[ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851962#comment-15851962 ] Enis Soztutar commented on HBASE-17437: --- Does branch-1 patch include HBASE-17588, or we have to backport that as well? > Support specifying a WAL directory outside of the root directory > > > Key: HBASE-17437 > URL: https://issues.apache.org/jira/browse/HBASE-17437 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration, wal >Affects Versions: 1.2.4 >Reporter: Yishan Yang >Assignee: Zach York > Labels: patch > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17437.branch-1.001.patch, > hbase-17437-branch-1.2.patch, HBASE-17437.master.001.patch, > HBASE-17437.master.002.patch, HBASE-17437.master.003.patch, > HBASE-17437.master.004.patch, HBASE-17437.master.005.patch, > HBASE-17437.master.006.patch, HBASE-17437.master.007.patch, > HBASE-17437.master.008.patch, HBASE-17437.master.009.patch, > HBASE-17437.master.010.patch, HBASE-17437.master.011.patch, > HBASE-17437.master.012.patch, hbase-17437-master.patch > > > Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some > FileSystems (such as Amazon S3) don’t support append or consistent writes. > These two properties are imperative for the WAL in order to avoid loss of > writes. However, StoreFiles don’t necessarily need the same consistency > guarantees (since writes are cached locally and if writes fail, they can > always be replayed from the WAL). > > This JIRA aims to allow users to configure a log directory (for WALs) that is > outside of the root directory or even in a different FileSystem. The default > value will still put the log directory under the root directory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17565) StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
[
https://issues.apache.org/jira/browse/HBASE-17565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851954#comment-15851954
]
Enis Soztutar commented on HBASE-17565:
---
I think we should add a couple of mock clusters which are like the cluster
described in the description, something like {{500, 500, 500, 500, 500, 0}},
{{1500, 500, 500, 500, 10, 0}}, {{1500, 500, 500, 10, 10, 0}}, etc.
TestStochasticLoadBalancer has very similar tests already, so we can extend the
tests to reproduce the case of needsBalance() not working correctly without the
patch, and fixing the test with the patch.
> StochasticLoadBalancer may incorrectly skip balancing due to skewed
> multiplier sum
> --
>
> Key: HBASE-17565
> URL: https://issues.apache.org/jira/browse/HBASE-17565
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17565.v1.txt, 17565.v2.txt
>
>
> I was investigating why a 6 node cluster kept skipping balancing requests.
> Here were the region counts on the servers:
> 449, 448, 447, 449, 453, 0
> {code}
> 2017-01-26 22:04:47,145 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=1,queue=0,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 127.0171157050385, sum multiplier is 111087.0 min cost
> which need balance is 0.05
> {code}
> The big multiplier sum caught my eyes. Here was what additional debug logging
> showed:
> {code}
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaHostCostFunction with multiplier 10.0
> 2017-01-27 23:25:31,749 DEBUG
> [RpcServer.deafult.FPBQ.Fifo.handler=9,queue=0,port=16000]
> balancer.StochasticLoadBalancer: class
> org.apache.hadoop.hbase.master.balancer.
> StochasticLoadBalancer$RegionReplicaRackCostFunction with multiplier 1.0
> {code}
> Note however, that no table in the cluster used read replica.
> I can think of two ways of fixing this situation:
> 1. If there is no read replica in the cluster, ignore the multipliers for the
> above two functions.
> 2. When cost() returned by the CostFunction is 0 (or very very close to 0.0),
> ignore the multiplier.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17389) Convert all internal usages from ReplicationAdmin to Admin
[
https://issues.apache.org/jira/browse/HBASE-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851927#comment-15851927
]
Enis Soztutar commented on HBASE-17389:
---
You should not create a connection inside the master:
{code}
-replicationAdmin = new ReplicationAdmin(master.getConfiguration());
+admin =
ConnectionFactory.createConnection(master.getConfiguration()).getAdmin();
{code}
> Convert all internal usages from ReplicationAdmin to Admin
> --
>
> Key: HBASE-17389
> URL: https://issues.apache.org/jira/browse/HBASE-17389
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17389-v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851903#comment-15851903
]
Hadoop QA commented on HBASE-17592:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 56s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 13s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 35s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12850854/HBASE-17592-1.patch |
| JIRA Issue | HBASE-17592 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ea5ac91979a0 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1159296 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5563/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5563/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Priority: Trivial
> Attachments: HBASE-17592-1.patch
>
[jira] [Commented] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851899#comment-15851899 ] Stephen Yuan Jiang commented on HBASE-16488: Re-attach the V2 patch and run it again. Last run seems like an env issue > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.0.0, 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Attachments: HBASE-16488.v1-branch-1.patch, > HBASE-16488.v1-master.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851884#comment-15851884 ] Vladimir Rodionov commented on HBASE-14123: --- Ping [~enis] > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0 > > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v19.txt, 14123-master.v20.txt, 14123-master.v21.txt, > 14123-master.v24.txt, 14123-master.v25.txt, 14123-master.v27.txt, > 14123-master.v28.txt, 14123-master.v29.full.txt, 14123-master.v2.txt, > 14123-master.v30.txt, 14123-master.v31.txt, 14123-master.v32.txt, > 14123-master.v33.txt, 14123-master.v34.txt, 14123-master.v35.txt, > 14123-master.v36.txt, 14123-master.v37.txt, 14123-master.v38.txt, > 14123.master.v39.patch, 14123-master.v3.txt, 14123.master.v40.patch, > 14123.master.v41.patch, 14123.master.v42.patch, 14123.master.v44.patch, > 14123.master.v45.patch, 14123.master.v46.patch, 14123.master.v48.patch, > 14123.master.v49.patch, 14123.master.v50.patch, 14123.master.v51.patch, > 14123.master.v52.patch, 14123.master.v54.patch, 14123-master.v5.txt, > 14123-master.v6.txt, 14123-master.v7.txt, 14123-master.v8.txt, > 14123-master.v9.txt, 14123-v14.txt, HBASE-14123-for-7912-v1.patch, > HBASE-14123-for-7912-v6.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch, > HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch, > HBASE-14123-v16.patch, HBASE-14123-v1.patch, HBASE-14123-v2.patch, > HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, > HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17172) Optimize mob compaction with _del files
[ https://issues.apache.org/jira/browse/HBASE-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-17172: - Status: Patch Available (was: Open) Since HBASE-16981 is committed, I rebased the patch based on HBASE-16981 and attached the patch here, it is also available at review board. > Optimize mob compaction with _del files > --- > > Key: HBASE-17172 > URL: https://issues.apache.org/jira/browse/HBASE-17172 > Project: HBase > Issue Type: Improvement > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17172-master-001.patch, > HBASE-17172.master.001.patch > > > Today, when there is a _del file in mobdir, with major mob compaction, every > mob file will be recompacted, this causes lots of IO and slow down major mob > compaction (may take months to finish). This needs to be improved. A few > ideas are: > 1) Do not compact all _del files into one, instead, compact them based on > groups with startKey as the key. Then use firstKey/startKey to make each mob > file to see if the _del file needs to be included for this partition. > 2). Based on the timerange of the _del file, compaction for files after that > timerange does not need to include the _del file as these are newer files. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-16488: --- Attachment: HBASE-16488.v2-branch-1.patch > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.0.0, 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Attachments: HBASE-16488.v1-branch-1.patch, > HBASE-16488.v1-master.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17172) Optimize mob compaction with _del files
[ https://issues.apache.org/jira/browse/HBASE-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-17172: - Attachment: HBASE-17172.master.001.patch > Optimize mob compaction with _del files > --- > > Key: HBASE-17172 > URL: https://issues.apache.org/jira/browse/HBASE-17172 > Project: HBase > Issue Type: Improvement > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17172-master-001.patch, > HBASE-17172.master.001.patch > > > Today, when there is a _del file in mobdir, with major mob compaction, every > mob file will be recompacted, this causes lots of IO and slow down major mob > compaction (may take months to finish). This needs to be improved. A few > ideas are: > 1) Do not compact all _del files into one, instead, compact them based on > groups with startKey as the key. Then use firstKey/startKey to make each mob > file to see if the _del file needs to be included for this partition. > 2). Based on the timerange of the _del file, compaction for files after that > timerange does not need to include the _del file as these are newer files. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17172) Optimize mob compaction with _del files
[ https://issues.apache.org/jira/browse/HBASE-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-17172: - Summary: Optimize mob compaction with _del files (was: Optimize major mob compaction with _del files) > Optimize mob compaction with _del files > --- > > Key: HBASE-17172 > URL: https://issues.apache.org/jira/browse/HBASE-17172 > Project: HBase > Issue Type: Improvement > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-17172-master-001.patch > > > Today, when there is a _del file in mobdir, with major mob compaction, every > mob file will be recompacted, this causes lots of IO and slow down major mob > compaction (may take months to finish). This needs to be improved. A few > ideas are: > 1) Do not compact all _del files into one, instead, compact them based on > groups with startKey as the key. Then use firstKey/startKey to make each mob > file to see if the _del file needs to be included for this partition. > 2). Based on the timerange of the _del file, compaction for files after that > timerange does not need to include the _del file as these are newer files. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17261) Balancer makes no sense on tip of branch-1: says balanced when not
[
https://issues.apache.org/jira/browse/HBASE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851835#comment-15851835
]
Ted Yu commented on HBASE-17261:
bq. This patch is only make the default behavior same with before HBASE-15529
Why restoring the default behavior to pre-HBASE-15529 status ? Isn't that
defeating the purpose of HBASE-15529 ?
I looked at HBASE-15529 but didn't see release note on which multiplier config
to change when minCostNeedBalance is set to a non-zero value.
Can you clarify ?
> Balancer makes no sense on tip of branch-1: says balanced when not
> --
>
> Key: HBASE-17261
> URL: https://issues.apache.org/jira/browse/HBASE-17261
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Guanghao Zhang
> Attachments: HBASE-17261.patch
>
>
> Running ITBLL on tip of branch-1, I see this in log when I try to balance:
> {code}
> 2016-12-05 16:42:21,031 INFO
> [RpcServer.deafult.FPBQ.Fifo.handler=46,queue=1,port=16000]
> balancer.StochasticLoadBalancer: Skipping load balancing because balanced
> cluster; total cost is 525.2547686174673|
> , sum multiplier is 111087.0 min cost which need balance is 0.05
> {code}
> Its some old nonsense.
> Does this every time I balance. Can't even force a balance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17592) Fix typo in IPCUtil and RpcConnection
[
https://issues.apache.org/jira/browse/HBASE-17592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated HBASE-17592:
---
Status: Patch Available (was: Open)
> Fix typo in IPCUtil and RpcConnection
> -
>
> Key: HBASE-17592
> URL: https://issues.apache.org/jira/browse/HBASE-17592
> Project: HBase
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Priority: Trivial
> Attachments: HBASE-17592-1.patch
>
>
> Fix typos in timeout related client exception messages in {{IPCUtil.java}}
> and {{RpcConnection.java}} - "rpcTimetout".
> {code}
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
> 179:+ (EnvironmentEdgeManager.currentTime() - call.getStartTime()) +
> ", rpcTimetout="
> hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcConnection.java
> 196: + (EnvironmentEdgeManager.currentTime() -
> call.getStartTime()) + ", rpcTimetout="
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
