[jira] [Commented] (HBASE-16309) TestDefaultCompactSelection.testCompactionRatio is flaky
[
https://issues.apache.org/jira/browse/HBASE-16309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476002#comment-15476002
]
Hadoop QA commented on HBASE-16309:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 47s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 40s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-09 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12827698/HBASE-16309-v1.patch |
| JIRA Issue | HBASE-16309 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux cb00a6b16bc8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 46c756a |
| Default Java | 1.7.0_111 |
| Multi-JDK versions |
[jira] [Commented] (HBASE-16345) RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions
[
https://issues.apache.org/jira/browse/HBASE-16345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475996#comment-15475996
]
Hadoop QA commented on HBASE-16345:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 58s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 1s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 101m 12s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 152m 29s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-09 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12827694/HBASE-16345.master.005.patch
|
| JIRA Issue | HBASE-16345 |
| Optional Tests | asflicense javac javadoc unit
[jira] [Commented] (HBASE-16505) Add AsyncRegion interface to pass deadline and support async operations
[ https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475984#comment-15475984 ] Duo Zhang commented on HBASE-16505: --- I think we should first find the most suitable way to solve the problem, and then consider how to backport it. For branch-1, if we can assume that the code is always execute in a rpc handler thread, then maybe we could use ThreadLocal to pass some parameters. For example, we already have a RpcServer.getCurrentCall to get the information of the current rpc call. Maybe we can add a getDeadline to the return value. But I think this is not a good practice so we should not use it on master. > Add AsyncRegion interface to pass deadline and support async operations > --- > > Key: HBASE-16505 > URL: https://issues.apache.org/jira/browse/HBASE-16505 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0 > > Attachments: HBASE-16505-v1.patch, HBASE-16505-v10.patch, > HBASE-16505-v10.patch, HBASE-16505-v2.patch, HBASE-16505-v3.patch, > HBASE-16505-v4.patch, HBASE-16505-v5.patch, HBASE-16505-v6.patch, > HBASE-16505-v7.patch, HBASE-16505-v8.patch, HBASE-16505-v9.patch > > > If we want to know the correct setting of timeout in read/write path, we need > add a new parameter in operation-methods of Region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16530) Reduce DBE code duplication
[ https://issues.apache.org/jira/browse/HBASE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475969#comment-15475969 ] ramkrishna.s.vasudevan commented on HBASE-16530: Thanks for committing this [~carp84]. > Reduce DBE code duplication > --- > > Key: HBASE-16530 > URL: https://issues.apache.org/jira/browse/HBASE-16530 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: binlijin >Assignee: binlijin > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16530-master_V1.patch, > HBASE-16530-master_V2.patch, HBASE-16530-master_V3.patch, > HBASE-16530-master_V4.patch, HBASE-16530-master_V4.patch, > HBASE-16530.branch-1.v1.patch, HBASE-16530.branch-1.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16505) Add AsyncRegion interface to pass deadline and support async operations
[
https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475957#comment-15475957
]
Phil Yang commented on HBASE-16505:
---
Hi [~enis] and [~Apache9]
{quote}
This is probably intended only for 2.0, so we can and should change the Region
interface instead of building on top. There are no guarantees for coprocessor
compatibility between 1.x and 2.0.
{quote}
{quote}
I think this will not be introduced in branch-1. Let's use CompletableFuture
{quote}
What I thought is that we can also port this to branch-1, so we can make
HBASE-16492 also committed to 1.x. Only setting deadline is not a very big
change. This is the reason why I did not use CompletableFuture. An alternative
solution is use different interface between 1.x and 2.0, but it may introduce
more maintenance work. What do you think for HBASE-16492 for branch-1?
{quote}
What are the plans for checking the deadline? Not every operation can throw
timeouts in the middle. The plan is to hand code timeout checks in most places
in get / put code ?
{quote}
I think the most suitable check points should be the blocking operations. For
example, WAl.sync(), Lock.lock(), or others. Although some of them may do not
support setting timeout, it is not difficult to change it. And for scan, we
have already heartbeat logic that check deadline in each 1 cells by
default. Scanning indeed may timeout on non-blocking methods. But other
operations in Region I think the most common check points are blocking methods.
> Add AsyncRegion interface to pass deadline and support async operations
> ---
>
> Key: HBASE-16505
> URL: https://issues.apache.org/jira/browse/HBASE-16505
> Project: HBase
> Issue Type: Sub-task
>Reporter: Phil Yang
>Assignee: Phil Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16505-v1.patch, HBASE-16505-v10.patch,
> HBASE-16505-v10.patch, HBASE-16505-v2.patch, HBASE-16505-v3.patch,
> HBASE-16505-v4.patch, HBASE-16505-v5.patch, HBASE-16505-v6.patch,
> HBASE-16505-v7.patch, HBASE-16505-v8.patch, HBASE-16505-v9.patch
>
>
> If we want to know the correct setting of timeout in read/write path, we need
> add a new parameter in operation-methods of Region.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16505) Add AsyncRegion interface to pass deadline and support async operations
[ https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475939#comment-15475939 ] Duo Zhang commented on HBASE-16505: --- Good. I think this will not be introduced in branch-1. Let's use CompletableFuture [~yangzhe1991]. And also, [~busbey], any updates on the JDK8 only pre-commit build? Thanks. > Add AsyncRegion interface to pass deadline and support async operations > --- > > Key: HBASE-16505 > URL: https://issues.apache.org/jira/browse/HBASE-16505 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0 > > Attachments: HBASE-16505-v1.patch, HBASE-16505-v10.patch, > HBASE-16505-v10.patch, HBASE-16505-v2.patch, HBASE-16505-v3.patch, > HBASE-16505-v4.patch, HBASE-16505-v5.patch, HBASE-16505-v6.patch, > HBASE-16505-v7.patch, HBASE-16505-v8.patch, HBASE-16505-v9.patch > > > If we want to know the correct setting of timeout in read/write path, we need > add a new parameter in operation-methods of Region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16257) Move staging dir to be under hbase root dir
[
https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475899#comment-15475899
]
Jerry He commented on HBASE-16257:
--
bq. How do we handle upgrades?
This would be compatible with rolling upgrade, as indicated in the parent JIRA.
Changing the staging dir in the middle won't break the compatibility. Old and
new staging dirs can co-exists. Bulk load token is full path.
> Move staging dir to be under hbase root dir
> ---
>
> Key: HBASE-16257
> URL: https://issues.apache.org/jira/browse/HBASE-16257
> Project: HBase
> Issue Type: Sub-task
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16257-v1.patch, HBASE-16257-v2.patch
>
>
> The hbase.bulkload.staging.dir defaults to hbase.fs.tmp.dir which then
> defaults to
> {code}
> public static final String DEFAULT_TEMPORARY_HDFS_DIRECTORY = "/user/"
> + System.getProperty("user.name") + "/hbase-staging";
> {code}
> This default would have problem on local file system standalone case.
> We can move the staging dir to be under hbase.rootdir. We are bringing
> secure bulkload to the core. It makes sense to bring it under core control as
> well, instead of an optional property.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16086) TableCfWALEntryFilter and ScopeWALEntryFilter should not redundantly iterate over cells.
[
https://issues.apache.org/jira/browse/HBASE-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475884#comment-15475884
]
Hadoop QA commented on HBASE-16086:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
27s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 29s
{color} | {color:red} hbase-server-jdk1.8.0_101 with JDK v1.8.0_101 generated 1
new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 41s
{color} | {color:red} hbase-server-jdk1.7.0_111 with JDK v1.7.0_111 generated 1
new + 2 unchanged - 0 fixed = 3 total (was 2) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 95m 12s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 141m 18s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-09 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12827690/HBASE-16086.v2.patch |
| JIRA Issue | HBASE-16086 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 03ec235fd0bb 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[
https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475866#comment-15475866
]
Phil Yang commented on HBASE-16583:
---
{quote}
One of the most common and classic critique of the SEDA architecture, by the
original proponent of the idea as well as others, is the overhead of connecting
stages through event queues lowers the ceiling for performance.
{quote}
SDEA is proposed "many" years ago, whose original idea is not perfectly matched
with current hardware/kernel. What we need is referencing this idea to help us.
I agree that queues decrease the performance. We don't need many stages. And
maybe we indeed can remove some queues like mentioned above that if RPC reader
threads directly executing the call, the performance can be improved. I think
we don't need to split into two stages if before and after this point the
bottleneck is almost same, for example, both of them mainly consume CPU. More
specifically, I think the only place we need to discuss if we should use a
queue is IO, especially network IO.
One of the features that HBase is different from other databases is that we use
a distributed file system, HDFS, rather than save data locally. In other
databases which saves data locally, in the database-engine logic(LSM-tree or
BTree) they don't need any RPC call. All the resources it need is
CPU/memory/disks. And the local resources(especially disks) will not be
accessed by other nodes. So in that architecture, if one of
resources(CPU/memory/disks) reach the bottleneck, the whole node reach the
bottleneck, one "stage" from start to end is enough.
However, HBase has RPC calls in read/write path. Even if we make the locality
to 1.0 and use short circuit read, we still call RPC. And now we call them in a
blocking way(even use AsyncWAL we will blocked on WAL.sync in handler). We
don't know if the RPC server works well. And if a DN is reaching its
bottleneck, it doesn't mean our RS is also reaching our bottleneck. So if we
can use two thread pools that one do the local logic and the other do the RPC
logic, I am not sure if the performance must can be improved, but I think the
availability which means the performance(both throughput and latency) when one
of DNs is slow than others can be protected.
> Staged Event-Driven Architecture
>
>
> Key: HBASE-16583
> URL: https://issues.apache.org/jira/browse/HBASE-16583
> Project: HBase
> Issue Type: Umbrella
>Reporter: Phil Yang
>
> Staged Event-Driven Architecture (SEDA) splits request-handling logic into
> several stages, each stage is executed in a thread pool and they are
> connected by queues.
> Currently, in region server we use a thread pool to handle requests from
> client. The number of handlers is configurable, reading and writing use
> different pools. The current architecture has two limitations:
> Performance:
> Different part of the handling path has different bottleneck. For example,
> accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly
> consumes network/disk IO. If we use SEDA and split them into two different
> stages, we can use different numbers for two pools according to the
> CPU/disk/network performance case by case.
> Availability:
> HBASE-16388 described a scene that if the client use a thread pool and use
> blocking methods to access region servers, only one slow server may exhaust
> most of threads of the client. For HBase, we are the client and HDFS
> datanodes are the servers. A slow datanode may exhaust most of handlers. The
> best way to resolve this issue is make HDFS requests non-blocking, which is
> exactly what SEDA does.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16257) Move staging dir to be under hbase root dir
[
https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475863#comment-15475863
]
Jerry He commented on HBASE-16257:
--
Hi, Enis
Thanks for the comments. I will update the patch as commented.
bq. Are these changes related?
Not directly related. The intention was to put the CORRUPT_DIR_NAME always
under hbase.rootdir and get rid of the property "hbase.hfile.quarantine.dir".
It is not referenced or documented anywhere else.
Ok to remove? I can put a deprecate warning in the log.
bq. Is this conf coming from the remote(sink) cluster for the HFileReplicator?
Yes
> Move staging dir to be under hbase root dir
> ---
>
> Key: HBASE-16257
> URL: https://issues.apache.org/jira/browse/HBASE-16257
> Project: HBase
> Issue Type: Sub-task
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16257-v1.patch, HBASE-16257-v2.patch
>
>
> The hbase.bulkload.staging.dir defaults to hbase.fs.tmp.dir which then
> defaults to
> {code}
> public static final String DEFAULT_TEMPORARY_HDFS_DIRECTORY = "/user/"
> + System.getProperty("user.name") + "/hbase-staging";
> {code}
> This default would have problem on local file system standalone case.
> We can move the staging dir to be under hbase.rootdir. We are bringing
> secure bulkload to the core. It makes sense to bring it under core control as
> well, instead of an optional property.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475839#comment-15475839 ] Hudson commented on HBASE-16589: FAILURE: Integrated in Jenkins build HBase-1.4 #402 (See [https://builds.apache.org/job/HBase-1.4/402/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev 52963b34286a8cfd5f7e5d0c9505f62b32f29814) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15947) Classes used only for tests included in main code base
[
https://issues.apache.org/jira/browse/HBASE-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475819#comment-15475819
]
Hadoop QA commented on HBASE-15947:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 18s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 52s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 17s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.0 Server=1.12.0 Image:yetus/hbase:date2016-09-09 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12810066/HBASE-15947-v3.patch |
| JIRA Issue | HBASE-15947 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 31b1f65ded13 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 46c756a |
| Default Java | 1.7.0_111 |
| Multi-JDK
[jira] [Commented] (HBASE-16309) TestDefaultCompactSelection.testCompactionRatio is flaky
[ https://issues.apache.org/jira/browse/HBASE-16309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475798#comment-15475798 ] Duo Zhang commented on HBASE-16309: --- Good. Will commit after the pre-commit report. Thanks. > TestDefaultCompactSelection.testCompactionRatio is flaky > > > Key: HBASE-16309 > URL: https://issues.apache.org/jira/browse/HBASE-16309 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-16309-v1.patch, HBASE-16309.patch > > > The aged major compaction condition is not stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16309) TestDefaultCompactSelection.testCompactionRatio is flaky
[ https://issues.apache.org/jira/browse/HBASE-16309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475796#comment-15475796 ] Matteo Bertozzi commented on HBASE-16309: - +1 on v1. patch looks good to me and I did 20 runs with it and the test have not failed. > TestDefaultCompactSelection.testCompactionRatio is flaky > > > Key: HBASE-16309 > URL: https://issues.apache.org/jira/browse/HBASE-16309 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-16309-v1.patch, HBASE-16309.patch > > > The aged major compaction condition is not stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16530) Reduce DBE code duplication
[
https://issues.apache.org/jira/browse/HBASE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475795#comment-15475795
]
Hadoop QA commented on HBASE-16530:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} branch-1 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s
{color} | {color:green} branch-1 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 3s
{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} branch-1 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} branch-1 passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 46s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 45s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 15s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 6s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegion |
\\
\\
|| Subsystem || Report/Notes ||
| Docker
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475774#comment-15475774 ] Hudson commented on HBASE-16589: FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #12 (See [https://builds.apache.org/job/HBase-1.3-JDK7/12/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev 493c31c297bbe2965ec295180cf627f9c7e01cec) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16309) TestDefaultCompactSelection.testCompactionRatio is flaky
[ https://issues.apache.org/jira/browse/HBASE-16309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-16309: -- Attachment: HBASE-16309-v1.patch Use EnvironmentEdgeManager to fix the issue. [~mbertozzi] Can you please try this patch to see if it works for you? Thanks. > TestDefaultCompactSelection.testCompactionRatio is flaky > > > Key: HBASE-16309 > URL: https://issues.apache.org/jira/browse/HBASE-16309 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-16309-v1.patch, HBASE-16309.patch > > > The aged major compaction condition is not stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-7708) deprecated call in region_mover.rb causing high cpu utilization
[
https://issues.apache.org/jira/browse/HBASE-7708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-7708.
Resolution: Won't Fix
0.92 is long since EOL, closing.
> deprecated call in region_mover.rb causing high cpu utilization
> ---
>
> Key: HBASE-7708
> URL: https://issues.apache.org/jira/browse/HBASE-7708
> Project: HBase
> Issue Type: Improvement
> Components: scripts
>Reporter: Eric Edgeworth
>Priority: Minor
>
> We noticed high CPU utilization while using region_mover.rb to unload
> regions. Consulted with Stack, who suggested changing a line in
> region_mover.rb from (the deprecated):
> {code}
> table = getTable(admin.getConfiguration(), r.getTableDesc().getName())
> {code}
> to:
> {code}
> table = getTable(admin.getConfiguration(), r.getTableName())
> {code}
> In our tests this dropped CPU utilization from ~200% over the course of the
> unload to ~0.5% after startup, and allowed unload of ~200 regions to complete
> in ~7 minutes instead of the original ~18 minutes, so the change seems to fix
> the CPU utilization issue with no untoward side effects we could find.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13879) Add hbase.hstore.compactionThreshold to HConstants
[ https://issues.apache.org/jira/browse/HBASE-13879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475750#comment-15475750 ] Sean Busbey commented on HBASE-13879: - if you're still interested in this [~gliptak], we're trying to avoid any additions to the Public/Stable HConstants. You could move everything to CompactionConfiguration and leave deprecated versions in HConstants. > Add hbase.hstore.compactionThreshold to HConstants > -- > > Key: HBASE-13879 > URL: https://issues.apache.org/jira/browse/HBASE-13879 > Project: HBase > Issue Type: Improvement >Reporter: Gabor Liptak >Assignee: Gabor Liptak >Priority: Minor > Attachments: HBASE-13879.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12786) List the zookeeper quorum nodes on different lines in the Master web ui
[
https://issues.apache.org/jira/browse/HBASE-12786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475746#comment-15475746
]
Hadoop QA commented on HBASE-12786:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-12786 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12689870/0001-HBASE-12786-Change-ZK-Quorum-display-list-each-serve.patch
|
| JIRA Issue | HBASE-12786 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3477/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> List the zookeeper quorum nodes on different lines in the Master web ui
> ---
>
> Key: HBASE-12786
> URL: https://issues.apache.org/jira/browse/HBASE-12786
> Project: HBase
> Issue Type: Task
> Components: master, UI
>Affects Versions: 0.89-fb
>Reporter: Rishit Shroff
>Assignee: kainan jiang
>Priority: Trivial
> Attachments:
> 0001-HBASE-12786-Change-ZK-Quorum-display-list-each-serve.patch
>
>
> HBase uses zookeeper to store the current state of the cluster and uses it to
> maintain the overall consistency of the cluster. The zookeeper is quorum is
> hosted on set of nodes (5 typically) in the same hbase cluster. The names of
> these hosts are accessible in the HMaster web ui page. However, they are all
> ',' separated. It would be nice to list them on different lines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13766) Cleanup several Checkstyle warnings
[
https://issues.apache.org/jira/browse/HBASE-13766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475748#comment-15475748
]
Hadoop QA commented on HBASE-13766:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HBASE-13766 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12741186/HBASE-13766.2.patch |
| JIRA Issue | HBASE-13766 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3478/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Cleanup several Checkstyle warnings
> ---
>
> Key: HBASE-13766
> URL: https://issues.apache.org/jira/browse/HBASE-13766
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Gabor Liptak
>Assignee: Gabor Liptak
>Priority: Minor
> Attachments: HBASE-13766.1.patch, HBASE-13766.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13879) Add hbase.hstore.compactionThreshold to HConstants
[
https://issues.apache.org/jira/browse/HBASE-13879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475749#comment-15475749
]
Hadoop QA commented on HBASE-13879:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HBASE-13879 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12739145/HBASE-13879.1.patch |
| JIRA Issue | HBASE-13879 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3480/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add hbase.hstore.compactionThreshold to HConstants
> --
>
> Key: HBASE-13879
> URL: https://issues.apache.org/jira/browse/HBASE-13879
> Project: HBase
> Issue Type: Improvement
>Reporter: Gabor Liptak
>Assignee: Gabor Liptak
>Priority: Minor
> Attachments: HBASE-13879.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13879) Add hbase.hstore.compactionThreshold to HConstants
[ https://issues.apache.org/jira/browse/HBASE-13879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13879: Assignee: Gabor Liptak > Add hbase.hstore.compactionThreshold to HConstants > -- > > Key: HBASE-13879 > URL: https://issues.apache.org/jira/browse/HBASE-13879 > Project: HBase > Issue Type: Improvement >Reporter: Gabor Liptak >Assignee: Gabor Liptak >Priority: Minor > Attachments: HBASE-13879.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16570) Compute region locality in parallel at startup
[
https://issues.apache.org/jira/browse/HBASE-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475740#comment-15475740
]
Hadoop QA commented on HBASE-16570:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 38s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 1s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 116m 9s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 167m 28s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.1 Server=1.12.1 Image:yetus/hbase:date2016-09-09 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12827679/HBASE-16570-master_V4.patch
|
| JIRA Issue | HBASE-16570 |
| Optional Tests | asflicense javac
[jira] [Updated] (HBASE-13766) Cleanup several Checkstyle warnings
[ https://issues.apache.org/jira/browse/HBASE-13766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13766: Assignee: Gabor Liptak > Cleanup several Checkstyle warnings > --- > > Key: HBASE-13766 > URL: https://issues.apache.org/jira/browse/HBASE-13766 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Gabor Liptak >Assignee: Gabor Liptak >Priority: Minor > Attachments: HBASE-13766.1.patch, HBASE-13766.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13766) Cleanup several Checkstyle warnings
[ https://issues.apache.org/jira/browse/HBASE-13766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13766: Component/s: build > Cleanup several Checkstyle warnings > --- > > Key: HBASE-13766 > URL: https://issues.apache.org/jira/browse/HBASE-13766 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Gabor Liptak >Priority: Minor > Attachments: HBASE-13766.1.patch, HBASE-13766.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15947) Classes used only for tests included in main code base
[ https://issues.apache.org/jira/browse/HBASE-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15947: Component/s: test build > Classes used only for tests included in main code base > -- > > Key: HBASE-15947 > URL: https://issues.apache.org/jira/browse/HBASE-15947 > Project: HBase > Issue Type: Bug > Components: build, test >Reporter: Sean Mackrory >Assignee: Sean Mackrory >Priority: Trivial > Attachments: HBASE-15947-branch-1-v2.patch, HBASE-15947-v2.patch, > HBASE-15947-v3.patch > > > The classes LoadTestKVGenerator and RedundantKVGenerator in > hbase-common/src/main/java/org/apache/hadoop/hbase/util/test are only used in > tests (at least in the HBase codebase itself - not sure if they would have > any use for other software). Should probably move them into src/test/java to > keep a good separation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12786) List the zookeeper quorum nodes on different lines in the Master web ui
[ https://issues.apache.org/jira/browse/HBASE-12786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475733#comment-15475733 ] Sean Busbey commented on HBASE-12786: - what do folks think? the comma separated version can be copy/pasted into cli or other tools that need a ZK quorum, but this is definitely more readable. > List the zookeeper quorum nodes on different lines in the Master web ui > --- > > Key: HBASE-12786 > URL: https://issues.apache.org/jira/browse/HBASE-12786 > Project: HBase > Issue Type: Task > Components: master, UI >Affects Versions: 0.89-fb >Reporter: Rishit Shroff >Assignee: kainan jiang >Priority: Trivial > Attachments: > 0001-HBASE-12786-Change-ZK-Quorum-display-list-each-serve.patch > > > HBase uses zookeeper to store the current state of the cluster and uses it to > maintain the overall consistency of the cluster. The zookeeper is quorum is > hosted on set of nodes (5 typically) in the same hbase cluster. The names of > these hosts are accessible in the HMaster web ui page. However, they are all > ',' separated. It would be nice to list them on different lines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15947) Classes used only for tests included in main code base
[ https://issues.apache.org/jira/browse/HBASE-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15947: Assignee: Sean Mackrory > Classes used only for tests included in main code base > -- > > Key: HBASE-15947 > URL: https://issues.apache.org/jira/browse/HBASE-15947 > Project: HBase > Issue Type: Bug > Components: build, test >Reporter: Sean Mackrory >Assignee: Sean Mackrory >Priority: Trivial > Attachments: HBASE-15947-branch-1-v2.patch, HBASE-15947-v2.patch, > HBASE-15947-v3.patch > > > The classes LoadTestKVGenerator and RedundantKVGenerator in > hbase-common/src/main/java/org/apache/hadoop/hbase/util/test are only used in > tests (at least in the HBase codebase itself - not sure if they would have > any use for other software). Should probably move them into src/test/java to > keep a good separation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12786) List the zookeeper quorum nodes on different lines in the Master web ui
[ https://issues.apache.org/jira/browse/HBASE-12786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-12786: Component/s: UI > List the zookeeper quorum nodes on different lines in the Master web ui > --- > > Key: HBASE-12786 > URL: https://issues.apache.org/jira/browse/HBASE-12786 > Project: HBase > Issue Type: Task > Components: master, UI >Affects Versions: 0.89-fb >Reporter: Rishit Shroff >Assignee: kainan jiang >Priority: Trivial > Attachments: > 0001-HBASE-12786-Change-ZK-Quorum-display-list-each-serve.patch > > > HBase uses zookeeper to store the current state of the cluster and uses it to > maintain the overall consistency of the cluster. The zookeeper is quorum is > hosted on set of nodes (5 typically) in the same hbase cluster. The names of > these hosts are accessible in the HMaster web ui page. However, they are all > ',' separated. It would be nice to list them on different lines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12786) List the zookeeper quorum nodes on different lines in the Master web ui
[ https://issues.apache.org/jira/browse/HBASE-12786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-12786: Assignee: kainan jiang > List the zookeeper quorum nodes on different lines in the Master web ui > --- > > Key: HBASE-12786 > URL: https://issues.apache.org/jira/browse/HBASE-12786 > Project: HBase > Issue Type: Task > Components: master, UI >Affects Versions: 0.89-fb >Reporter: Rishit Shroff >Assignee: kainan jiang >Priority: Trivial > Attachments: > 0001-HBASE-12786-Change-ZK-Quorum-display-list-each-serve.patch > > > HBase uses zookeeper to store the current state of the cluster and uses it to > maintain the overall consistency of the cluster. The zookeeper is quorum is > hosted on set of nodes (5 typically) in the same hbase cluster. The names of > these hosts are accessible in the HMaster web ui page. However, they are all > ',' separated. It would be nice to list them on different lines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16345) RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions
[ https://issues.apache.org/jira/browse/HBASE-16345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-16345: - Attachment: HBASE-16345.master.005.patch > RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer > Exceptions > -- > > Key: HBASE-16345 > URL: https://issues.apache.org/jira/browse/HBASE-16345 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16345-v001.patch, HBASE-16345.master.001.patch, > HBASE-16345.master.002.patch, HBASE-16345.master.003.patch, > HBASE-16345.master.004.patch, HBASE-16345.master.005.patch, > HBASE-16345.master.005.patch > > > Update for the description. Debugged more at this front based on the comments > from Enis. > The cause is that for the primary replica, if its retry is exhausted too > fast, f.get() [1] returns ExecutionException. This Exception needs to be > ignored and continue with the replicas. > The other issue is that after adding calls for the replicas, if the first > completed task gets ExecutionException (due to the retry exhausted), it > throws the exception to the client[2]. > In this case, it needs to loop through these tasks, waiting for the success > one. If no one succeeds, throw exception. > Similar for the scan as well > [1] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L197 > [2] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L219 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16345) RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions
[ https://issues.apache.org/jira/browse/HBASE-16345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475723#comment-15475723 ] huaxiang sun commented on HBASE-16345: -- Locally run TestMasterFailoverWithProcedures passed. Reattach to trigger a new run. > RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer > Exceptions > -- > > Key: HBASE-16345 > URL: https://issues.apache.org/jira/browse/HBASE-16345 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16345-v001.patch, HBASE-16345.master.001.patch, > HBASE-16345.master.002.patch, HBASE-16345.master.003.patch, > HBASE-16345.master.004.patch, HBASE-16345.master.005.patch > > > Update for the description. Debugged more at this front based on the comments > from Enis. > The cause is that for the primary replica, if its retry is exhausted too > fast, f.get() [1] returns ExecutionException. This Exception needs to be > ignored and continue with the replicas. > The other issue is that after adding calls for the replicas, if the first > completed task gets ExecutionException (due to the retry exhausted), it > throws the exception to the client[2]. > In this case, it needs to loop through these tasks, waiting for the success > one. If no one succeeds, throw exception. > Similar for the scan as well > [1] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L197 > [2] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L219 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15829) hbase.client.retries.number has different meanings in branch-1 and master
[
https://issues.apache.org/jira/browse/HBASE-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-15829:
Resolution: Invalid
Fix Version/s: (was: 2.0.0)
Status: Resolved (was: Patch Available)
closing this as superceded by HBASE-14521, since this change in meaning was
intentional.
> hbase.client.retries.number has different meanings in branch-1 and master
> -
>
> Key: HBASE-15829
> URL: https://issues.apache.org/jira/browse/HBASE-15829
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-15829.patch
>
>
> The comment of hbase.client.retries.number is:
> {code}
> /**
>* Parameter name for maximum retries, used as maximum for all retryable
>* operations such as fetching of the root region from root region server,
>* getting a cell's value, starting a row update, etc.
>*/
> public static final String HBASE_CLIENT_RETRIES_NUMBER =
> "hbase.client.retries.number";
> {code}
> In branch-1, the max attempts number equals with hbase.client.retries.number.
> But in master, the max attempts number equals with
> hbase.client.retries.number + 1.
> For RpcRetryingCaller.
> {code}
> this.retries = retries; // branch-1
> {code}
> {code}
> this.maxAttempts = retries + 1; // master
> {code}
> For AsyncProcess:
> {code}
> this.numTries = conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER,
> HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER); // branch-1
> {code}
> {code}
> // how many times we could try in total, one more than retry number
> this.numTries = conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER,
> HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER) + 1; // master
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15134) Add visibility into Flush and Compaction queues
[ https://issues.apache.org/jira/browse/HBASE-15134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475713#comment-15475713 ] Sean Busbey commented on HBASE-15134: - Thanks for your work on this so far! we should avoid needing synchronization unless we need it for correctness. generally, an approximation should be fine for metrics like this. in the future, please use `git format-patch` to create your patches, so we get a commit message and your preferred authorship line. For more details on this, and pointers to some helper scripts see [the ref guide section on submitting patches|http://hbase.apache.org/book.html#submitting.patches] > Add visibility into Flush and Compaction queues > --- > > Key: HBASE-15134 > URL: https://issues.apache.org/jira/browse/HBASE-15134 > Project: HBase > Issue Type: New Feature > Components: Compaction, metrics, regionserver >Reporter: Elliott Clark >Assignee: Abhishek Singh Chouhan > Attachments: HBASE-15134.patch, HBASE-15134.patch > > > On busy spurts we can see regionservers start to see large queues for > compaction. It's really hard to tell if the server is queueing a lot of > compactions for the same region, lots of compactions for lots of regions, or > just falling behind. > For flushes much the same. There can be flushes in queue that aren't being > run because of delayed flushes. There's no way to know from the metrics how > many flushes are for each region, how many are delayed. Etc. > We should add either more metrics around this ( num per region, max per > region, min per region ) or add on a UI page that has the list of compactions > and flushes. > Or both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15134) Add visibility into Flush and Compaction queues
[ https://issues.apache.org/jira/browse/HBASE-15134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15134: Component/s: regionserver metrics Compaction > Add visibility into Flush and Compaction queues > --- > > Key: HBASE-15134 > URL: https://issues.apache.org/jira/browse/HBASE-15134 > Project: HBase > Issue Type: New Feature > Components: Compaction, metrics, regionserver >Reporter: Elliott Clark >Assignee: Abhishek Singh Chouhan > Attachments: HBASE-15134.patch, HBASE-15134.patch > > > On busy spurts we can see regionservers start to see large queues for > compaction. It's really hard to tell if the server is queueing a lot of > compactions for the same region, lots of compactions for lots of regions, or > just falling behind. > For flushes much the same. There can be flushes in queue that aren't being > run because of delayed flushes. There's no way to know from the metrics how > many flushes are for each region, how many are delayed. Etc. > We should add either more metrics around this ( num per region, max per > region, min per region ) or add on a UI page that has the list of compactions > and flushes. > Or both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15134) Add visibility into Flush and Compaction queues
[ https://issues.apache.org/jira/browse/HBASE-15134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15134: Assignee: Abhishek Singh Chouhan > Add visibility into Flush and Compaction queues > --- > > Key: HBASE-15134 > URL: https://issues.apache.org/jira/browse/HBASE-15134 > Project: HBase > Issue Type: New Feature > Components: Compaction, metrics, regionserver >Reporter: Elliott Clark >Assignee: Abhishek Singh Chouhan > Attachments: HBASE-15134.patch, HBASE-15134.patch > > > On busy spurts we can see regionservers start to see large queues for > compaction. It's really hard to tell if the server is queueing a lot of > compactions for the same region, lots of compactions for lots of regions, or > just falling behind. > For flushes much the same. There can be flushes in queue that aren't being > run because of delayed flushes. There's no way to know from the metrics how > many flushes are for each region, how many are delayed. Etc. > We should add either more metrics around this ( num per region, max per > region, min per region ) or add on a UI page that has the list of compactions > and flushes. > Or both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13306) Change RegionScanner interface from stable to evolving
[
https://issues.apache.org/jira/browse/HBASE-13306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475698#comment-15475698
]
Hadoop QA commented on HBASE-13306:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-13306 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12706017/HBASE-13306-v1.patch |
| JIRA Issue | HBASE-13306 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3475/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Change RegionScanner interface from stable to evolving
> --
>
> Key: HBASE-13306
> URL: https://issues.apache.org/jira/browse/HBASE-13306
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0
>Reporter: Jonathan Lawlor
>Assignee: Jonathan Lawlor
>Priority: Trivial
> Attachments: HBASE-13306-v1.patch
>
>
> This is a follow on issue to HBASE-11544.
> Since the RegionScanner interface needed to be changed, the annotation on
> RegionScanner should be evolving rather than stable in order to match semver
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13306) Change RegionScanner interface from stable to evolving
[ https://issues.apache.org/jira/browse/HBASE-13306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13306: Assignee: Jonathan Lawlor > Change RegionScanner interface from stable to evolving > -- > > Key: HBASE-13306 > URL: https://issues.apache.org/jira/browse/HBASE-13306 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.0, 2.0.0, 1.1.0 >Reporter: Jonathan Lawlor >Assignee: Jonathan Lawlor >Priority: Trivial > Attachments: HBASE-13306-v1.patch > > > This is a follow on issue to HBASE-11544. > Since the RegionScanner interface needed to be changed, the annotation on > RegionScanner should be evolving rather than stable in order to match semver -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13306) Change RegionScanner interface from stable to evolving
[ https://issues.apache.org/jira/browse/HBASE-13306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475691#comment-15475691 ] Sean Busbey commented on HBASE-13306: - This would be fine to include in 2.0, now that we're past 1.0 coming out. > Change RegionScanner interface from stable to evolving > -- > > Key: HBASE-13306 > URL: https://issues.apache.org/jira/browse/HBASE-13306 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.0, 2.0.0, 1.1.0 >Reporter: Jonathan Lawlor >Assignee: Jonathan Lawlor >Priority: Trivial > Attachments: HBASE-13306-v1.patch > > > This is a follow on issue to HBASE-11544. > Since the RegionScanner interface needed to be changed, the annotation on > RegionScanner should be evolving rather than stable in order to match semver -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16570) Compute region locality in parallel at startup
[ https://issues.apache.org/jira/browse/HBASE-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475688#comment-15475688 ] Heng Chen commented on HBASE-16570: --- v4 lgtm. Failed TestHRegion has no relates with the patch, and it could pass locally, will commit it if no other concerns. > Compute region locality in parallel at startup > -- > > Key: HBASE-16570 > URL: https://issues.apache.org/jira/browse/HBASE-16570 > Project: HBase > Issue Type: Sub-task >Reporter: binlijin >Assignee: binlijin > Attachments: HBASE-16570-master_V1.patch, > HBASE-16570-master_V2.patch, HBASE-16570-master_V3.patch, > HBASE-16570-master_V4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15535) HBase Reference Guide has an incorrect link for Trafodion
[ https://issues.apache.org/jira/browse/HBASE-15535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15535: Assignee: Gabor Liptak > HBase Reference Guide has an incorrect link for Trafodion > - > > Key: HBASE-15535 > URL: https://issues.apache.org/jira/browse/HBASE-15535 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 2.0.0 >Reporter: Atanu Mishra >Assignee: Gabor Liptak >Priority: Blocker > Attachments: HBASE-15535.1.patch > > > Appendix F in the HBase Reference Guide available here > (https://hbase.apache.org/book.html#sql) has an incorrect link to the site > for Trafodion. > The new Trafodion URL is: http://trafodion.incubator.apache.org/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14505) Reenable tests disabled by HBASE-14430 in TestHttpServerLifecycle
[
https://issues.apache.org/jira/browse/HBASE-14505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475679#comment-15475679
]
Hadoop QA commented on HBASE-14505:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-14505 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12767191/HBASE-14505.3%20%281%29.patch
|
| JIRA Issue | HBASE-14505 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3474/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Reenable tests disabled by HBASE-14430 in TestHttpServerLifecycle
> -
>
> Key: HBASE-14505
> URL: https://issues.apache.org/jira/browse/HBASE-14505
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: stack
>Assignee: Gabor Liptak
> Labels: beginner
> Attachments: HBASE-14505.1.patch, HBASE-14505.2.patch, HBASE-14505.3
> (1).patch, HBASE-14505.3.patch, HBASE-14505.3.patch, HBASE-14505.3.patch
>
>
> Probably needs newer version of jetty or some cryptic JVM version.
> See HBASE-14430 for litany on how hard this is to reproduce, not only in
> hbase, but back up in the Jetty where the issue was also reported.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14505) Reenable tests disabled by HBASE-14430 in TestHttpServerLifecycle
[ https://issues.apache.org/jira/browse/HBASE-14505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14505: Assignee: Gabor Liptak > Reenable tests disabled by HBASE-14430 in TestHttpServerLifecycle > - > > Key: HBASE-14505 > URL: https://issues.apache.org/jira/browse/HBASE-14505 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Gabor Liptak > Labels: beginner > Attachments: HBASE-14505.1.patch, HBASE-14505.2.patch, HBASE-14505.3 > (1).patch, HBASE-14505.3.patch, HBASE-14505.3.patch, HBASE-14505.3.patch > > > Probably needs newer version of jetty or some cryptic JVM version. > See HBASE-14430 for litany on how hard this is to reproduce, not only in > hbase, but back up in the Jetty where the issue was also reported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16345) RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions
[
https://issues.apache.org/jira/browse/HBASE-16345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475654#comment-15475654
]
Hadoop QA commented on HBASE-16345:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 19m 23s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 37s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 2s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 118m 7s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 190m 43s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-08 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-16309) TestDefaultCompactSelection.testCompactionRatio is flaky
[ https://issues.apache.org/jira/browse/HBASE-16309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475644#comment-15475644 ] Duo Zhang commented on HBASE-16309: --- OK, let me pick this up > TestDefaultCompactSelection.testCompactionRatio is flaky > > > Key: HBASE-16309 > URL: https://issues.apache.org/jira/browse/HBASE-16309 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-16309.patch > > > The aged major compaction condition is not stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16576) Shell add_peer doesn't allow setting cluster_key for custom endpoints
[ https://issues.apache.org/jira/browse/HBASE-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475643#comment-15475643 ] Geoffrey Jacoby commented on HBASE-16576: - Thanks, I'll supply an 0.98 patch. > Shell add_peer doesn't allow setting cluster_key for custom endpoints > - > > Key: HBASE-16576 > URL: https://issues.apache.org/jira/browse/HBASE-16576 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0, 1.1.5, 0.98.22 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Attachments: HBASE-16576.patch, HBASE-16576.v1.patch > > > The HBase shell allows a user to create a replication peer using the add_peer > method, which can take a peer id and a Ruby hash. It creates a > ReplicationPeerConfig and passes it through to the Java > ReplicationAdmin#addPeer. > The Ruby code makes an assumption that the Java API doesn't: that CLUSTER_KEY > and ENDPOINT_CLASSNAME are mutually exclusive. If both are specified, it > throws an error. If only ENDPOINT_CLASSNAME is set, the add_peer logic > derives a local dummy cluster key based on the local cluster's configuration. > CLUSTER_KEY shouldn't be required when an ENDPOINT_CLASSNAME is specified, > because a custom endpoint might not need it. The dummy default logic is fine. > > But if an endpoint does require a remote cluster key, it shouldn't be > forbidden to provide one, especially since the Java API permits it, and even > the custom replication endpoint Java tests rely on this. (See > TestReplicationEndpoint#testCustomReplicationEndpoint) > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475625#comment-15475625 ] Hudson commented on HBASE-16589: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #26 (See [https://builds.apache.org/job/HBase-1.2-JDK7/26/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev 4611b15f177bf1ff5cfe0535e34e481a2da36b06) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16086) TableCfWALEntryFilter and ScopeWALEntryFilter should not redundantly iterate over cells.
[ https://issues.apache.org/jira/browse/HBASE-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Poon updated HBASE-16086: - Attachment: HBASE-16086.v2.patch [~chenheng] Thanks for the suggestion, I refactored and made it significantly cleaner. Let me know what you think. > TableCfWALEntryFilter and ScopeWALEntryFilter should not redundantly iterate > over cells. > > > Key: HBASE-16086 > URL: https://issues.apache.org/jira/browse/HBASE-16086 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: churro morales >Assignee: Vincent Poon > Fix For: 2.0.0, 1.3.1 > > Attachments: HBASE-16086.patch, HBASE-16086.v2.patch > > > TableCfWALEntryFilter and ScopeWALEntryFilter both filter by iterating over > cells. Since the filters are chained we do this work twice. Instead iterate > over cells once and apply the "cell filtering" logic to these cells. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15165) AsyncProcess can spin wait indefinitly
[
https://issues.apache.org/jira/browse/HBASE-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475621#comment-15475621
]
Hadoop QA commented on HBASE-15165:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 30s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 1s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 58s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 5s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.wal.TestLogRolling |
| | hadoop.hbase.regionserver.wal.TestAsyncLogRolling |
\\
\\
|| Subsystem
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[ https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475617#comment-15475617 ] Duo Zhang commented on HBASE-16583: --- In fact, the current architecture of RS is already SEDA. We have several threads reading or writing socket, and a thread pool for read request(read handler), a thead pool for write request(write handler), and a thread for writing WAL(or multiple threads if you use MultiWAL). I think the thread pool introduced in the WAL layer is reasonble as we will doing network IO, this is consider to be time consuming. But the rpc handler thread pool is just a simple pattern which is copied from other rpc framework. For a general rpc framework, the thread pool is needed as we do not know what the users will do in a rpc call so the safe way is to give them a seperated thread. But for us, we do know what will happen in the rpc handler, the code is written by us. For example, with AsyncFSWAL, ideally, we could execute a write request directly in the RpcServer's thread. When we reach the WAL layer, we schedule a sync request and just return. The AsyncFSWAL will trigger a callback when the sync is done and finish the remaining work and write the response back(or let the RpcServer's thread to do the work). And if we make the RpcServer also run in the netty EventLoopGroup, we get a TPC architecture, right? But sadly, it is only an ideal... We have increment, which will acquire a write row lock and hold it for a really long time, and also, we will wait mvcc completion before return. So the first intention here, is to find the place where the thread could be blocked for a long time. We could split our workflow to several stages using these points. And in fact, thread swtich is not always needed when we cross these points. For example, if you can acquire the row lock directly, just run it directly. At a high level, this means we could share thread pool between different stages. And if we could share a thread pool across all stages finally, this is TPC. Of course, there are still lots of work to be done, such as making the request from one connection always run in the same thread(good for locking and cpu caching), using lock-free data structure as much as possible...(And how to implement priority?). Thanks. > Staged Event-Driven Architecture > > > Key: HBASE-16583 > URL: https://issues.apache.org/jira/browse/HBASE-16583 > Project: HBase > Issue Type: Umbrella >Reporter: Phil Yang > > Staged Event-Driven Architecture (SEDA) splits request-handling logic into > several stages, each stage is executed in a thread pool and they are > connected by queues. > Currently, in region server we use a thread pool to handle requests from > client. The number of handlers is configurable, reading and writing use > different pools. The current architecture has two limitations: > Performance: > Different part of the handling path has different bottleneck. For example, > accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly > consumes network/disk IO. If we use SEDA and split them into two different > stages, we can use different numbers for two pools according to the > CPU/disk/network performance case by case. > Availability: > HBASE-16388 described a scene that if the client use a thread pool and use > blocking methods to access region servers, only one slow server may exhaust > most of threads of the client. For HBase, we are the client and HDFS > datanodes are the servers. A slow datanode may exhaust most of handlers. The > best way to resolve this issue is make HDFS requests non-blocking, which is > exactly what SEDA does. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14328) WebUI error 500 table doesn't exist
[
https://issues.apache.org/jira/browse/HBASE-14328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475592#comment-15475592
]
Hadoop QA commented on HBASE-14328:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s {color}
| {color:red} HBASE-14328 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12752900/HBASE-14328.1.patch |
| JIRA Issue | HBASE-14328 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3472/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> WebUI error 500 table doesn't exist
> ---
>
> Key: HBASE-14328
> URL: https://issues.apache.org/jira/browse/HBASE-14328
> Project: HBase
> Issue Type: Bug
> Components: Interface
>Affects Versions: 1.1.0.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Gabor Liptak
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-14328.1.patch
>
>
> When trying to load a non-existing table using the WebUI we get an ugly error
> 500. Instead, we should display a nice "The table you are trying to reach
> doesn't exit" message...
> http://server.com:60010/table.jsp?name=TestTable
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> TestTable
> Caused by:
> org.apache.hadoop.hbase.TableNotFoundException: TestTable
> at
> org.apache.hadoop.hbase.client.HTable.getTableDescriptor(HTable.java:651)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:74)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1351)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> Powered by Jetty://
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14328) WebUI error 500 table doesn't exist
[ https://issues.apache.org/jira/browse/HBASE-14328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14328: Status: Patch Available (was: Open) anyone mind seeing if this applies still? [~gliptak] if you're still interested in implementing this, could you see about adding a test? > WebUI error 500 table doesn't exist > --- > > Key: HBASE-14328 > URL: https://issues.apache.org/jira/browse/HBASE-14328 > Project: HBase > Issue Type: Bug > Components: Interface >Affects Versions: 1.1.0.1 >Reporter: Jean-Marc Spaggiari >Assignee: Gabor Liptak >Priority: Minor > Labels: beginner > Attachments: HBASE-14328.1.patch > > > When trying to load a non-existing table using the WebUI we get an ugly error > 500. Instead, we should display a nice "The table you are trying to reach > doesn't exit" message... > http://server.com:60010/table.jsp?name=TestTable > HTTP ERROR 500 > Problem accessing /table.jsp. Reason: > TestTable > Caused by: > org.apache.hadoop.hbase.TableNotFoundException: TestTable > at > org.apache.hadoop.hbase.client.HTable.getTableDescriptor(HTable.java:651) > at > org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:74) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) > at > org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1351) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767) > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > at org.mortbay.jetty.Server.handle(Server.java:326) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) > at > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) > at > org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) > Powered by Jetty:// -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14328) WebUI error 500 table doesn't exist
[ https://issues.apache.org/jira/browse/HBASE-14328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14328: Assignee: Gabor Liptak > WebUI error 500 table doesn't exist > --- > > Key: HBASE-14328 > URL: https://issues.apache.org/jira/browse/HBASE-14328 > Project: HBase > Issue Type: Bug > Components: Interface >Affects Versions: 1.1.0.1 >Reporter: Jean-Marc Spaggiari >Assignee: Gabor Liptak >Priority: Minor > Labels: beginner > Attachments: HBASE-14328.1.patch > > > When trying to load a non-existing table using the WebUI we get an ugly error > 500. Instead, we should display a nice "The table you are trying to reach > doesn't exit" message... > http://server.com:60010/table.jsp?name=TestTable > HTTP ERROR 500 > Problem accessing /table.jsp. Reason: > TestTable > Caused by: > org.apache.hadoop.hbase.TableNotFoundException: TestTable > at > org.apache.hadoop.hbase.client.HTable.getTableDescriptor(HTable.java:651) > at > org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:74) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) > at > org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1351) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767) > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > at org.mortbay.jetty.Server.handle(Server.java:326) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) > at > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) > at > org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) > Powered by Jetty:// -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16530) Reduce DBE code duplication
[ https://issues.apache.org/jira/browse/HBASE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475566#comment-15475566 ] Yu Li commented on HBASE-16530: --- Could see UT runs normally for post-commit now, thanks for the quick response and actions [~stack] [~appy] [~busbey] > Reduce DBE code duplication > --- > > Key: HBASE-16530 > URL: https://issues.apache.org/jira/browse/HBASE-16530 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: binlijin >Assignee: binlijin > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16530-master_V1.patch, > HBASE-16530-master_V2.patch, HBASE-16530-master_V3.patch, > HBASE-16530-master_V4.patch, HBASE-16530-master_V4.patch, > HBASE-16530.branch-1.v1.patch, HBASE-16530.branch-1.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16576) Shell add_peer doesn't allow setting cluster_key for custom endpoints
[ https://issues.apache.org/jira/browse/HBASE-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1547#comment-1547 ] Andrew Purtell commented on HBASE-16576: TestShell passes when this is applied to master and branch-1 (with a small fixup). Branch-1 patch also applies to 0.98 but the test fails there. Shall we stop at master and branch-1, or would you like to supply an 0.98 patch [~gjacoby] ? > Shell add_peer doesn't allow setting cluster_key for custom endpoints > - > > Key: HBASE-16576 > URL: https://issues.apache.org/jira/browse/HBASE-16576 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0, 1.1.5, 0.98.22 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Attachments: HBASE-16576.patch, HBASE-16576.v1.patch > > > The HBase shell allows a user to create a replication peer using the add_peer > method, which can take a peer id and a Ruby hash. It creates a > ReplicationPeerConfig and passes it through to the Java > ReplicationAdmin#addPeer. > The Ruby code makes an assumption that the Java API doesn't: that CLUSTER_KEY > and ENDPOINT_CLASSNAME are mutually exclusive. If both are specified, it > throws an error. If only ENDPOINT_CLASSNAME is set, the add_peer logic > derives a local dummy cluster key based on the local cluster's configuration. > CLUSTER_KEY shouldn't be required when an ENDPOINT_CLASSNAME is specified, > because a custom endpoint might not need it. The dummy default logic is fine. > > But if an endpoint does require a remote cluster key, it shouldn't be > forbidden to provide one, especially since the Java API permits it, and even > the custom replication endpoint Java tests rely on this. (See > TestReplicationEndpoint#testCustomReplicationEndpoint) > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16530) Reduce DBE code duplication
[ https://issues.apache.org/jira/browse/HBASE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475550#comment-15475550 ] Yu Li edited comment on HBASE-16530 at 9/9/16 1:24 AM: --- Reattach patch to retry HadoopQA for branch-1, failed cases in previous run should be irrelative to changes here. was (Author: carp84): Reattach patch to retry HadoopQA for branch-1 > Reduce DBE code duplication > --- > > Key: HBASE-16530 > URL: https://issues.apache.org/jira/browse/HBASE-16530 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: binlijin >Assignee: binlijin > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16530-master_V1.patch, > HBASE-16530-master_V2.patch, HBASE-16530-master_V3.patch, > HBASE-16530-master_V4.patch, HBASE-16530-master_V4.patch, > HBASE-16530.branch-1.v1.patch, HBASE-16530.branch-1.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16530) Reduce DBE code duplication
[ https://issues.apache.org/jira/browse/HBASE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-16530: -- Attachment: HBASE-16530.branch-1.v1.patch Reattach patch to retry HadoopQA for branch-1 > Reduce DBE code duplication > --- > > Key: HBASE-16530 > URL: https://issues.apache.org/jira/browse/HBASE-16530 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: binlijin >Assignee: binlijin > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16530-master_V1.patch, > HBASE-16530-master_V2.patch, HBASE-16530-master_V3.patch, > HBASE-16530-master_V4.patch, HBASE-16530-master_V4.patch, > HBASE-16530.branch-1.v1.patch, HBASE-16530.branch-1.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475546#comment-15475546 ] Hudson commented on HBASE-16589: FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #12 (See [https://builds.apache.org/job/HBase-1.3-JDK8/12/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev 493c31c297bbe2965ec295180cf627f9c7e01cec) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475543#comment-15475543 ] Hudson commented on HBASE-16589: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #22 (See [https://builds.apache.org/job/HBase-1.2-JDK8/22/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev 4611b15f177bf1ff5cfe0535e34e481a2da36b06) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475510#comment-15475510 ] Hudson commented on HBASE-16589: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1566 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1566/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev 46c756a4a7f2bbce7f9b276e413636eda3d86e30) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[ https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475492#comment-15475492 ] Duo Zhang commented on HBASE-16583: --- I'd say TPC is the dream but SEDA is the reality :) For C*, the first problem is how to deal with disk io. There is no AIO Filesystem implementation in Java, and after a learning of ScyllaDB, one of the TPC implementation in real world, I found that only zfs has a good support of AIO... So they still need a thread pool for disk io, that is still SEDA:) And we have the same problem...Although most of our IO is network based, we have short circuit read... And TPC will make the code very very flaky, a simple sleep or some other time consuming operation can kill all the server. For ScyllaDB, they have a really powerful framework to write TPC code, for example, a sleep in that framework does not hang the current thread, it equals to schedule a delayed task. And more, for some time consuming work, such as filtering or compaction, we need to cut timeslice and do scheduling by ourselves. The guys of ScyllaDB used to write KVM if I do not remember wrong, so maybe this is not a difficult mission for them. But for us, with Java, I will not say it is impossible, but... Will add more comments later. I need to go to work now... Thanks. > Staged Event-Driven Architecture > > > Key: HBASE-16583 > URL: https://issues.apache.org/jira/browse/HBASE-16583 > Project: HBase > Issue Type: Umbrella >Reporter: Phil Yang > > Staged Event-Driven Architecture (SEDA) splits request-handling logic into > several stages, each stage is executed in a thread pool and they are > connected by queues. > Currently, in region server we use a thread pool to handle requests from > client. The number of handlers is configurable, reading and writing use > different pools. The current architecture has two limitations: > Performance: > Different part of the handling path has different bottleneck. For example, > accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly > consumes network/disk IO. If we use SEDA and split them into two different > stages, we can use different numbers for two pools according to the > CPU/disk/network performance case by case. > Availability: > HBASE-16388 described a scene that if the client use a thread pool and use > blocking methods to access region servers, only one slow server may exhaust > most of threads of the client. For HBase, we are the client and HDFS > datanodes are the servers. A slow datanode may exhaust most of handlers. The > best way to resolve this issue is make HDFS requests non-blocking, which is > exactly what SEDA does. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[ https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475486#comment-15475486 ] Dave Latham commented on HBASE-16583: - >From the last comment on CASSANDRA-8520, it looks like it was resolved simply >due to being superseded by CASSANDRA-10989 as the effort to move away from >SEDA. > Staged Event-Driven Architecture > > > Key: HBASE-16583 > URL: https://issues.apache.org/jira/browse/HBASE-16583 > Project: HBase > Issue Type: Umbrella >Reporter: Phil Yang > > Staged Event-Driven Architecture (SEDA) splits request-handling logic into > several stages, each stage is executed in a thread pool and they are > connected by queues. > Currently, in region server we use a thread pool to handle requests from > client. The number of handlers is configurable, reading and writing use > different pools. The current architecture has two limitations: > Performance: > Different part of the handling path has different bottleneck. For example, > accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly > consumes network/disk IO. If we use SEDA and split them into two different > stages, we can use different numbers for two pools according to the > CPU/disk/network performance case by case. > Availability: > HBASE-16388 described a scene that if the client use a thread pool and use > blocking methods to access region servers, only one slow server may exhaust > most of threads of the client. For HBase, we are the client and HDFS > datanodes are the servers. A slow datanode may exhaust most of handlers. The > best way to resolve this issue is make HDFS requests non-blocking, which is > exactly what SEDA does. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475475#comment-15475475 ] Ted Yu commented on HBASE-14123: >From >https://builds.apache.org/job/PreCommit-HBASE-Build/3463/artifact/patchprocess/patch-unit-hbase-server.txt > : Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.742 sec - in org.apache.hadoop.hbase.backup.TestFullBackup Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.174 sec - in org.apache.hadoop.hbase.backup.TestBackupSystemTable > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v2.txt, 14123-master.v3.txt, 14123-master.v5.txt, > 14123-master.v6.txt, 14123-master.v7.txt, 14123-master.v8.txt, > 14123-master.v9.txt, 14123-v14.txt, HBASE-14123-for-7912-v1.patch, > HBASE-14123-for-7912-v6.patch, HBASE-14123-v1.patch, HBASE-14123-v10.patch, > HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v13.patch, > HBASE-14123-v15.patch, HBASE-14123-v16.patch, HBASE-14123-v2.patch, > HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, > HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475464#comment-15475464 ] Hudson commented on HBASE-16589: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1782 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1782/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev ff914375f6954dc78760edb049c9cb9f86e7aa83) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[ https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475456#comment-15475456 ] Enis Soztutar commented on HBASE-16583: --- This is a pretty good point. I think [~stack] experimented with RPC reader threads directly executing the call instead of using the RPC queues, and saw dramatic improvements. Adding more thread pools and queues in the mix will only increase latency and decrease throughput. See https://issues.apache.org/jira/browse/HBASE-15967?focusedCommentId=15316448=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15316448 > Staged Event-Driven Architecture > > > Key: HBASE-16583 > URL: https://issues.apache.org/jira/browse/HBASE-16583 > Project: HBase > Issue Type: Umbrella >Reporter: Phil Yang > > Staged Event-Driven Architecture (SEDA) splits request-handling logic into > several stages, each stage is executed in a thread pool and they are > connected by queues. > Currently, in region server we use a thread pool to handle requests from > client. The number of handlers is configurable, reading and writing use > different pools. The current architecture has two limitations: > Performance: > Different part of the handling path has different bottleneck. For example, > accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly > consumes network/disk IO. If we use SEDA and split them into two different > stages, we can use different numbers for two pools according to the > CPU/disk/network performance case by case. > Availability: > HBASE-16388 described a scene that if the client use a thread pool and use > blocking methods to access region servers, only one slow server may exhaust > most of threads of the client. For HBase, we are the client and HDFS > datanodes are the servers. A slow datanode may exhaust most of handlers. The > best way to resolve this issue is make HDFS requests non-blocking, which is > exactly what SEDA does. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14123: --- Attachment: 14123-master.v18.txt Only difference between v17 and v18 is that the dead store to status is removed. I ran the two backup related tests locally and they passed. > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: 14123-master.v14.txt, 14123-master.v15.txt, > 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, > 14123-master.v2.txt, 14123-master.v3.txt, 14123-master.v5.txt, > 14123-master.v6.txt, 14123-master.v7.txt, 14123-master.v8.txt, > 14123-master.v9.txt, 14123-v14.txt, HBASE-14123-for-7912-v1.patch, > HBASE-14123-for-7912-v6.patch, HBASE-14123-v1.patch, HBASE-14123-v10.patch, > HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v13.patch, > HBASE-14123-v15.patch, HBASE-14123-v16.patch, HBASE-14123-v2.patch, > HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, > HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16576) Shell add_peer doesn't allow setting cluster_key for custom endpoints
[ https://issues.apache.org/jira/browse/HBASE-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475433#comment-15475433 ] Enis Soztutar commented on HBASE-16576: --- Makes sense. Here is the code that does this: ReplicationPeerZKImpl.getPeerConf(). We split the clusterKey into 3 pieces, and manually set the zk quorum and root znode in the cloned configuration for the peer. Then we do a compound configuration from the base one + ReplicationPeerConfig.getConfiguration() entries. This was designed to be the way to pass custom configuration (like hbase-site.xml properties) per peer. For example, you can have different rpc timeouts, retries etc per peer if you manually set it in the peer config. > Shell add_peer doesn't allow setting cluster_key for custom endpoints > - > > Key: HBASE-16576 > URL: https://issues.apache.org/jira/browse/HBASE-16576 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0, 1.1.5, 0.98.22 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Attachments: HBASE-16576.patch, HBASE-16576.v1.patch > > > The HBase shell allows a user to create a replication peer using the add_peer > method, which can take a peer id and a Ruby hash. It creates a > ReplicationPeerConfig and passes it through to the Java > ReplicationAdmin#addPeer. > The Ruby code makes an assumption that the Java API doesn't: that CLUSTER_KEY > and ENDPOINT_CLASSNAME are mutually exclusive. If both are specified, it > throws an error. If only ENDPOINT_CLASSNAME is set, the add_peer logic > derives a local dummy cluster key based on the local cluster's configuration. > CLUSTER_KEY shouldn't be required when an ENDPOINT_CLASSNAME is specified, > because a custom endpoint might not need it. The dummy default logic is fine. > > But if an endpoint does require a remote cluster key, it shouldn't be > forbidden to provide one, especially since the Java API permits it, and even > the custom replication endpoint Java tests rely on this. (See > TestReplicationEndpoint#testCustomReplicationEndpoint) > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16570) Compute region locality in parallel at startup
[ https://issues.apache.org/jira/browse/HBASE-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-16570: - Attachment: HBASE-16570-master_V4.patch > Compute region locality in parallel at startup > -- > > Key: HBASE-16570 > URL: https://issues.apache.org/jira/browse/HBASE-16570 > Project: HBase > Issue Type: Sub-task >Reporter: binlijin >Assignee: binlijin > Attachments: HBASE-16570-master_V1.patch, > HBASE-16570-master_V2.patch, HBASE-16570-master_V3.patch, > HBASE-16570-master_V4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16309) TestDefaultCompactSelection.testCompactionRatio is flaky
[ https://issues.apache.org/jira/browse/HBASE-16309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475409#comment-15475409 ] Matteo Bertozzi commented on HBASE-16309: - +1, patch looks good and seems to solve the flakiness for me. (maybe instead of the sleep we can do injectEdge() to move the time forward) > TestDefaultCompactSelection.testCompactionRatio is flaky > > > Key: HBASE-16309 > URL: https://issues.apache.org/jira/browse/HBASE-16309 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-16309.patch > > > The aged major compaction condition is not stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475390#comment-15475390
]
Hadoop QA commented on HBASE-14123:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 7m 32s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 6s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 10s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 46 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 32s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 21s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 47s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 15m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
16s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 57s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 4s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 14s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 43s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 43s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 38s {color}
| {color:red} hbase-server-jdk1.7.0_111 with JDK v1.7.0_111 generated 2 new + 4
unchanged - 2 fixed = 6 total (was 6) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 50s {color}
| {color:red} root-jdk1.7.0_111 with JDK v1.7.0_111 generated 2 new + 30
unchanged - 2 fixed = 32 total (was 32) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 17m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
5s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s
{color} | {color:red} The patch has 540 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 16s
{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
[jira] [Commented] (HBASE-16570) Compute region locality in parallel at startup
[ https://issues.apache.org/jira/browse/HBASE-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475380#comment-15475380 ] binlijin commented on HBASE-16570: -- OK > Compute region locality in parallel at startup > -- > > Key: HBASE-16570 > URL: https://issues.apache.org/jira/browse/HBASE-16570 > Project: HBase > Issue Type: Sub-task >Reporter: binlijin >Assignee: binlijin > Attachments: HBASE-16570-master_V1.patch, > HBASE-16570-master_V2.patch, HBASE-16570-master_V3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16583) Staged Event-Driven Architecture
[ https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475377#comment-15475377 ] binlijin edited comment on HBASE-16583 at 9/8/16 11:49 PM: --- Yes，they have the same concerns on SEDA, and do not have too much progress on the new model "TPC( thread per core)". Just for reference. was (Author: aoxiang): Yes，they have the same concerns on SEDA, and do not have too much progress on the new model "TPC( thread per core)". > Staged Event-Driven Architecture > > > Key: HBASE-16583 > URL: https://issues.apache.org/jira/browse/HBASE-16583 > Project: HBase > Issue Type: Umbrella >Reporter: Phil Yang > > Staged Event-Driven Architecture (SEDA) splits request-handling logic into > several stages, each stage is executed in a thread pool and they are > connected by queues. > Currently, in region server we use a thread pool to handle requests from > client. The number of handlers is configurable, reading and writing use > different pools. The current architecture has two limitations: > Performance: > Different part of the handling path has different bottleneck. For example, > accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly > consumes network/disk IO. If we use SEDA and split them into two different > stages, we can use different numbers for two pools according to the > CPU/disk/network performance case by case. > Availability: > HBASE-16388 described a scene that if the client use a thread pool and use > blocking methods to access region servers, only one slow server may exhaust > most of threads of the client. For HBase, we are the client and HDFS > datanodes are the servers. A slow datanode may exhaust most of handlers. The > best way to resolve this issue is make HDFS requests non-blocking, which is > exactly what SEDA does. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[ https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475377#comment-15475377 ] binlijin commented on HBASE-16583: -- Yes，they have the same concerns on SEDA, and do not have too much progress on the new model "TPC( thread per core)". > Staged Event-Driven Architecture > > > Key: HBASE-16583 > URL: https://issues.apache.org/jira/browse/HBASE-16583 > Project: HBase > Issue Type: Umbrella >Reporter: Phil Yang > > Staged Event-Driven Architecture (SEDA) splits request-handling logic into > several stages, each stage is executed in a thread pool and they are > connected by queues. > Currently, in region server we use a thread pool to handle requests from > client. The number of handlers is configurable, reading and writing use > different pools. The current architecture has two limitations: > Performance: > Different part of the handling path has different bottleneck. For example, > accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly > consumes network/disk IO. If we use SEDA and split them into two different > stages, we can use different numbers for two pools according to the > CPU/disk/network performance case by case. > Availability: > HBASE-16388 described a scene that if the client use a thread pool and use > blocking methods to access region servers, only one slow server may exhaust > most of threads of the client. For HBase, we are the client and HDFS > datanodes are the servers. A slow datanode may exhaust most of handlers. The > best way to resolve this issue is make HDFS requests non-blocking, which is > exactly what SEDA does. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15165) AsyncProcess can spin wait indefinitly
[ https://issues.apache.org/jira/browse/HBASE-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengyue Li updated HBASE-15165: --- Attachment: HBASE-15165.004.patch Trigger test for client side change HBASE-15165 > AsyncProcess can spin wait indefinitly > -- > > Key: HBASE-15165 > URL: https://issues.apache.org/jira/browse/HBASE-15165 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Pengyue Li >Priority: Critical > Attachments: HBASE-15165.001.patch, HBASE-15165.002.patch, > HBASE-15165.003.patch, HBASE-15165.004.patch > > > When the max outstanding requests per region or per server is reached, all > threads trying to send more requests to that server will spin and will spin > forever with no sleep, and no regard for timeouts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15165) AsyncProcess can spin wait indefinitly
[ https://issues.apache.org/jira/browse/HBASE-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengyue Li updated HBASE-15165: --- Attachment: HBASE-15165.003.patch Trigger test for client side change HBASE-15165 > AsyncProcess can spin wait indefinitly > -- > > Key: HBASE-15165 > URL: https://issues.apache.org/jira/browse/HBASE-15165 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Pengyue Li >Priority: Critical > Attachments: HBASE-15165.001.patch, HBASE-15165.002.patch, > HBASE-15165.003.patch > > > When the max outstanding requests per region or per server is reached, all > threads trying to send more requests to that server will spin and will spin > forever with no sleep, and no regard for timeouts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475331#comment-15475331 ] Hudson commented on HBASE-16589: FAILURE: Integrated in Jenkins build HBase-1.1-JDK8 #1867 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1867/]) HBASE-16589 Adjust log level for FATAL messages from (apurtell: rev ff914375f6954dc78760edb049c9cb9f86e7aa83) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16345) RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions
[ https://issues.apache.org/jira/browse/HBASE-16345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-16345: - Attachment: HBASE-16345.master.005.patch > RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer > Exceptions > -- > > Key: HBASE-16345 > URL: https://issues.apache.org/jira/browse/HBASE-16345 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16345-v001.patch, HBASE-16345.master.001.patch, > HBASE-16345.master.002.patch, HBASE-16345.master.003.patch, > HBASE-16345.master.004.patch, HBASE-16345.master.005.patch > > > Update for the description. Debugged more at this front based on the comments > from Enis. > The cause is that for the primary replica, if its retry is exhausted too > fast, f.get() [1] returns ExecutionException. This Exception needs to be > ignored and continue with the replicas. > The other issue is that after adding calls for the replicas, if the first > completed task gets ExecutionException (due to the retry exhausted), it > throws the exception to the client[2]. > In this case, it needs to loop through these tasks, waiting for the success > one. If no one succeeds, throw exception. > Similar for the scan as well > [1] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L197 > [2] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L219 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16588) Remove ConnectionFactory#createConnection from backup / restore server code
[
https://issues.apache.org/jira/browse/HBASE-16588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-16588.
Resolution: Fixed
Hadoop Flags: Reviewed
All backup tests passed.
Thanks for the review, Stephen.
> Remove ConnectionFactory#createConnection from backup / restore server code
> ---
>
> Key: HBASE-16588
> URL: https://issues.apache.org/jira/browse/HBASE-16588
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
> Attachments: 16588.v1.txt
>
>
> Currently there are several places in backup / restore server code where
> Connection is created:
> {code}
> try (Connection conn = ConnectionFactory.createConnection(conf);
> {code}
> This should not be necessary - we can retrieve Connection from environment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[ https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475247#comment-15475247 ] Ted Yu commented on HBASE-16583: The above JIRA has been inactive for more than half year. CASSANDRA-8520, referenced by the JIRA, was marked Won't Fix. > Staged Event-Driven Architecture > > > Key: HBASE-16583 > URL: https://issues.apache.org/jira/browse/HBASE-16583 > Project: HBase > Issue Type: Umbrella >Reporter: Phil Yang > > Staged Event-Driven Architecture (SEDA) splits request-handling logic into > several stages, each stage is executed in a thread pool and they are > connected by queues. > Currently, in region server we use a thread pool to handle requests from > client. The number of handlers is configurable, reading and writing use > different pools. The current architecture has two limitations: > Performance: > Different part of the handling path has different bottleneck. For example, > accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly > consumes network/disk IO. If we use SEDA and split them into two different > stages, we can use different numbers for two pools according to the > CPU/disk/network performance case by case. > Availability: > HBASE-16388 described a scene that if the client use a thread pool and use > blocking methods to access region servers, only one slow server may exhaust > most of threads of the client. For HBase, we are the client and HDFS > datanodes are the servers. A slow datanode may exhaust most of handlers. The > best way to resolve this issue is make HDFS requests non-blocking, which is > exactly what SEDA does. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16576) Shell add_peer doesn't allow setting cluster_key for custom endpoints
[ https://issues.apache.org/jira/browse/HBASE-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475144#comment-15475144 ] Andrew Purtell commented on HBASE-16576: I will commit this later today after poking around with it for a bit > Shell add_peer doesn't allow setting cluster_key for custom endpoints > - > > Key: HBASE-16576 > URL: https://issues.apache.org/jira/browse/HBASE-16576 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0, 1.1.5, 0.98.22 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Attachments: HBASE-16576.patch, HBASE-16576.v1.patch > > > The HBase shell allows a user to create a replication peer using the add_peer > method, which can take a peer id and a Ruby hash. It creates a > ReplicationPeerConfig and passes it through to the Java > ReplicationAdmin#addPeer. > The Ruby code makes an assumption that the Java API doesn't: that CLUSTER_KEY > and ENDPOINT_CLASSNAME are mutually exclusive. If both are specified, it > throws an error. If only ENDPOINT_CLASSNAME is set, the add_peer logic > derives a local dummy cluster key based on the local cluster's configuration. > CLUSTER_KEY shouldn't be required when an ENDPOINT_CLASSNAME is specified, > because a custom endpoint might not need it. The dummy default logic is fine. > > But if an endpoint does require a remote cluster key, it shouldn't be > forbidden to provide one, especially since the Java API permits it, and even > the custom replication endpoint Java tests rely on this. (See > TestReplicationEndpoint#testCustomReplicationEndpoint) > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16576) Shell add_peer doesn't allow setting cluster_key for custom endpoints
[ https://issues.apache.org/jira/browse/HBASE-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475141#comment-15475141 ] Andrew Purtell commented on HBASE-16576: +1 bq. Others are left unaddressed – for example, it's complaining about the cyclomatic complexity being too high in replication_admin, The blame for this is not the proposed changes > Shell add_peer doesn't allow setting cluster_key for custom endpoints > - > > Key: HBASE-16576 > URL: https://issues.apache.org/jira/browse/HBASE-16576 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0, 1.1.5, 0.98.22 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby > Attachments: HBASE-16576.patch, HBASE-16576.v1.patch > > > The HBase shell allows a user to create a replication peer using the add_peer > method, which can take a peer id and a Ruby hash. It creates a > ReplicationPeerConfig and passes it through to the Java > ReplicationAdmin#addPeer. > The Ruby code makes an assumption that the Java API doesn't: that CLUSTER_KEY > and ENDPOINT_CLASSNAME are mutually exclusive. If both are specified, it > throws an error. If only ENDPOINT_CLASSNAME is set, the add_peer logic > derives a local dummy cluster key based on the local cluster's configuration. > CLUSTER_KEY shouldn't be required when an ENDPOINT_CLASSNAME is specified, > because a custom endpoint might not need it. The dummy default logic is fine. > > But if an endpoint does require a remote cluster key, it shouldn't be > forbidden to provide one, especially since the Java API permits it, and even > the custom replication endpoint Java tests rely on this. (See > TestReplicationEndpoint#testCustomReplicationEndpoint) > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16590) TestRestoreSnapshotFromClientWithRegionReplicas needs to use scan with TIMELINE consistency to countRows
[
https://issues.apache.org/jira/browse/HBASE-16590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475137#comment-15475137
]
Hadoop QA commented on HBASE-16590:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 42s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 92m 10s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 135m 40s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-08 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12827634/HBASE-16590-master-v001.patch
|
| JIRA Issue | HBASE-16590 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6e3f68f5b22b 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d7e5c6f |
| Default Java | 1.7.0_111 |
| Multi-JDK versions |
[jira] [Commented] (HBASE-16583) Staged Event-Driven Architecture
[
https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475134#comment-15475134
]
Andrew Purtell commented on HBASE-16583:
One of the most common and classic critique of the SEDA architecture, by the
original proponent of the idea as well as others, is the overhead of connecting
stages through event queues lowers the ceiling for performance. Consider
http://matt-welsh.blogspot.com/2010/07/retrospective-on-seda.html .
{quote}
If I were to design SEDA today, I would decouple stages (i.e., code modules)
from queues and thread pools (i.e., concurrency boundaries). Stages are still
useful as a structuring primitive, but it is probably best to group multiple
stages within a single "thread pool domain" where latency is critical. Most
stages should be connected via direct function call. I would only put a
separate thread pool and queue in front of a group of stages that have long
latency or nondeterministic runtime, such as performing disk I/O.
{quote}
and note he no longer considers the benchmarks that validated the original SEDA
paper as actually supporting application of SEDA for real-world applications.
I'm not saying don't go in this direction, but let's not assume the payoff will
be automatic.
> Staged Event-Driven Architecture
>
>
> Key: HBASE-16583
> URL: https://issues.apache.org/jira/browse/HBASE-16583
> Project: HBase
> Issue Type: Umbrella
>Reporter: Phil Yang
>
> Staged Event-Driven Architecture (SEDA) splits request-handling logic into
> several stages, each stage is executed in a thread pool and they are
> connected by queues.
> Currently, in region server we use a thread pool to handle requests from
> client. The number of handlers is configurable, reading and writing use
> different pools. The current architecture has two limitations:
> Performance:
> Different part of the handling path has different bottleneck. For example,
> accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly
> consumes network/disk IO. If we use SEDA and split them into two different
> stages, we can use different numbers for two pools according to the
> CPU/disk/network performance case by case.
> Availability:
> HBASE-16388 described a scene that if the client use a thread pool and use
> blocking methods to access region servers, only one slow server may exhaust
> most of threads of the client. For HBase, we are the client and HDFS
> datanodes are the servers. A slow datanode may exhaust most of handlers. The
> best way to resolve this issue is make HDFS requests non-blocking, which is
> exactly what SEDA does.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16505) Add AsyncRegion interface to pass deadline and support async operations
[ https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475133#comment-15475133 ] Hudson commented on HBASE-16505: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1565 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1565/]) HBASE-16505 Add AsyncRegion interface to pass deadline and support async (tedyu: rev d7e5c6fa8e9c562d1d66b2da0edb5831b51fe013) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcControllerImpl.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/OperationListener.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionOperationContext.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SynchronousOperationListener.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcController.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AsyncRegion.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DelegatingHBaseRpcController.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java > Add AsyncRegion interface to pass deadline and support async operations > --- > > Key: HBASE-16505 > URL: https://issues.apache.org/jira/browse/HBASE-16505 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0 > > Attachments: HBASE-16505-v1.patch, HBASE-16505-v10.patch, > HBASE-16505-v10.patch, HBASE-16505-v2.patch, HBASE-16505-v3.patch, > HBASE-16505-v4.patch, HBASE-16505-v5.patch, HBASE-16505-v6.patch, > HBASE-16505-v7.patch, HBASE-16505-v8.patch, HBASE-16505-v9.patch > > > If we want to know the correct setting of timeout in read/write path, we need > add a new parameter in operation-methods of Region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16577) Toward a more useful slow query log
[ https://issues.apache.org/jira/browse/HBASE-16577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16577: --- Summary: Toward a more useful slow query log (was: On reducing the log level of *TooSlow log lines) > Toward a more useful slow query log > --- > > Key: HBASE-16577 > URL: https://issues.apache.org/jira/browse/HBASE-16577 > Project: HBase > Issue Type: Brainstorming >Reporter: Andrew Purtell > > I was looking at the nature and distribution of responseTooSlow messages on > our clusters. The majority of responseTooSlow warnings are for Scan or Multi. > Scans may take a long time to return. It will totally depend on how much data > is in the table, how the data is distributed, and the range and selectivity > of the query. We are not measuring response time in a way to know what is > proportionate to the work requested. Another problematic example is Multi. We > don't get valid results considering a multi with 1 op and a multi with 100 > ops to be equivalent as far as being "too slow". > If we aggressively filter responseTooSlow messages to just include the ops we > can expect to be small, constant, request-independent units of work, this > leaves us with Get and Mutate. This gives us no more information then we get > from the Canary with read and write checks turned on. The Canary issues Get > and Mutate ops and, better, measures availability and latency from the client > perspective. > > Where I end up is I think my shop should ignore responseTooSlow as signal as > being far too noisy. The trouble then is it is logged at WARN level. This > implies there is something wrong that needs to be fixed. That may not be the > case. It's going to require some analysis of the application and the request > particulars extracted from the log line. WARN seems inappropriate for this > type of indication. There's nothing (necessarily) wrong with HBase, or the > app. Should be logged at more like INFO. > Furthermore the response might not be too slow, so calling it > "responseTooSlow" isn't quite right. More like "responseMaybeSlow" :-) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16577) On reducing the log level of *TooSlow log lines
[ https://issues.apache.org/jira/browse/HBASE-16577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475114#comment-15475114 ] Andrew Purtell commented on HBASE-16577: bq. So IMO we'd better just keep the real slow request (Get, mutation) as 'WARN' Level, ignore the uncertain ones (If we can't distinguish them, just remove them from 'WARN' level). The answer to the question 'What is a "real" slow request?' is "It depends", though. bq. Maybe have different limits to scan vs get vs multi? This is a generalization of the above suggestion. So we would have new configuration that sets a threshold for every op type, and one as a catch-all for op types that don't have a specific threshold set? Only warn if above the threshold? Default for < branch-2 is 1, a super small threshold leading to warns on anything like today. Default for >= branch-2 is 0, meaning no warn. > On reducing the log level of *TooSlow log lines > --- > > Key: HBASE-16577 > URL: https://issues.apache.org/jira/browse/HBASE-16577 > Project: HBase > Issue Type: Brainstorming >Reporter: Andrew Purtell > > I was looking at the nature and distribution of responseTooSlow messages on > our clusters. The majority of responseTooSlow warnings are for Scan or Multi. > Scans may take a long time to return. It will totally depend on how much data > is in the table, how the data is distributed, and the range and selectivity > of the query. We are not measuring response time in a way to know what is > proportionate to the work requested. Another problematic example is Multi. We > don't get valid results considering a multi with 1 op and a multi with 100 > ops to be equivalent as far as being "too slow". > If we aggressively filter responseTooSlow messages to just include the ops we > can expect to be small, constant, request-independent units of work, this > leaves us with Get and Mutate. This gives us no more information then we get > from the Canary with read and write checks turned on. The Canary issues Get > and Mutate ops and, better, measures availability and latency from the client > perspective. > > Where I end up is I think my shop should ignore responseTooSlow as signal as > being far too noisy. The trouble then is it is logged at WARN level. This > implies there is something wrong that needs to be fixed. That may not be the > case. It's going to require some analysis of the application and the request > particulars extracted from the log line. WARN seems inappropriate for this > type of indication. There's nothing (necessarily) wrong with HBase, or the > app. Should be logged at more like INFO. > Furthermore the response might not be too slow, so calling it > "responseTooSlow" isn't quite right. More like "responseMaybeSlow" :-) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16577) On reducing the log level of *TooSlow log lines
[ https://issues.apache.org/jira/browse/HBASE-16577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475115#comment-15475115 ] Andrew Purtell commented on HBASE-16577: bq. So IMO we'd better just keep the real slow request (Get, mutation) as 'WARN' Level, ignore the uncertain ones (If we can't distinguish them, just remove them from 'WARN' level). The answer to the question 'What is a "real" slow request?' is "It depends", though. bq. Maybe have different limits to scan vs get vs multi? This is a generalization of the above suggestion. So we would have new configuration that sets a threshold for every op type, and one as a catch-all for op types that don't have a specific threshold set? Only warn if above the threshold? Default for < branch-2 is 1, a super small threshold leading to warns on anything like today. Default for >= branch-2 is 0, meaning no warn. > On reducing the log level of *TooSlow log lines > --- > > Key: HBASE-16577 > URL: https://issues.apache.org/jira/browse/HBASE-16577 > Project: HBase > Issue Type: Brainstorming >Reporter: Andrew Purtell > > I was looking at the nature and distribution of responseTooSlow messages on > our clusters. The majority of responseTooSlow warnings are for Scan or Multi. > Scans may take a long time to return. It will totally depend on how much data > is in the table, how the data is distributed, and the range and selectivity > of the query. We are not measuring response time in a way to know what is > proportionate to the work requested. Another problematic example is Multi. We > don't get valid results considering a multi with 1 op and a multi with 100 > ops to be equivalent as far as being "too slow". > If we aggressively filter responseTooSlow messages to just include the ops we > can expect to be small, constant, request-independent units of work, this > leaves us with Get and Mutate. This gives us no more information then we get > from the Canary with read and write checks turned on. The Canary issues Get > and Mutate ops and, better, measures availability and latency from the client > perspective. > > Where I end up is I think my shop should ignore responseTooSlow as signal as > being far too noisy. The trouble then is it is logged at WARN level. This > implies there is something wrong that needs to be fixed. That may not be the > case. It's going to require some analysis of the application and the request > particulars extracted from the log line. WARN seems inappropriate for this > type of indication. There's nothing (necessarily) wrong with HBase, or the > app. Should be logged at more like INFO. > Furthermore the response might not be too slow, so calling it > "responseTooSlow" isn't quite right. More like "responseMaybeSlow" :-) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16345) RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer Exceptions
[ https://issues.apache.org/jira/browse/HBASE-16345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475108#comment-15475108 ] huaxiang sun commented on HBASE-16345: -- HBASE-16590 is invalid so I reviewed ScannerCallableWithReplicas's code. Found that Consistenty.STRONG is not handled in a specific case which caused unittest failure. Addressed that and a new patch will be uploaded soon. > RpcRetryingCallerWithReadReplicas#call() should catch some RegionServer > Exceptions > -- > > Key: HBASE-16345 > URL: https://issues.apache.org/jira/browse/HBASE-16345 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16345-v001.patch, HBASE-16345.master.001.patch, > HBASE-16345.master.002.patch, HBASE-16345.master.003.patch, > HBASE-16345.master.004.patch > > > Update for the description. Debugged more at this front based on the comments > from Enis. > The cause is that for the primary replica, if its retry is exhausted too > fast, f.get() [1] returns ExecutionException. This Exception needs to be > ignored and continue with the replicas. > The other issue is that after adding calls for the replicas, if the first > completed task gets ExecutionException (due to the retry exhausted), it > throws the exception to the client[2]. > In this case, it needs to loop through these tasks, waiting for the success > one. If no one succeeds, throw exception. > Similar for the scan as well > [1] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L197 > [2] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L219 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16191) Add stop_regionserver and stop_master to shell
[ https://issues.apache.org/jira/browse/HBASE-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475068#comment-15475068 ] Andrew Purtell commented on HBASE-16191: This seems useful. I'll carry it to the finish line if you'd like [~enis] > Add stop_regionserver and stop_master to shell > -- > > Key: HBASE-16191 > URL: https://issues.apache.org/jira/browse/HBASE-16191 > Project: HBase > Issue Type: Improvement >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.4.0 > > Attachments: hbase-16191_v1.patch > > > Admin.stopMaster() and Admin.stopRegionServer() is missing in shell. I needed > that for testing something. Seems useful to add. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[ https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16589: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks [~enis] Pushed to all branches > Adjust log level for FATAL messages from HBaseReplicationEndpoint that are > not fatal > > > Key: HBASE-16589 > URL: https://issues.apache.org/jira/browse/HBASE-16589 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16589.patch > > > HBaseReplicationEndpoint logs at FATAL level some messages that are not > actually fatal. Drop to ERROR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16590) TestRestoreSnapshotFromClientWithRegionReplicas needs to use scan with TIMELINE consistency to countRows
[ https://issues.apache.org/jira/browse/HBASE-16590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun resolved HBASE-16590. -- Resolution: Invalid According to Matteo, the original one is valid. > TestRestoreSnapshotFromClientWithRegionReplicas needs to use scan with > TIMELINE consistency to countRows > > > Key: HBASE-16590 > URL: https://issues.apache.org/jira/browse/HBASE-16590 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-16590-master-v001.patch > > > TestRestoreSnapshotFromClientWithRegionReplicas uses Consistency.STRONG when > doing a scan for counting rows. This is not right as it will not send out > scan requests to replicas. It caused issue when read replica logic is > corrected in HBASE-16345. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16590) TestRestoreSnapshotFromClientWithRegionReplicas needs to use scan with TIMELINE consistency to countRows
[ https://issues.apache.org/jira/browse/HBASE-16590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-16590: - Status: Open (was: Patch Available) > TestRestoreSnapshotFromClientWithRegionReplicas needs to use scan with > TIMELINE consistency to countRows > > > Key: HBASE-16590 > URL: https://issues.apache.org/jira/browse/HBASE-16590 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-16590-master-v001.patch > > > TestRestoreSnapshotFromClientWithRegionReplicas uses Consistency.STRONG when > doing a scan for counting rows. This is not right as it will not send out > scan requests to replicas. It caused issue when read replica logic is > corrected in HBASE-16345. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16589) Adjust log level for FATAL messages from HBaseReplicationEndpoint that are not fatal
[
https://issues.apache.org/jira/browse/HBASE-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474980#comment-15474980
]
Hadoop QA commented on HBASE-16589:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 4s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 52s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 139m 15s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-08 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12827624/HBASE-16589.patch |
| JIRA Issue | HBASE-16589 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e93d82d64c83 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Updated] (HBASE-16465) Disable region splits and merges, balancer during full backup
[ https://issues.apache.org/jira/browse/HBASE-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-16465: -- Attachment: HBASE-16465-v6.patch rebased patch > Disable region splits and merges, balancer during full backup > - > > Key: HBASE-16465 > URL: https://issues.apache.org/jira/browse/HBASE-16465 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16465-v1.patch, HBASE-16465-v2.patch, > HBASE-16465-v3.patch, HBASE-16465-v4.patch, HBASE-16465-v5.patch, > HBASE-16465-v6.patch > > > Incorporate HBASE-15128 > Balancer, catalog janitor and region normalizer should be disabled as well > during full backup -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16465) Disable region splits and merges, balancer during full backup
[
https://issues.apache.org/jira/browse/HBASE-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474940#comment-15474940
]
Vladimir Rodionov commented on HBASE-16465:
---
{quote}
See HBaseFSCK#setMasterInMaintenanceMode(). Stephen Yuan Jiang what do you
think?
{quote}
This code is not in HBASE-7912 yet. I will postpone this JIRA until master
merge is complete.
> Disable region splits and merges, balancer during full backup
> -
>
> Key: HBASE-16465
> URL: https://issues.apache.org/jira/browse/HBASE-16465
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Labels: backup
> Attachments: HBASE-16465-v1.patch, HBASE-16465-v2.patch,
> HBASE-16465-v3.patch, HBASE-16465-v4.patch, HBASE-16465-v5.patch
>
>
> Incorporate HBASE-15128
> Balancer, catalog janitor and region normalizer should be disabled as well
> during full backup
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16588) Remove ConnectionFactory#createConnection from backup / restore server code
[
https://issues.apache.org/jira/browse/HBASE-16588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474919#comment-15474919
]
Stephen Yuan Jiang commented on HBASE-16588:
LGTM
> Remove ConnectionFactory#createConnection from backup / restore server code
> ---
>
> Key: HBASE-16588
> URL: https://issues.apache.org/jira/browse/HBASE-16588
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
> Attachments: 16588.v1.txt
>
>
> Currently there are several places in backup / restore server code where
> Connection is created:
> {code}
> try (Connection conn = ConnectionFactory.createConnection(conf);
> {code}
> This should not be necessary - we can retrieve Connection from environment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16588) Remove ConnectionFactory#createConnection from backup / restore server code
[
https://issues.apache.org/jira/browse/HBASE-16588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16588:
---
Attachment: 16588.v1.txt
> Remove ConnectionFactory#createConnection from backup / restore server code
> ---
>
> Key: HBASE-16588
> URL: https://issues.apache.org/jira/browse/HBASE-16588
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
> Attachments: 16588.v1.txt
>
>
> Currently there are several places in backup / restore server code where
> Connection is created:
> {code}
> try (Connection conn = ConnectionFactory.createConnection(conf);
> {code}
> This should not be necessary - we can retrieve Connection from environment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16590) TestRestoreSnapshotFromClientWithRegionReplicas needs to use scan with TIMELINE consistency to countRows
[ https://issues.apache.org/jira/browse/HBASE-16590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474886#comment-15474886 ] huaxiang sun commented on HBASE-16590: -- Sorry, Matteo just raised a comment that it needs to be STRONG to get the updated counts. So this may not be a valid fix. > TestRestoreSnapshotFromClientWithRegionReplicas needs to use scan with > TIMELINE consistency to countRows > > > Key: HBASE-16590 > URL: https://issues.apache.org/jira/browse/HBASE-16590 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-16590-master-v001.patch > > > TestRestoreSnapshotFromClientWithRegionReplicas uses Consistency.STRONG when > doing a scan for counting rows. This is not right as it will not send out > scan requests to replicas. It caused issue when read replica logic is > corrected in HBASE-16345. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16497) Add test for backup / restore involving MOB table
[ https://issues.apache.org/jira/browse/HBASE-16497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-16497. Resolution: Fixed Hadoop Flags: Reviewed Thanks for the review, Vlad. > Add test for backup / restore involving MOB table > - > > Key: HBASE-16497 > URL: https://issues.apache.org/jira/browse/HBASE-16497 > Project: HBase > Issue Type: Test >Reporter: Ted Yu > Labels: backup, mob > Attachments: 16497.v1.txt > > > Currently backup / restore tests only deal with non-MOB tables. > This issue is to add coverage for backup / restore of table with MOB column > family. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
