[jira] [Updated] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-15209:
-
Attachment: HBASE-15209-v2.patch

re-uploading same patch to trigger another run

> disable table in HBaseTestingUtility.truncateTable 
> ---
>
> Key: HBASE-15209
> URL: https://issues.apache.org/jira/browse/HBASE-15209
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
>  Labels: compatibility
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15209-v1.patch, HBASE-15209-v2.patch
>
>
> Old truncate table expected table to be enabled and removed rows by scanning 
> over them.
> In HBASE-13764, the function was changed to make it consistent with how shell 
> worked, so new one used admin.truncateTable.
> Since HBTU.truncateTable expects table to be enabled (backward compat) 
> whereas admin.truncateTable expects table to be disabled, we should add a 
> disable table there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[huaxiang sun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129974#comment-15129974
 ] 

huaxiang sun commented on HBASE-15206:
--

[~eclark] Not sure about the real cluster. With 2.0, it is less likely to 
happen, it seems that child region compaction does not happen after split. With 
branch1.2, child region compaction happens right after child regions are open 
when useZKForAssignment is true.

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129971#comment-15129971
 ] 

Hadoop QA commented on HBASE-15197:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 7 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 51s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 11s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 53s 
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 18m 
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
10s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 8s 
{color} | {color:red} hbase-protocol in the patch failed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s 
{color} | {color:green} master passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 41s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 16s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 16s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 0s 
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 0s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 0s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 21m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
9s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
31m 56s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 8s 
{color} | {color:red} hbase-protocol in the patch failed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m 
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 27s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 8s 
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 42s 
{color} | {color:green} hbase-protocol in the patch passed with JDK v1.8.0_72. 
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 30s 
{color} | {color:green} hba

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129950#comment-15129950
 ] 

Hudson commented on HBASE-15206:


FAILURE: Integrated in HBase-1.2 #526 (See 
[https://builds.apache.org/job/HBase-1.2/526/])
HBASE-15206 Fix flakey testSplitDaughtersNotInMeta (Huaxiang Sun) (jmhsieh: rev 
54e5ca7da785fb62a57ec1fcd7750bfffd87dd76)
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15210) Undo crazy load balancer logging at tens of lines per millisecond

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15210:
--
Description: 
Sometimes the load balancer falls into a state of crazy logging in 1.2.0 
writing out ten lines a millisecond like this:

{code}
2016-02-02 17:15:01,382 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,382 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,384 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,384 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,386 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,386 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 2 and its region 
server contains 4 regions
2016-02-02 17:15:01,387 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,387 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,388 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,388 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,388 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
{code}

This issue is about less logging.

This log was added recently by this:

commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
Author: tedyu 
Date: Thu Aug 13 09:11:59 2015 -0700
HBASE-13376 Improvements to Stochastic load balancer (Vandana Ayyalasomayajula)


  was:
Sometimes the load balancer falls into a state of crazy logging in 1.2.0 
writing out ten lines a millisecond like this:

{code}
2016-02-02 17:15:01,382 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,382 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,384 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,384 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,386 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,386 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 2 and its region 
server contains 4 regions
2016-02-02 17:15:01,387 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,387 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,388 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halx

[jira] [Updated] (HBASE-15210) Undo crazy load balancer logging at tens of lines per millisecond

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15210:
--
Status: Patch Available  (was: Open)

> Undo crazy load balancer logging at tens of lines per millisecond
> -
>
> Key: HBASE-15210
> URL: https://issues.apache.org/jira/browse/HBASE-15210
> Project: HBase
>  Issue Type: Sub-task
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>Assignee: stack
> Attachments: 15210.patch
>
>
> Sometimes the load balancer falls into a state of crazy logging in 1.2.0 
> writing out ten lines a millisecond like this:
> {code}
> 2016-02-02 17:15:01,382 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,382 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,384 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,384 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,386 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,386 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 2 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,387 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,387 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,388 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,388 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,388 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> {code}
> This issue is about less logging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15210) Undo crazy load balancer logging at tens of lines per millisecond

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15210:
--
Attachment: 15210.patch

> Undo crazy load balancer logging at tens of lines per millisecond
> -
>
> Key: HBASE-15210
> URL: https://issues.apache.org/jira/browse/HBASE-15210
> Project: HBase
>  Issue Type: Sub-task
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>Assignee: stack
> Attachments: 15210.patch
>
>
> Sometimes the load balancer falls into a state of crazy logging in 1.2.0 
> writing out ten lines a millisecond like this:
> {code}
> 2016-02-02 17:15:01,382 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,382 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,384 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,384 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,386 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,386 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 2 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,387 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,387 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,388 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> 2016-02-02 17:15:01,388 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
> server contains 4 regions
> 2016-02-02 17:15:01,388 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.22908874
> {code}
> This issue is about less logging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15210) Undo crazy load balancer logging at tens of lines per millisecond

[stack (JIRA)]

stack created HBASE-15210:
-

 Summary: Undo crazy load balancer logging at tens of lines per 
millisecond
 Key: HBASE-15210
 URL: https://issues.apache.org/jira/browse/HBASE-15210
 Project: HBase
  Issue Type: Sub-task
  Components: Balancer
Affects Versions: 1.2.0
Reporter: stack
Assignee: stack


Sometimes the load balancer falls into a state of crazy logging in 1.2.0 
writing out ten lines a millisecond like this:

{code}
2016-02-02 17:15:01,382 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,382 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,384 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,384 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,386 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,386 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 2 and its region 
server contains 4 regions
2016-02-02 17:15:01,387 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,387 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,388 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
2016-02-02 17:15:01,388 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 3 and its region 
server contains 4 regions
2016-02-02 17:15:01,388 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.22908874
{code}

This issue is about less logging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15129) Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129935#comment-15129935
 ] 

Hudson commented on HBASE-15129:


SUCCESS: Integrated in HBase-1.3 #531 (See 
[https://builds.apache.org/job/HBase-1.3/531/])
HBASE-15129 Set default value for hbase.fs.tmp.dir rather than fully (enis: rev 
f1e7c06539e892f2e9092506e595404c5ab6be50)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/security/SecureBulkLoadUtil.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java


> Set default value for hbase.fs.tmp.dir rather than fully depend on 
> hbase-default.xml
> 
>
> Key: HBASE-15129
> URL: https://issues.apache.org/jira/browse/HBASE-15129
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15129.patch, HBASE-15129_v2.patch
>
>
> One of our users has observed below error when our cluster upgrades from 
> 0.98.12 to 1.1.2:
> {noformat}
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:639)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:489)
> {noformat}
> And checking the application code:
> {code}
>   private Job createJob(Configuration jobConf) throws IOException {
> String tableName = jobConf.get(TABLE_NAME_KEY);
> Job job = new Job(jobConf);
> Configuration tableConf = HBaseConfiguration.create();
> HTable table = new HTable(tableConf, tableName);
> HFileOutputFormat2.configureIncrementalLoad(job, table);
> return job;
>   }
> {code}
> We could see the user has his own hbase-independent job configuration, so our 
> current code in HFileOutputFormat2:
> {code}
> Path partitionsPath = new Path(conf.get("hbase.fs.tmp.dir"), "partitions_" + 
> UUID.randomUUID());
> {code}
> will return a null string and cause the above exception.
> We propose to add default value in the code rather than depend on 
> hbase-default.xml in this JIRA.
> This is an improvement for HBASE-13625



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129936#comment-15129936
 ] 

Hudson commented on HBASE-15206:


SUCCESS: Integrated in HBase-1.3 #531 (See 
[https://builds.apache.org/job/HBase-1.3/531/])
HBASE-15206 Fix flakey testSplitDaughtersNotInMeta (Huaxiang Sun) (jmhsieh: rev 
b69b8aced6317f6017b9e81c3ca10d1a9a9d3dda)
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15207) Stuck balancer

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129933#comment-15129933
 ] 

stack commented on HBASE-15207:
---

Happened again in new loading. Its hard to make sense of because the logging 
overwhelms. Let me address that in a patch first. Can then look at hang. In 
this current case we filled 10 log files of 256MB each... but it looks like we 
'recovered'. LB reports:

2016-02-02 17:15:05,123 DEBUG 
[ve0524.halxg.cloudera.com,16000,1454461776827_ChoreService_1] 
balancer.StochasticLoadBalancer: Finished computing new load balance plan.  
Computation took 3761ms to try 217600 different iterations.  Found a solution 
that moves 31 regions; Going from a computed cost of 341.5035724035313 to a new 
cost of 47.8518130393211

And going back over logs, yeah, I get sessions of log spewing maybe the 
load balancer is ok... its just this crazy logging phenomenon




> Stuck balancer
> --
>
> Key: HBASE-15207
> URL: https://issues.apache.org/jira/browse/HBASE-15207
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>
> Balancer seems to have gotten stuck in 1.2.0RC1 soon after Master joins 
> running cluster (previous Master had been killed by chaos monkey). 
> Investigate. At least fix the crazy logging which made me notice the stuck 
> balancer.
> Last night my logs filled with this (10x256MB log files):
> 
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer: 
> Lowest locality region server with non zero regions is 
> ve0542.halxg.cloudera.com with locality 0.0
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer:  
> Lowest locality region index is 0 and its region server contains 1 regions
> ...
> Added by this:
> commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
> Author: tedyu 
> Date:   Thu Aug 13 09:11:59 2015 -0700
> HBASE-13376 Improvements to Stochastic load balancer (Vandana 
> Ayyalasomayajula)
> Looks like balancer got stuck. Logging at ten lines a millisecond.
> Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.
> {code}
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 
> {code}
> Nothing else is happening on this master
> Happens just after a Master joins cluster after being killed by a monkey.



--
This message was sen

[jira] [Commented] (HBASE-15167) Deadlock in TestNamespaceAuditor.testRegionOperations on 1.1

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129927#comment-15129927
 ] 

Heng Chen commented on HBASE-15167:
---

Ping [~ndimiduk]

> Deadlock in TestNamespaceAuditor.testRegionOperations on 1.1
> 
>
> Key: HBASE-15167
> URL: https://issues.apache.org/jira/browse/HBASE-15167
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 1.1.3
>Reporter: Nick Dimiduk
>Assignee: Heng Chen
>Priority: Critical
> Fix For: 1.1.4
>
> Attachments: HBASE-15167-branch-1.1.patch, blocked.log
>
>
> This was left as a zombie after one of my test runs this weekend. 
> {noformat}
> "WALProcedureStoreSyncThread" daemon prio=10 tid=0x7f3ccc209000 
> nid=0x3960 in Object.wait() [0x7f3c6b6b5000]
>java.lang.Thread.State: BLOCKED (on object monitor)
>   at java.lang.Object.wait(Native Method)
>   at java.lang.Object.wait(Object.java:503)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1397)
>   - locked <0x0007f2813390> (a org.apache.hadoop.ipc.Client$Call)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1364)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
>   at com.sun.proxy.$Proxy23.create(Unknown Source)
>   at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
>   at com.sun.proxy.$Proxy23.create(Unknown Source)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.create(ClientNamenodeProtocolTranslatorPB.java:264)
>   at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)
>   at com.sun.proxy.$Proxy24.create(Unknown Source)
>   at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)
>   at com.sun.proxy.$Proxy24.create(Unknown Source)
>   at 
> org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1612)
>   at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1488)
>   at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1413)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:387)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:383)
>   at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:383)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:327)
>   at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
>   at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887)
>   at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:784)
>   at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:766)
>   at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:733)
>   at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.tryRollWriter(WALProcedureStore.java:668)
>   at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.periodicRoll(WALProcedureStore.java:711)
>   at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.syncLoop(WALProcedureStore.java:531)
>   at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.access$000(WALProcedureStore.java:66)
>   at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore$1.run(WALProcedureStore.java:180)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129924#comment-15129924
 ] 

Hadoop QA commented on HBASE-15209:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
22m 55s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 102m 45s 
{color} | {color:green} hbase-server in the patch passed with JDK v1.8.0_72. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 3s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
17s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 252m 25s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_91 Failed junit tests | 
hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-02-03 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12785916/HBASE-15209-v1.patch |
| JIRA Issue | HBASE-15209 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 5893b850ed06 3.13.0-36-lowlatency #63-Ubuntu SMP PREE

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129916#comment-15129916
 ] 

stack commented on HBASE-15177:
---

+1

There may be some overlap w/ [~ram_krish] patch in same area and I can see the 
boys messing w/ the BBIS so can work w/ DBB but good for now. Nice one.

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch, hbase-15177_v3.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129909#comment-15129909
 ] 

Heng Chen commented on HBASE-15197:
---

trigger QA.  https://builds.apache.org/job/PreCommit-HBASE-Build/410/

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197-v1.patch, HBASE-15197-v2.patch, 
> HBASE-15197.patch, HBASE-15197.patch, WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15201) Add hbase-spark to hbase assembly

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129908#comment-15129908
 ] 

stack commented on HBASE-15201:
---

+1 from me [~jerryhe] but suggest waiting on the blessing from mighty 
[~busbey]; it'll make you feel good.

> Add hbase-spark to hbase assembly
> -
>
> Key: HBASE-15201
> URL: https://issues.apache.org/jira/browse/HBASE-15201
> Project: HBase
>  Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Attachments: HBASE-15201.patch
>
>
> hbase-spark currently is missing from hbase assembly.
> We should add it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15158) Change order in which we do write pipeline operations; do all under row locks!

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15158:
--
Attachment: 15158v4.patch

Fix javadoc, whitespace, and all but the checkstyle complaint that a method is 
too long -- its doMiniBatchMutate which needs rewrite... not for this patch.

Review please.

Will run on cluster in meantime.

> Change order in which we do write pipeline operations; do all under row locks!
> --
>
> Key: HBASE-15158
> URL: https://issues.apache.org/jira/browse/HBASE-15158
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 15158.patch, 15158v2.patch, 15158v3.patch, 15158v4.patch
>
>
> Change how we do our write pipeline. I want to do all write pipeline ops 
> under row lock so I lean on this fact fixing performance regression in 
> check-and-set type operations like increment, append, and checkAnd* (see 
> sibling issue HBASE-15082).
> To be specific, we write like this now:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # add to memstore
> # let go of rowlock
> # sync WAL
> # in case of error: rollback memstore
> {code}
> Instead, write like this:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # sync WAL
> # add to memstore
> # let go of rowlock
> ... no need to do rollback.
> {code}
> The old ordering was put in place because it got better performance in a time 
> when WAL was different and before row locks were read/write (HBASE-12751).
> Testing in branch-1 shows that a reordering and skipping mvcc waits gets us 
> back to the performance we had before we unified mvcc and sequenceid 
> (HBASE-8763). Tests in HBASE-15046 show that at the macro level using our 
> usual perf tools, reordering pipeline seems to cause no slowdown (see 
> HBASE-15046). A rough compare of increments with reordered write pipeline 
> seems to have us getting back a bunch of our performance (see tail of 
> https://issues.apache.org/jira/browse/HBASE-15082?focusedCommentId=15111703&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15111703
>  and subsequent comment).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15201) Add hbase-spark to hbase assembly

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129889#comment-15129889
 ] 

Jerry He commented on HBASE-15201:
--

I have done some local testing to verify the above analysis 
Doing a top level build and install without the patch then going into 
hbase-assembly with the patch causes the problem.

In a real build we should be fine.

> Add hbase-spark to hbase assembly
> -
>
> Key: HBASE-15201
> URL: https://issues.apache.org/jira/browse/HBASE-15201
> Project: HBase
>  Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Attachments: HBASE-15201.patch
>
>
> hbase-spark currently is missing from hbase assembly.
> We should add it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15156) Support first assignment of split daughters to non-parent RS

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129843#comment-15129843
 ] 

stack commented on HBASE-15156:
---

I like the idea of difference being insignificant but the AM is such a fragile 
bit of code and state machine, I'd be wary messing with it. If it won't take 
much work, give it a go... if it looks good, I can try it on little rig here to 
see if it destabilizes.

> Support first assignment of split daughters to non-parent RS
> 
>
> Key: HBASE-15156
> URL: https://issues.apache.org/jira/browse/HBASE-15156
> Project: HBase
>  Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-15156.patch
>
>
> On region split, the region's daughter is always opened by the same region 
> server hosting the parent region. In some cases this is not ideal:
> This feature was mainly needed for favored nodes to allow for more freedom 
> when selecting favored nodes for daughter regions. ie The daughter doesn't 
> have to always select the regionserver hosting the split as a favored node 
> which should allow for better favored node distribution.
> Though this feature is actually useful in cases where region splits occur 
> much more often than the balancer is run. It also is a bit more efficient as 
> the major compaction that occurs after daughter assignment does not go to 
> waste (ie cancelled half-way, loss of locality due to move, etc). We actually 
> run it this way in some of our clusters even without favored nodes enabled. 
> Hence I am supplying a patch which is independent of favored nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15156) Support first assignment of split daughters to non-parent RS

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129839#comment-15129839
 ] 

stack commented on HBASE-15156:
---

bq. What was the reason it was faster?

No trip to the master to have it do assigment rpc'ing other servers; all was 
done local on the splitting regionserver.

bq. I don't think there's much difference in zkless land since everything has 
to go through the master now before becoming visible.

True

bq. What would need to be done to get it into trunk?

We can commit to trunk. It will be refactored later as part of the AM redo that 
is ongoing (Matteo and Stephen).

bq. ; .. the parent?

Comment then please.

bq. Because it's just a waste of RPC. We already do something similar if the RS 
performing the split dies, so this is not unprecendent.

HBase is like the Bible; you can find a precedent for any argument you might 
like to make in it (smile). onRegionSplit is where I'd expect to find the 
assign. Its ok.




> Support first assignment of split daughters to non-parent RS
> 
>
> Key: HBASE-15156
> URL: https://issues.apache.org/jira/browse/HBASE-15156
> Project: HBase
>  Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-15156.patch
>
>
> On region split, the region's daughter is always opened by the same region 
> server hosting the parent region. In some cases this is not ideal:
> This feature was mainly needed for favored nodes to allow for more freedom 
> when selecting favored nodes for daughter regions. ie The daughter doesn't 
> have to always select the regionserver hosting the split as a favored node 
> which should allow for better favored node distribution.
> Though this feature is actually useful in cases where region splits occur 
> much more often than the balancer is run. It also is a bit more efficient as 
> the major compaction that occurs after daughter assignment does not go to 
> waste (ie cancelled half-way, loss of locality due to move, etc). We actually 
> run it this way in some of our clusters even without favored nodes enabled. 
> Hence I am supplying a patch which is independent of favored nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15201) Add hbase-spark to hbase assembly

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129828#comment-15129828
 ] 

Jerry He commented on HBASE-15201:
--

I dig a little more.  
Here is the reason for the QA build problem.
The steps involved in the QA build:

{noformat}


  Pre-patch master maven install


cd /testptch/hbase
mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hbase-master-1 
-DHBasePatchProcess -fae clean install -DskipTests=true 
-Dmaven.javadoc.skip=true > /testptch/patchprocess/branch-mvninstall-root.txt 
2>&1
Elapsed:   3m 21s
{noformat}

Note that we do a top level mvn install without the patch.  The hbase-spark pom 
is updated in the patch (mark the spark-streaming as 'provided').  But the 
installed artifact pom is without the change.
Then in the next step:

{noformat}


Patch maven install




cd /testptch/hbase/hbase-assembly
mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hbase-master-1 
-DHBasePatchProcess -fae clean install -DskipTests=true 
-Dmaven.javadoc.skip=true > 
/testptch/patchprocess/patch-mvninstall-hbase-assembly.txt 2>&1
Elapsed:   0m 18s
cd /testptch/hbase/hbase-spark
mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hbase-master-1 
-DHBasePatchProcess -fae clean install -DskipTests=true 
-Dmaven.javadoc.skip=true > 
/testptch/patchprocess/patch-mvninstall-hbase-spark.txt 2>&1
Elapsed:   1m 16s

hbase-assembly in the patch failed.
{noformat}

We went into hbase-assembly directory and did a 'mvn install' in there.  But 
the other pom in the local repo was still the old bad one.
Thus it tried to pull in all the dependencies of spark-streaming, which it 
should not have done if it had seen the new pom in the local repo.

If it had reversed the order shown in the quoted text (built and installed the 
hbase-spark module first), we would be good as well.

These new steps of this new QA run are interesting ...  Looks like it only goes 
into the modules touched by the patch to do the mvn install again ...

> Add hbase-spark to hbase assembly
> -
>
> Key: HBASE-15201
> URL: https://issues.apache.org/jira/browse/HBASE-15201
> Project: HBase
>  Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Attachments: HBASE-15201.patch
>
>
> hbase-spark currently is missing from hbase assembly.
> We should add it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15201) Add hbase-spark to hbase assembly

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129797#comment-15129797
 ] 

Jerry He commented on HBASE-15201:
--

HI, [~busbey]  It just adds the hbase-spark-2.0.0-SNAPSHOT-jar to the tarball 
lib folder.



> Add hbase-spark to hbase assembly
> -
>
> Key: HBASE-15201
> URL: https://issues.apache.org/jira/browse/HBASE-15201
> Project: HBase
>  Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Attachments: HBASE-15201.patch
>
>
> hbase-spark currently is missing from hbase assembly.
> We should add it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129748#comment-15129748
 ] 

Hadoop QA commented on HBASE-15177:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 1s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 22s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m 
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 18s 
{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 30s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 29s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_66. 
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 31s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_91. 
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 31s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m 
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
27m 49s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 20s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 3s 
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0_66. 
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 0s 
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_66. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 32s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 6s 
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_91. 
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 55s 
{color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_91. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 32s {color} 
| {color:red} hbase-s

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129747#comment-15129747
 ] 

Hudson commented on HBASE-15206:


SUCCESS: Integrated in HBase-1.2-IT #416 (See 
[https://builds.apache.org/job/HBase-1.2-IT/416/])
HBASE-15206 Fix flakey testSplitDaughtersNotInMeta (Huaxiang Sun) (jmhsieh: rev 
54e5ca7da785fb62a57ec1fcd7750bfffd87dd76)
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15194) TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky on trunk

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129728#comment-15129728
 ] 

Hudson commented on HBASE-15194:


FAILURE: Integrated in HBase-1.3 #530 (See 
[https://builds.apache.org/job/HBase-1.3/530/])
HBASE-15194 (stack: rev 1bf58675c7d91fcdd3b4965dd0b86b8702758538)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java


> TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky 
> on trunk
> 
>
> Key: HBASE-15194
> URL: https://issues.apache.org/jira/browse/HBASE-15194
> Project: HBase
>  Issue Type: Sub-task
>  Components: flakey, test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: disable.patch
>
>
> Fails 25% of the time: 
> https://builds.apache.org/job/PreCommit-HBASE-Build/349/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/testRegionReplicationOnMidClusterSameHosts/history/
> I'm just going to disable till someone has time to dig in on the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129725#comment-15129725
 ] 

Hudson commented on HBASE-15206:


SUCCESS: Integrated in HBase-1.3-IT #471 (See 
[https://builds.apache.org/job/HBase-1.3-IT/471/])
HBASE-15206 Fix flakey testSplitDaughtersNotInMeta (Huaxiang Sun) (jmhsieh: rev 
b69b8aced6317f6017b9e81c3ca10d1a9a9d3dda)
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15129) Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129724#comment-15129724
 ] 

Hudson commented on HBASE-15129:


SUCCESS: Integrated in HBase-1.3-IT #471 (See 
[https://builds.apache.org/job/HBase-1.3-IT/471/])
HBASE-15129 Set default value for hbase.fs.tmp.dir rather than fully (enis: rev 
f1e7c06539e892f2e9092506e595404c5ab6be50)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/security/SecureBulkLoadUtil.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java


> Set default value for hbase.fs.tmp.dir rather than fully depend on 
> hbase-default.xml
> 
>
> Key: HBASE-15129
> URL: https://issues.apache.org/jira/browse/HBASE-15129
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15129.patch, HBASE-15129_v2.patch
>
>
> One of our users has observed below error when our cluster upgrades from 
> 0.98.12 to 1.1.2:
> {noformat}
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:639)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:489)
> {noformat}
> And checking the application code:
> {code}
>   private Job createJob(Configuration jobConf) throws IOException {
> String tableName = jobConf.get(TABLE_NAME_KEY);
> Job job = new Job(jobConf);
> Configuration tableConf = HBaseConfiguration.create();
> HTable table = new HTable(tableConf, tableName);
> HFileOutputFormat2.configureIncrementalLoad(job, table);
> return job;
>   }
> {code}
> We could see the user has his own hbase-independent job configuration, so our 
> current code in HFileOutputFormat2:
> {code}
> Path partitionsPath = new Path(conf.get("hbase.fs.tmp.dir"), "partitions_" + 
> UUID.randomUUID());
> {code}
> will return a null string and cause the above exception.
> We propose to add default value in the code rather than depend on 
> hbase-default.xml in this JIRA.
> This is an improvement for HBASE-13625



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Matteo Bertozzi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129709#comment-15129709
 ] 

Matteo Bertozzi commented on HBASE-15209:
-

the current HTU.truncateTable() has nothing to do with the old truncate (pre 
HBASE-13764, < 1.0).
The "old truncate" is now deleteTableData() and it was removing row by row with 
Deletes. the new truncate is just a call to admin.truncate.

The only user or HTU.truncateTable() is AccessController and it does not even 
use/close the returned Table instance. I'd say we can remove that method and 
force everyone to use Admin as they do for the other operations.

but patch looks ok, since at least follows the comment that is there 
"Effectively disables, deletes, and recreates the table."

> disable table in HBaseTestingUtility.truncateTable 
> ---
>
> Key: HBASE-15209
> URL: https://issues.apache.org/jira/browse/HBASE-15209
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
>  Labels: compatibility
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15209-v1.patch
>
>
> Old truncate table expected table to be enabled and removed rows by scanning 
> over them.
> In HBASE-13764, the function was changed to make it consistent with how shell 
> worked, so new one used admin.truncateTable.
> Since HBTU.truncateTable expects table to be enabled (backward compat) 
> whereas admin.truncateTable expects table to be disabled, we should add a 
> disable table there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15156) Support first assignment of split daughters to non-parent RS

[Francis Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129710#comment-15129710
 ] 

Francis Liu commented on HBASE-15156:
-

{quote}
For instance, could the assign happen inside in the onRegionSplit? 
{quote}
Another approach is to remove onRegionSplit altogether and have invokeAssign() 
assign the daughters to the RS that initiated the split. This way the forked 
code between the two behaviors becomes insignificant. This is also more 
efficient it seems. Thoughts?

> Support first assignment of split daughters to non-parent RS
> 
>
> Key: HBASE-15156
> URL: https://issues.apache.org/jira/browse/HBASE-15156
> Project: HBase
>  Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-15156.patch
>
>
> On region split, the region's daughter is always opened by the same region 
> server hosting the parent region. In some cases this is not ideal:
> This feature was mainly needed for favored nodes to allow for more freedom 
> when selecting favored nodes for daughter regions. ie The daughter doesn't 
> have to always select the regionserver hosting the split as a favored node 
> which should allow for better favored node distribution.
> Though this feature is actually useful in cases where region splits occur 
> much more often than the balancer is run. It also is a bit more efficient as 
> the major compaction that occurs after daughter assignment does not go to 
> waste (ie cancelled half-way, loss of locality due to move, etc). We actually 
> run it this way in some of our clusters even without favored nodes enabled. 
> Hence I am supplying a patch which is independent of favored nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15156) Support first assignment of split daughters to non-parent RS

[Francis Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129708#comment-15129708
 ] 

Francis Liu commented on HBASE-15156:
-

Thanks for the review [~stack] some responses:

{quote}
We assign daughters to the same node as parent because this way we can get the 
data back on line more promptly.
{quote}
What was the reason it was faster? In terms of assigment overhead I don't think 
there's much difference in zkless land since everything has to go through the 
master now before becoming visible.

{quote}
On patch, seems fine for branch-1. Should update doc on method to say sna and 
snb are optional on splitRegion.
{quote}
Will do.

{quote}
On patch, seems fine for branch-1. Should update doc on method to say sna and 
snb are optional on splitRegion.
{quote}
What would need to be done to get it into trunk?

{quote}
public static final String ASSIGN_DAUGHTERS_AFTER_SPLIT =
"hbase.split.unassign";
{quote}
will fix.

{quote}
Could you group the two clausees that get invoked when shouldAssignAfterSplit 
== true ?
{quote}
will do.

{quote}
What region is being offlined here: regionOffline(hri, State.SPLIT); .. the 
parent?
{quote}
Yup parent. 

{quote}
Why not have this assign happen in the call to onRegionSplit?
{quote}
Because it's just a waste of RPC. We already do something similar if the RS 
performing the split dies, so this is not unprecendent. 

{quote}
!services.getConfiguration().getBoolean(HConstants.ASSIGN_DAUGHTERS_AFTER_SPLIT,
 false)) {
{quote}
Will consolidate. we can actually take this a step further and embed in the 
response to reportRegionInTransition() call to master wether the RS should 
online the daughter regions. This way we need only the config set on the master 
minimizes config errors?





> Support first assignment of split daughters to non-parent RS
> 
>
> Key: HBASE-15156
> URL: https://issues.apache.org/jira/browse/HBASE-15156
> Project: HBase
>  Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-15156.patch
>
>
> On region split, the region's daughter is always opened by the same region 
> server hosting the parent region. In some cases this is not ideal:
> This feature was mainly needed for favored nodes to allow for more freedom 
> when selecting favored nodes for daughter regions. ie The daughter doesn't 
> have to always select the regionserver hosting the split as a favored node 
> which should allow for better favored node distribution.
> Though this feature is actually useful in cases where region splits occur 
> much more often than the balancer is run. It also is a bit more efficient as 
> the major compaction that occurs after daughter assignment does not go to 
> waste (ie cancelled half-way, loss of locality due to move, etc). We actually 
> run it this way in some of our clusters even without favored nodes enabled. 
> Hence I am supplying a patch which is independent of favored nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129707#comment-15129707
 ] 

Hudson commented on HBASE-15206:


FAILURE: Integrated in HBase-Trunk_matrix #675 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/675/])
HBASE-15206 Fix flakey testSplitDaughtersNotInMeta test (Huaxiang Sun) 
(jmhsieh: rev ecf3efadc04473c36a16d85baafe9d1e3ced355a)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java


> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15129) Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129706#comment-15129706
 ] 

Hudson commented on HBASE-15129:


FAILURE: Integrated in HBase-Trunk_matrix #675 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/675/])
HBASE-15129 Set default value for hbase.fs.tmp.dir rather than fully (enis: rev 
2f5767376f42c0416e025df412e3d5944a1b2a67)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/security/SecureBulkLoadUtil.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java


> Set default value for hbase.fs.tmp.dir rather than fully depend on 
> hbase-default.xml
> 
>
> Key: HBASE-15129
> URL: https://issues.apache.org/jira/browse/HBASE-15129
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15129.patch, HBASE-15129_v2.patch
>
>
> One of our users has observed below error when our cluster upgrades from 
> 0.98.12 to 1.1.2:
> {noformat}
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:639)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:489)
> {noformat}
> And checking the application code:
> {code}
>   private Job createJob(Configuration jobConf) throws IOException {
> String tableName = jobConf.get(TABLE_NAME_KEY);
> Job job = new Job(jobConf);
> Configuration tableConf = HBaseConfiguration.create();
> HTable table = new HTable(tableConf, tableName);
> HFileOutputFormat2.configureIncrementalLoad(job, table);
> return job;
>   }
> {code}
> We could see the user has his own hbase-independent job configuration, so our 
> current code in HFileOutputFormat2:
> {code}
> Path partitionsPath = new Path(conf.get("hbase.fs.tmp.dir"), "partitions_" + 
> UUID.randomUUID());
> {code}
> will return a null string and cause the above exception.
> We propose to add default value in the code rather than depend on 
> hbase-default.xml in this JIRA.
> This is an improvement for HBASE-13625



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129689#comment-15129689
 ] 

Heng Chen commented on HBASE-15197:
---

+1 on patch v2.   If no other concerns, let me commit it later.  [~tedyu] :)

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197-v1.patch, HBASE-15197-v2.patch, 
> HBASE-15197.patch, HBASE-15197.patch, WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15158) Change order in which we do write pipeline operations; do all under row locks!

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129667#comment-15129667
 ] 

Hadoop QA commented on HBASE-15158:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 16s 
{color} | {color:red} Patch generated 20 new checkstyle issues in hbase-server 
(total was 331, now 267). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
22m 1s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 3m 13s 
{color} | {color:red} hbase-server-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 
new issues (was 1, now 3). {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m 16s 
{color} | {color:green} hbase-server in the patch passed with JDK v1.8.0_66. 
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 0s 
{color} | {color:green} hbase-server in the patch passed with JDK v1.7.0_91. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
16s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 238m 11s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-02-02 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12785875/15158v3.patch |
| JIRA Issue | HBASE-15158 |
| Optional Tests |  a

[jira] [Updated] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-15209:
-
Labels: compatibility  (was: )

> disable table in HBaseTestingUtility.truncateTable 
> ---
>
> Key: HBASE-15209
> URL: https://issues.apache.org/jira/browse/HBASE-15209
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
>  Labels: compatibility
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15209-v1.patch
>
>
> Old truncate table expected table to be enabled and removed rows by scanning 
> over them.
> In HBASE-13764, the function was changed to make it consistent with how shell 
> worked, so new one used admin.truncateTable.
> Since HBTU.truncateTable expects table to be enabled (backward compat) 
> whereas admin.truncateTable expects table to be disabled, we should add a 
> disable table there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-15209:
-
Fix Version/s: 1.2.0
   2.0.0

> disable table in HBaseTestingUtility.truncateTable 
> ---
>
> Key: HBASE-15209
> URL: https://issues.apache.org/jira/browse/HBASE-15209
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15209-v1.patch
>
>
> Old truncate table expected table to be enabled and removed rows by scanning 
> over them.
> In HBASE-13764, the function was changed to make it consistent with how shell 
> worked, so new one used admin.truncateTable.
> Since HBTU.truncateTable expects table to be enabled (backward compat) 
> whereas admin.truncateTable expects table to be disabled, we should add a 
> disable table there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-15209:
-
Status: Patch Available  (was: In Progress)

> disable table in HBaseTestingUtility.truncateTable 
> ---
>
> Key: HBASE-15209
> URL: https://issues.apache.org/jira/browse/HBASE-15209
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Attachments: HBASE-15209-v1.patch
>
>
> Old truncate table expected table to be enabled and removed rows by scanning 
> over them.
> In HBASE-13764, the function was changed to make it consistent with how shell 
> worked, so new one used admin.truncateTable.
> Since HBTU.truncateTable expects table to be enabled (backward compat) 
> whereas admin.truncateTable expects table to be disabled, we should add a 
> disable table there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-15209 started by Appy.

> disable table in HBaseTestingUtility.truncateTable 
> ---
>
> Key: HBASE-15209
> URL: https://issues.apache.org/jira/browse/HBASE-15209
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Fix For: 2.0.0, 1.2.0
>
> Attachments: HBASE-15209-v1.patch
>
>
> Old truncate table expected table to be enabled and removed rows by scanning 
> over them.
> In HBASE-13764, the function was changed to make it consistent with how shell 
> worked, so new one used admin.truncateTable.
> Since HBTU.truncateTable expects table to be enabled (backward compat) 
> whereas admin.truncateTable expects table to be disabled, we should add a 
> disable table there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-15209:
-
Attachment: HBASE-15209-v1.patch

> disable table in HBaseTestingUtility.truncateTable 
> ---
>
> Key: HBASE-15209
> URL: https://issues.apache.org/jira/browse/HBASE-15209
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Attachments: HBASE-15209-v1.patch
>
>
> Old truncate table expected table to be enabled and removed rows by scanning 
> over them.
> In HBASE-13764, the function was changed to make it consistent with how shell 
> worked, so new one used admin.truncateTable.
> Since HBTU.truncateTable expects table to be enabled (backward compat) 
> whereas admin.truncateTable expects table to be disabled, we should add a 
> disable table there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15175) Add support for checkAndXX methods in JavaHBaseContext for Spark

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15175:
---
Component/s: spark

> Add support for checkAndXX methods in JavaHBaseContext for Spark
> 
>
> Key: HBASE-15175
> URL: https://issues.apache.org/jira/browse/HBASE-15175
> Project: HBase
>  Issue Type: Improvement
>  Components: spark
>Reporter: Ted Yu
>
> Currently JavaHBaseContext has methods for bulkPut, bulkDelete, etc
> It is desirable to add support for checkAndPut, checkAndMutate, 
> checkAndDelete as well.
> See related thread:
> http://search-hadoop.com/m/YGbbt3Szd24Vtpr&subj=Re+HBase+atomic+update
> Thanks [~zzhan] for offline discussion



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15129) Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129630#comment-15129630
 ] 

Yu Li commented on HBASE-15129:
---

Thanks for help review and commit [~enis]!

Thanks [~ashish singhi], [~syuanjiang] and [~ndimiduk] for review.

> Set default value for hbase.fs.tmp.dir rather than fully depend on 
> hbase-default.xml
> 
>
> Key: HBASE-15129
> URL: https://issues.apache.org/jira/browse/HBASE-15129
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15129.patch, HBASE-15129_v2.patch
>
>
> One of our users has observed below error when our cluster upgrades from 
> 0.98.12 to 1.1.2:
> {noformat}
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:639)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:489)
> {noformat}
> And checking the application code:
> {code}
>   private Job createJob(Configuration jobConf) throws IOException {
> String tableName = jobConf.get(TABLE_NAME_KEY);
> Job job = new Job(jobConf);
> Configuration tableConf = HBaseConfiguration.create();
> HTable table = new HTable(tableConf, tableName);
> HFileOutputFormat2.configureIncrementalLoad(job, table);
> return job;
>   }
> {code}
> We could see the user has his own hbase-independent job configuration, so our 
> current code in HFileOutputFormat2:
> {code}
> Path partitionsPath = new Path(conf.get("hbase.fs.tmp.dir"), "partitions_" + 
> UUID.randomUUID());
> {code}
> will return a null string and cause the above exception.
> We propose to add default value in the code rather than depend on 
> hbase-default.xml in this JIRA.
> This is an improvement for HBASE-13625



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15177) Reduce garbage created under high load

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-15177:
--
Attachment: hbase-15177_v3.patch

This should fix the UT issues. Had to bring back the buffer.position(offset) 
that was removed from v0. This is needed since in case cells are compressed, we 
are creating an BBIS over the BB. 

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch, hbase-15177_v3.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129609#comment-15129609
 ] 

Hadoop QA commented on HBASE-15197:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 7 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m 
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m 
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 22s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 15s 
{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 10s 
{color} | {color:red} hbase-hadoop2-compat in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 25s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 23s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72. 
{color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 23s {color} | 
{color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 23s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 25s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_91. 
{color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 25s {color} | 
{color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 25s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 1s 
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-server 
(total was 435, now 435). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
55s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
22m 13s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m 
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 50s 
{color} | {c

[jira] [Created] (HBASE-15209) disable table in HBaseTestingUtility.truncateTable

[Appy (JIRA)]

Appy created HBASE-15209:


 Summary: disable table in HBaseTestingUtility.truncateTable 
 Key: HBASE-15209
 URL: https://issues.apache.org/jira/browse/HBASE-15209
 Project: HBase
  Issue Type: Bug
Reporter: Appy
Assignee: Appy
Priority: Minor


Old truncate table expected table to be enabled and removed rows by scanning 
over them.
In HBASE-13764, the function was changed to make it consistent with how shell 
worked, so new one used admin.truncateTable.
Since HBTU.truncateTable expects table to be enabled (backward compat) whereas 
admin.truncateTable expects table to be disabled, we should add a disable table 
there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129546#comment-15129546
 ] 

Enis Soztutar commented on HBASE-15177:
---

bq. I do not believe this is true, and would rather not get into it with ASF 
policy folks on the distinction between non-ASF folks who are not a part of the 
Yetus development community vs ASF folks who are not a part of the Yetus 
development community.
An apache project depending on another apache project, I am not sure I 
understand the (hypothetical) controversy. If we are depending on yetus only 
for the pre-commit (and not for release artifacts), we should be able to depend 
any SNAPSHOT build. Anyway, if yetus is doing more frequent releases, it might 
not matter.  

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15129) Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-15129:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 1.3.0
   2.0.0
   Status: Resolved  (was: Patch Available)

Committed this. Thanks [~carp84]. 

> Set default value for hbase.fs.tmp.dir rather than fully depend on 
> hbase-default.xml
> 
>
> Key: HBASE-15129
> URL: https://issues.apache.org/jira/browse/HBASE-15129
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15129.patch, HBASE-15129_v2.patch
>
>
> One of our users has observed below error when our cluster upgrades from 
> 0.98.12 to 1.1.2:
> {noformat}
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:639)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:489)
> {noformat}
> And checking the application code:
> {code}
>   private Job createJob(Configuration jobConf) throws IOException {
> String tableName = jobConf.get(TABLE_NAME_KEY);
> Job job = new Job(jobConf);
> Configuration tableConf = HBaseConfiguration.create();
> HTable table = new HTable(tableConf, tableName);
> HFileOutputFormat2.configureIncrementalLoad(job, table);
> return job;
>   }
> {code}
> We could see the user has his own hbase-independent job configuration, so our 
> current code in HFileOutputFormat2:
> {code}
> Path partitionsPath = new Path(conf.get("hbase.fs.tmp.dir"), "partitions_" + 
> UUID.randomUUID());
> {code}
> will return a null string and cause the above exception.
> We propose to add default value in the code rather than depend on 
> hbase-default.xml in this JIRA.
> This is an improvement for HBASE-13625



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15203) Reduce garbage created by path.toString() during Checksum verification

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129512#comment-15129512
 ] 

Nick Dimiduk commented on HBASE-15203:
--

+1 for branch-1.1.

> Reduce garbage created by path.toString() during Checksum verification
> --
>
> Key: HBASE-15203
> URL: https://issues.apache.org/jira/browse/HBASE-15203
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15203.patch, HBASE-15203_1.patch
>
>
> When we try to read a block we do checksum verification for which we need the 
> file name in which the block belongs to. So we do Path.toString() every time. 
> This seems to create around 163MB of char[] that is garbage collected in a 
> simple scan run. It is also visible in writes but the impact is lesser. In 
> overall write/read profile the top 2 factors are byte[] and char[]. This 
> toString() can easily be avoided and reduce its share from the total. To make 
> it more precise in 1 min of profiling, among the 1.8G of garbage created by 
> StringBuilder.toString - this path.toString() was contributing around 3.5%. 
> After the patch this is totally not there. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Elliott Clark (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129503#comment-15129503
 ] 

Elliott Clark commented on HBASE-15206:
---

Can this happen on a real cluster resulting in the same clean up of the parent 
region? We've recently seen something similar.

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Jonathan Hsieh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-15206:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 1.1.4
   1.3.0
   1.2.0
   2.0.0
   Status: Resolved  (was: Patch Available)

Thanks huaxiang.  

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129478#comment-15129478
 ] 

Hadoop QA commented on HBASE-15177:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 38s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 47s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 29s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m 
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 6s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 1s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 55s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 43s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 49s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 49s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 56s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 27s 
{color} | {color:red} Patch generated 3 new checkstyle issues in hbase-common 
(total was 7, now 10). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
36m 26s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 11s 
{color} | {color:red} hbase-common introduced 1 new FindBugs issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 59s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 30s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 59s 
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_72. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 4s {color} | 
{color:red} hbase-client in the patch failed with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 36s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 57s 
{color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_91. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 6s {color} | 
{color:red} hbase-client in the patch failed with JDK v

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129479#comment-15129479
 ] 

Jonathan Hsieh commented on HBASE-15206:


+1 huaxiang.  Committing to master, 1.x, 1.2, 1.1. Assuming we are done with 
1.0 but can commit there if asked to.

The failed test on the first yetus run cannot be related. (test only fix, 
problem in other unrelated test.)

The second yetus run is the applying the 1.x branch patch to master and failing.



> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Eungsop Yoo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eungsop Yoo updated HBASE-15197:

Status: Patch Available  (was: Open)

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197-v1.patch, HBASE-15197-v2.patch, 
> HBASE-15197.patch, HBASE-15197.patch, WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Eungsop Yoo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eungsop Yoo updated HBASE-15197:

Attachment: HBASE-15197-v2.patch

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197-v1.patch, HBASE-15197-v2.patch, 
> HBASE-15197.patch, HBASE-15197.patch, WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Eungsop Yoo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eungsop Yoo updated HBASE-15197:

Status: Open  (was: Patch Available)

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197-v1.patch, HBASE-15197.patch, 
> HBASE-15197.patch, WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Eungsop Yoo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eungsop Yoo updated HBASE-15197:

Status: Patch Available  (was: Open)

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197-v1.patch, HBASE-15197.patch, 
> HBASE-15197.patch, WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Eungsop Yoo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eungsop Yoo updated HBASE-15197:

Attachment: HBASE-15197-v1.patch

The comment is modified.

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197-v1.patch, HBASE-15197.patch, 
> HBASE-15197.patch, WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15197) Expose filtered read requests metric to metrics framework and Web UI

[Eungsop Yoo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eungsop Yoo updated HBASE-15197:

Status: Open  (was: Patch Available)

> Expose filtered read requests metric to metrics framework and Web UI
> 
>
> Key: HBASE-15197
> URL: https://issues.apache.org/jira/browse/HBASE-15197
> Project: HBase
>  Issue Type: Improvement
>Reporter: Eungsop Yoo
>Assignee: Eungsop Yoo
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15197.patch, HBASE-15197.patch, 
> WebUI-screenshot.png
>
>
> Filtered read requests metric for scan is added in 
> [HBASE-5980|https://issues.apache.org/jira/browse/HBASE-5980]. But it can be 
> retrieved from Scan object only. I think that it would be more informative 
> when it is exposed to metrics framework. So I suggest to add filtered read 
> requests metric and to expose it metrics framework and Web UI.
> ps. I think I found a bug; read requests count increased when Get operation 
> returns no record by filtering out. Should it be fixed?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15207) Stuck balancer

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129423#comment-15129423
 ] 

Sean Busbey commented on HBASE-15207:
-

I'd say blocker status will depend on if we can determine if the root cause is 
new in 1.2

> Stuck balancer
> --
>
> Key: HBASE-15207
> URL: https://issues.apache.org/jira/browse/HBASE-15207
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>
> Balancer seems to have gotten stuck in 1.2.0RC1 soon after Master joins 
> running cluster (previous Master had been killed by chaos monkey). 
> Investigate. At least fix the crazy logging which made me notice the stuck 
> balancer.
> Last night my logs filled with this (10x256MB log files):
> 
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer: 
> Lowest locality region server with non zero regions is 
> ve0542.halxg.cloudera.com with locality 0.0
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer:  
> Lowest locality region index is 0 and its region server contains 1 regions
> ...
> Added by this:
> commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
> Author: tedyu 
> Date:   Thu Aug 13 09:11:59 2015 -0700
> HBASE-13376 Improvements to Stochastic load balancer (Vandana 
> Ayyalasomayajula)
> Looks like balancer got stuck. Logging at ten lines a millisecond.
> Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.
> {code}
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 
> {code}
> Nothing else is happening on this master
> Happens just after a Master joins cluster after being killed by a monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15208) AsyncProcess seems to rely on regionId() to be unique which is not

[Matteo Bertozzi (JIRA)]

Matteo Bertozzi created HBASE-15208:
---

 Summary: AsyncProcess seems to rely on regionId() to be unique 
which is not
 Key: HBASE-15208
 URL: https://issues.apache.org/jira/browse/HBASE-15208
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 0.98.16.1, 1.1.3, 1.0.3, 2.0.0, 1.2.0, 1.3.0
Reporter: Matteo Bertozzi


AsyncProcess is using getRegionId() to identify a region. It looks like the 
original implementation was using encodedName which is unique, but then we 
switched to regionIds with HBASE-9988.
{code}
protected boolean canTakeOperation(HRegionLocation loc,
 Map regionsIncluded,
 Map serversIncluded) {
long regionId = loc.getRegionInfo().getRegionId();
Boolean regionPrevious = regionsIncluded.get(regionId);
if (regionPrevious != null) {
  // We already know what to do with this region.
  return regionPrevious;
}
{code}

The RegionId is not unique, since it is a timestamp. and specifically in case 
we create a table with splits each region get the same regionId. (from the doc 
of HRegionInfo about regionId, looks like that field should be named 
creationTimestamp or something like that for a more expressive name).

I'm unsure on what are the consequences. from a couple of tests and the code 
looks like it may just allow more tasks than the one configured by the max task 
allowed conf. but I'd let someone else familiar with this code look into it.

Also TestAsyncProcess has the HRegionInfo hr1, hr2, hr3 with different 
regionIds, if we switch them to be the same id we have a 
testSubmitBusyRegionServer() failing



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15207) Stuck balancer

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129420#comment-15129420
 ] 

stack commented on HBASE-15207:
---

Fixed summary and description. It is about the stuck balancer. Should also fix 
the crazy logging. Can do that in subtask. Thanks [~busbey]


> Stuck balancer
> --
>
> Key: HBASE-15207
> URL: https://issues.apache.org/jira/browse/HBASE-15207
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>
> Balancer seems to have gotten stuck in 1.2.0RC1 soon after Master joins 
> running cluster (previous Master had been killed by chaos monkey). 
> Investigate. At least fix the crazy logging which made me notice the stuck 
> balancer.
> Last night my logs filled with this (10x256MB log files):
> 
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer: 
> Lowest locality region server with non zero regions is 
> ve0542.halxg.cloudera.com with locality 0.0
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer:  
> Lowest locality region index is 0 and its region server contains 1 regions
> ...
> Added by this:
> commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
> Author: tedyu 
> Date:   Thu Aug 13 09:11:59 2015 -0700
> HBASE-13376 Improvements to Stochastic load balancer (Vandana 
> Ayyalasomayajula)
> Looks like balancer got stuck. Logging at ten lines a millisecond.
> Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.
> {code}
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 
> {code}
> Nothing else is happening on this master
> Happens just after a Master joins cluster after being killed by a monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15207) Stuck balancer

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15207:
--
Description: 
Balancer seems to have gotten stuck in 1.2.0RC1 soon after Master joins running 
cluster (previous Master had been killed by chaos monkey). Investigate. At 
least fix the crazy logging which made me notice the stuck balancer.

Last night my logs filled with this (10x256MB log files):


2016-02-01 11:25:26,958 DEBUG [B.defaultRpcServer.handler=9,queue=0,port=16000] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0542.halxg.cloudera.com with locality 0.0
2016-02-01 11:25:26,958 DEBUG [B.defaultRpcServer.handler=9,queue=0,port=16000] 
balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
server contains 1 regions
...


Added by this:

commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
Author: tedyu 
Date:   Thu Aug 13 09:11:59 2015 -0700

HBASE-13376 Improvements to Stochastic load balancer (Vandana 
Ayyalasomayajula)

Looks like balancer got stuck. Logging at ten lines a millisecond.

Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.

{code}
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.0
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
server contains 1 regions
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.0
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
server contains 1 regions
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.

{code}

Nothing else is happening on this master

Happens just after a Master joins cluster after being killed by a monkey.

  was:
Last night my logs filled with this (10x256MB log files):


2016-02-01 11:25:26,958 DEBUG [B.defaultRpcServer.handler=9,queue=0,port=16000] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0542.halxg.cloudera.com with locality 0.0
2016-02-01 11:25:26,958 DEBUG [B.defaultRpcServer.handler=9,queue=0,port=16000] 
balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
server contains 1 regions
...


Added by this:

commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
Author: tedyu 
Date:   Thu Aug 13 09:11:59 2015 -0700

HBASE-13376 Improvements to Stochastic load balancer (Vandana 
Ayyalasomayajula)

Looks like balancer got stuck. Logging at ten lines a millisecond.

Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.

{code}
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreS

[jira] [Updated] (HBASE-15207) Stuck balancer

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15207:
--
Summary: Stuck balancer  (was: Stuck balancer; logging ten lines a 
millisecond)

> Stuck balancer
> --
>
> Key: HBASE-15207
> URL: https://issues.apache.org/jira/browse/HBASE-15207
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>
> Last night my logs filled with this (10x256MB log files):
> 
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer: 
> Lowest locality region server with non zero regions is 
> ve0542.halxg.cloudera.com with locality 0.0
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer:  
> Lowest locality region index is 0 and its region server contains 1 regions
> ...
> Added by this:
> commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
> Author: tedyu 
> Date:   Thu Aug 13 09:11:59 2015 -0700
> HBASE-13376 Improvements to Stochastic load balancer (Vandana 
> Ayyalasomayajula)
> Looks like balancer got stuck. Logging at ten lines a millisecond.
> Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.
> {code}
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 
> {code}
> Nothing else is happening on this master
> Happens just after a Master joins cluster after being killed by a monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15129) Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129404#comment-15129404
 ] 

Enis Soztutar commented on HBASE-15129:
---

Thanks Yu, the patch looks good. Both of the findbugs warnings do not look they 
are relevant. It is probably due to the YETUS module ordering bug. 

TEMPORARY_HDFS_DIRECTORY_KEY should be named TEMPORARY_FS_DIRECTORY_KEY since 
it is not HDFS specific. I'll do that on commit. 

> Set default value for hbase.fs.tmp.dir rather than fully depend on 
> hbase-default.xml
> 
>
> Key: HBASE-15129
> URL: https://issues.apache.org/jira/browse/HBASE-15129
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15129.patch, HBASE-15129_v2.patch
>
>
> One of our users has observed below error when our cluster upgrades from 
> 0.98.12 to 1.1.2:
> {noformat}
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:639)
> at 
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:489)
> {noformat}
> And checking the application code:
> {code}
>   private Job createJob(Configuration jobConf) throws IOException {
> String tableName = jobConf.get(TABLE_NAME_KEY);
> Job job = new Job(jobConf);
> Configuration tableConf = HBaseConfiguration.create();
> HTable table = new HTable(tableConf, tableName);
> HFileOutputFormat2.configureIncrementalLoad(job, table);
> return job;
>   }
> {code}
> We could see the user has his own hbase-independent job configuration, so our 
> current code in HFileOutputFormat2:
> {code}
> Path partitionsPath = new Path(conf.get("hbase.fs.tmp.dir"), "partitions_" + 
> UUID.randomUUID());
> {code}
> will return a null string and cause the above exception.
> We propose to add default value in the code rather than depend on 
> hbase-default.xml in this JIRA.
> This is an improvement for HBASE-13625



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15207) Stuck balancer; logging ten lines a millisecond

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129390#comment-15129390
 ] 

Sean Busbey commented on HBASE-15207:
-

is this ticket for the balancer getting stuck, or for the logging rate when it 
is stuck?

> Stuck balancer; logging ten lines a millisecond
> ---
>
> Key: HBASE-15207
> URL: https://issues.apache.org/jira/browse/HBASE-15207
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>
> Last night my logs filled with this (10x256MB log files):
> 
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer: 
> Lowest locality region server with non zero regions is 
> ve0542.halxg.cloudera.com with locality 0.0
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer:  
> Lowest locality region index is 0 and its region server contains 1 regions
> ...
> Added by this:
> commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
> Author: tedyu 
> Date:   Thu Aug 13 09:11:59 2015 -0700
> HBASE-13376 Improvements to Stochastic load balancer (Vandana 
> Ayyalasomayajula)
> Looks like balancer got stuck. Logging at ten lines a millisecond.
> Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.
> {code}
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 
> {code}
> Nothing else is happening on this master
> Happens just after a Master joins cluster after being killed by a monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15181) A simple implementation of date based tiered compaction

[Clara Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129372#comment-15129372
 ] 

Clara Xiong commented on HBASE-15181:
-

With this implementation, we have an overall compaction policy and a per-window 
compaction policy which is default to exploring policy. So I have to use two 
separate configuration.

> A simple implementation of date based tiered compaction
> ---
>
> Key: HBASE-15181
> URL: https://issues.apache.org/jira/browse/HBASE-15181
> Project: HBase
>  Issue Type: New Feature
>  Components: Compaction
>Reporter: Clara Xiong
>Assignee: Clara Xiong
> Fix For: 2.0.0
>
> Attachments: HBASE-15181-v1.patch, HBASE-15181-v2.patch
>
>
> This is a simple implementation of date-based tiered compaction similar to 
> Cassandra's for the following benefits:
> 1. Improve date-range-based scan by structuring store files in date-based 
> tiered layout.
> 2. Reduce compaction overhead.
> 3. Improve TTL efficiency.
> Perfect fit for the use cases that:
> 1. has mostly date-based date write and scan and a focus on the most recent 
> data. 
> 2. never or rarely deletes data.
> Out-of-order writes are handled gracefully so the data will still get to the 
> right store file for time-range-scan and re-compacton with existing store 
> file in the same time window is handled by ExploringCompactionPolicy.
> Time range overlapping among store files is tolerated and the performance 
> impact is minimized.
> Configuration can be set at hbase-site or overriden at per-table or 
> per-column-famly level by hbase shell.
> Design spec is at 
> https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15181) A simple implementation of date based tiered compaction

[Clara Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129366#comment-15129366
 ] 

Clara Xiong commented on HBASE-15181:
-

Thank you for your input. I will update the spec. 

We have date tiering like the spotify blog but with  tweaks. The overall theme 
is tiered store files based on the max time stamp for their data. If new data 
comes in perfect order, there is only one file per window except the incoming 
window or when they move across tiers. If there is out-of-order  data/late 
arrival/bulk loaded data, they will fall into older windows for compaction 
selection and may trigger compactions. We allow user to plug in a compaction 
policy for this case, default at exploring compaction, to reduce compaction 
storms. Other policy can be plugged in if user want to keep file count small by 
paying the price of re-compaction.

We use max time stamp instead of min as in the spotify blog to reduce 
performance penalty for out-of-order data, assuming no timestamp will be set to 
future time.

TTL works out of box by skipping the whole files.

Major compaction is disabled except pushed manually.

> A simple implementation of date based tiered compaction
> ---
>
> Key: HBASE-15181
> URL: https://issues.apache.org/jira/browse/HBASE-15181
> Project: HBase
>  Issue Type: New Feature
>  Components: Compaction
>Reporter: Clara Xiong
>Assignee: Clara Xiong
> Fix For: 2.0.0
>
> Attachments: HBASE-15181-v1.patch, HBASE-15181-v2.patch
>
>
> This is a simple implementation of date-based tiered compaction similar to 
> Cassandra's for the following benefits:
> 1. Improve date-range-based scan by structuring store files in date-based 
> tiered layout.
> 2. Reduce compaction overhead.
> 3. Improve TTL efficiency.
> Perfect fit for the use cases that:
> 1. has mostly date-based date write and scan and a focus on the most recent 
> data. 
> 2. never or rarely deletes data.
> Out-of-order writes are handled gracefully so the data will still get to the 
> right store file for time-range-scan and re-compacton with existing store 
> file in the same time window is handled by ExploringCompactionPolicy.
> Time range overlapping among store files is tolerated and the performance 
> impact is minimized.
> Configuration can be set at hbase-site or overriden at per-table or 
> per-column-famly level by hbase shell.
> Design spec is at 
> https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129365#comment-15129365
 ] 

Hadoop QA commented on HBASE-15206:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color} 
| {color:red} HBASE-15206 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/latest/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12785883/HBASE-15206-001_branch-1.2.patch
 |
| JIRA Issue | HBASE-15206 |
| Powered by | Apache Yetus 0.1.0   http://yetus.apache.org |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/405/console |


This message was automatically generated.



> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15194) TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky on trunk

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129353#comment-15129353
 ] 

Hudson commented on HBASE-15194:


SUCCESS: Integrated in HBase-1.3-IT #470 (See 
[https://builds.apache.org/job/HBase-1.3-IT/470/])
HBASE-15194 (stack: rev 1bf58675c7d91fcdd3b4965dd0b86b8702758538)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java


> TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky 
> on trunk
> 
>
> Key: HBASE-15194
> URL: https://issues.apache.org/jira/browse/HBASE-15194
> Project: HBase
>  Issue Type: Sub-task
>  Components: flakey, test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: disable.patch
>
>
> Fails 25% of the time: 
> https://builds.apache.org/job/PreCommit-HBASE-Build/349/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/testRegionReplicationOnMidClusterSameHosts/history/
> I'm just going to disable till someone has time to dig in on the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15158) Change order in which we do write pipeline operations; do all under row locks!

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129348#comment-15129348
 ] 

Hudson commented on HBASE-15158:


FAILURE: Integrated in HBase-Trunk_matrix #674 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/674/])
HBASE-15196 HBASE-15158 Preamble 2 of 2:Add Increment tests (stack: rev 
ed46591f30d4d66e89bc8157fc866c2aad77ce2c)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTags.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementsFromClientSide.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementFromClientSideWithCoprocessor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConcurrencyControl.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/IncrementPerformanceTest.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionIncrement.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSideWithCoprocessor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSmallTests.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ReplayHLogKey.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java
* hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestIncrement.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestFSHLog.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicaFailover.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java


> Change order in which we do write pipeline operations; do all under row locks!
> --
>
> Key: HBASE-15158
> URL: https://issues.apache.org/jira/browse/HBASE-15158
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 15158.patch, 15158v2.patch, 15158v3.patch
>
>
> Change how we do our write pipeline. I want to do all write pipeline ops 
> under row lock so I lean on this fact fixing performance regression in 
> check-and-set type operations like increment, append, and checkAnd* (see 
> sibling issue HBASE-15082).
> To be specific, we write like this now:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # add to memstore
> # let go of rowlock
> # sync WAL
> # in case of error: rollback memstore
> {code}
> Instead, write like this:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # sync WAL
> # add to memstore
> # let go of rowlock
> ... no need to do rollback.
> {code}
> The old ordering was put in place because it got better performance in a time 
> when WAL was different and before row locks were read/write (HBASE-12751).
> Testing in branch-1 shows that a reordering and skipping mvcc waits gets us 
> back to the performance we had before we unified mvcc and sequenceid 
> (HBASE-8763). Tests in HBASE-15046 show that at the macro level using our 
> usual perf tools, reordering pipeline seems to cause no slowdown (see 
> HBASE-15046). A rough compare of increments with reordered write pipeline 
> seems to have us getting back a bunch of our performance (see tail of 
> https://issues.apache.org/jira/browse/HBASE-15082?focusedCommentId=15111703&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15111703
>  and subsequent comment).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15192) TestRegionMergeTransactionOnCluster#testCleanMergeReference is flaky

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129350#comment-15129350
 ] 

Hudson commented on HBASE-15192:


FAILURE: Integrated in HBase-Trunk_matrix #674 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/674/])
HBASE-15192 TestRegionMergeTransactionOnCluster#testCleanMergeReference (tedyu: 
rev 243e6cc5293dc1e2a4dfd3af4ee29087c84184c8)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java


> TestRegionMergeTransactionOnCluster#testCleanMergeReference is flaky
> 
>
> Key: HBASE-15192
> URL: https://issues.apache.org/jira/browse/HBASE-15192
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15192.v1.patch, HBASE-15192.v2.patch
>
>
> TestRegionMergeTransactionOnCluster#testCleanMergeReference fails 
> intermittently due to failed assertion on cleaned merge region count:
> {code}
> testCleanMergeReference(org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster)
>   Time elapsed: 64.183 sec  <<< FAILURE!
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster.testCleanMergeReference(TestRegionMergeTransactionOnCluster.java:284)
> {code}
> Before calling CatalogJanitor#scan(), the test does:
> {code}
>   int newcount1 = 0;
>   while (System.currentTimeMillis() < timeout) {
> for(HColumnDescriptor colFamily : columnFamilies) {
>   newcount1 += hrfs.getStoreFiles(colFamily.getName()).size();
> }
> if(newcount1 <= 1) {
>   break;
> }
> Thread.sleep(50);
>   }
> {code}
> newcount1 is not cleared at the beginning of the loop.
> This means that if the check for newcount1 <= 1 doesn't pass the first 
> iteration, it wouldn't pass in subsequent iterations.
> After timeout is exhausted, admin.runCatalogScan() is called. However, there 
> is a chance that CatalogJanitor#scan() has been called by the Chore already 
> (during the wait period), leaving the cleaned count 0 and failing the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15196) HBASE-15158 Preamble 2 of 2:Add Increment tests

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129349#comment-15129349
 ] 

Hudson commented on HBASE-15196:


FAILURE: Integrated in HBase-Trunk_matrix #674 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/674/])
HBASE-15196 HBASE-15158 Preamble 2 of 2:Add Increment tests (stack: rev 
ed46591f30d4d66e89bc8157fc866c2aad77ce2c)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTags.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementsFromClientSide.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementFromClientSideWithCoprocessor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConcurrencyControl.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/IncrementPerformanceTest.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionIncrement.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSideWithCoprocessor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSmallTests.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ReplayHLogKey.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java
* hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestIncrement.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestFSHLog.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicaFailover.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java


> HBASE-15158 Preamble 2 of 2:Add Increment tests
> ---
>
> Key: HBASE-15196
> URL: https://issues.apache.org/jira/browse/HBASE-15196
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 15196.patch, 15196v2.patch, 15196v3.patch, 15196v3.patch
>
>
> Add the tests that were added by parent issue HBASE-15158; adds 
> IncrementPerformanceTest, TestRegionIncrement, and moves Increment testing 
> 'from client side' out into own test suite. This stuff is mostly forward-post 
> of tests  committed to branch-1 to test increment regressions 'narorow' fix 
> (See HBASE-15031). We are adding the tests separately here in an effort at 
> making the HBASE-15158 patch smaller. This is preamble 2 of 2. Preamble 1 was 
> HBASE-15186



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[huaxiang sun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

huaxiang sun updated HBASE-15206:
-
Attachment: HBASE-15206-001_branch-1.2.patch

patch for branch-1.2.

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch, HBASE-15206-001_branch-1.2.patch, 
> HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[huaxiang sun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129339#comment-15129339
 ] 

huaxiang sun commented on HBASE-15206:
--

Thanks Jon. I uploaded the patch which includes the comments. I am going to 
upload a patch for branch-1.2 as well.

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch, HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[huaxiang sun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

huaxiang sun updated HBASE-15206:
-
Attachment: HBASE-15206-002.patch

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch, HBASE-15206-002.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129291#comment-15129291
 ] 

Hadoop QA commented on HBASE-15206:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m 
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} master passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
22m 8s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 0s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 59s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 80m 28s 
{color} | {color:green} hbase-server in the patch passed with JDK v1.7.0_91. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
15s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 203m 29s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_72 Failed junit tests | 
hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-02-02 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12785831/HBASE-15206-001.patch 
|
| JIRA Issue | HBASE-15206 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux ff080b1a7fe3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129214#comment-15129214
 ] 

Jonathan Hsieh commented on HBASE-15206:


For #2. 

Got it. The catalogjanitor cleans up meta after splits.  So I think we either 
disable catalogjanitor and assert that the parent is still present (e.g. assert 
that hbck didn't remove the parent) , or force the catalog janitor to run and 
then assert that the parent is not (e.g. either hbck or the janitor removed). 

Since the first case is more deterministic I prefer it.  Can you add some 
comments to where we disable the catalog janitor with the justifcation for 
disabling?  Then I'll commit.

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15158) Change order in which we do write pipeline operations; do all under row locks!

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15158:
--
Attachment: 15158v3.patch

Just the changes in HRegion now (with a few little ripples).  Here is changelog:

{code}
   4 Subject: [PATCH] HBASE-15158 Change order in which we do write pipeline
   5  operations; do all under row locks!
   6
   7 M 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java
   8   White space
   9
  10 M 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  11   Removed timeout for latch that has break waiting on sequenceid
  12   assignment.
  13
  14   In flush, add doc and formatting. Be sure to complete mvcc on exit.
  15   Formatting is in part because complaint that method is too long so
  16   broke out stuff like fat loggint to own methods.
  17
  18   Undid an extraneous mvcc transaction that wrapped flush. This changed
  19   indent. Also moved call to sync out to WALUtil call.
  20
  21   Added logFatLineOnFlush and doAbortFlushWal, and
  22   doSyncOfUnflushedWALChange methods to make the flush method smaller.
  23
  24   Changed getNextSequenceId so it no longer adds fake edit to WAL only
  25   to have it discarded.. Since sequenceid font is inside mvcc, just get
  26   the next.
  27
  28   Renamed BatchOperationInProgress to BatchOperation and renamed
  29   doMiniBatchMutation as doMiniBatchMutate so it goes with its caller,
  30   batchMutate.
  31
  32   Some refactor of doMiniBatchMutate to make slightly smaller adding
  33   doc, breaking out methods, and removing redundancies but not enough. 
Needs rewrite.
  34
  35   Did the new common treatment of append/sync/mvcc careful on the end
  36   to call complete on mvcc before skipping out. This pattern is done
  37   in all wal writing in this patch. Also includes the flip in order
  38   where we do mvcc, append, sync, update memstore, stop mvcc instead
  39   of what we had previous which was mvcc, append, memstore, sync, mvcc.
  40
  41   Redo increment and append. They were 95% the same thing.
  42   Both now call doDelta.
  43
  44   Renamed doGet as get.
  45
  46   Removed main and a few DLR methods no longer used.
  47
  48 M 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSWALEntry.java
  49   Less dancing.
  50
  51 M 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
  52   Use methods because I want to clean up all the variants of sequenceid
  53   in here.
  54
  55 M 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALUtil.java
  56   Cleanup and do complete append, sync and mvcc in here rather than in
  57   caller. That is ok because methods in here are adding to WAL only, not
  58   to mem.
  59
  60 M hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALKey.java
  61   Doc. Cleanup.
  62
  63 M 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
  64   Test changes with above changes. Used to expect 'true' though we were
  65   asked to skip the WAL because our trick of doing fake empty append was
  66   tripping the coprocessor append flag.
{code}

> Change order in which we do write pipeline operations; do all under row locks!
> --
>
> Key: HBASE-15158
> URL: https://issues.apache.org/jira/browse/HBASE-15158
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 15158.patch, 15158v2.patch, 15158v3.patch
>
>
> Change how we do our write pipeline. I want to do all write pipeline ops 
> under row lock so I lean on this fact fixing performance regression in 
> check-and-set type operations like increment, append, and checkAnd* (see 
> sibling issue HBASE-15082).
> To be specific, we write like this now:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # add to memstore
> # let go of rowlock
> # sync WAL
> # in case of error: rollback memstore
> {code}
> Instead, write like this:
> {code}
> # take rowlock
> # start mvcc
> # append to WAL
> # sync WAL
> # add to memstore
> # let go of rowlock
> ... no need to do rollback.
> {code}
> The old ordering was put in place because it got better performance in a time 
> when WAL was different and before row locks were read/write (HBASE-12751).
> Testing in branch-1 shows that a reordering and skipping mvcc waits gets us 
> back to the performance we had before we unified mvcc and sequenceid 
> (HBASE-8763). Tests in HBASE-15046 show that at the macro level using our 
> usual perf tools, reordering pipeline seems to cause no slowdown (see 
> HBASE-15046). A rough compare of increments with reordered write pipeline 
> seems to have us getting 

[jira] [Commented] (HBASE-15202) Reduce garbage while setting response

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129190#comment-15129190
 ] 

Enis Soztutar commented on HBASE-15202:
---

Good one. Can we move Bytes.toBytes() to be inside IPCUtil instead. Bytes is a 
public facing class, and this CIS method is only for internal consumption. 
Other than that +1. 

> Reduce garbage while setting response
> -
>
> Key: HBASE-15202
> URL: https://issues.apache.org/jira/browse/HBASE-15202
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15202.patch, HBASE-15202_1.patch
>
>
> Probably this can move under HBASE-15177.  During RpcServer#setResponse we do 
> IPCUtil.getDelimitedMessageAsByteBuffer for the header and result. This 
> internally creates a byte[], CodedOutputStream and a BB. Then the totalSize 
> is also wrapped in a BB. 
> Finally all these BB are passed to BufferChain along with the Cellblock.
> This JIRA is to reduce the number of objects that we create here and allow 
> one BB from this header, result and total Size along with one 
> CodedOutputStream and pass this BB along with the Cellblocks to the 
> Bufferchain. From the JMC tool can observe around 2% lesser object allocation 
> arising out of these objects. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129187#comment-15129187
 ] 

Sean Busbey commented on HBASE-15177:
-

bq.  We do not have to depend on a released version.

I do not believe this is true, and would rather not get into it with ASF policy 
folks on the distinction between non-ASF folks who are not a part of the Yetus 
development community vs ASF folks who are not a part of the Yetus development 
community.

Yetus is blocked on their 0.2.0 release on me making a guide for the next 
release manager. Once that is in place, the plan is a weekly release cadence, 
which should be sufficient for these kinds of issues.

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15181) A simple implementation of date based tiered compaction

[Clara Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129144#comment-15129144
 ] 

Clara Xiong commented on HBASE-15181:
-

Uploaded to RB at https://reviews.apache.org/r/43114/.

> A simple implementation of date based tiered compaction
> ---
>
> Key: HBASE-15181
> URL: https://issues.apache.org/jira/browse/HBASE-15181
> Project: HBase
>  Issue Type: New Feature
>  Components: Compaction
>Reporter: Clara Xiong
>Assignee: Clara Xiong
> Fix For: 2.0.0
>
> Attachments: HBASE-15181-v1.patch, HBASE-15181-v2.patch
>
>
> This is a simple implementation of date-based tiered compaction similar to 
> Cassandra's for the following benefits:
> 1. Improve date-range-based scan by structuring store files in date-based 
> tiered layout.
> 2. Reduce compaction overhead.
> 3. Improve TTL efficiency.
> Perfect fit for the use cases that:
> 1. has mostly date-based date write and scan and a focus on the most recent 
> data. 
> 2. never or rarely deletes data.
> Out-of-order writes are handled gracefully so the data will still get to the 
> right store file for time-range-scan and re-compacton with existing store 
> file in the same time window is handled by ExploringCompactionPolicy.
> Time range overlapping among store files is tolerated and the performance 
> impact is minimized.
> Configuration can be set at hbase-site or overriden at per-table or 
> per-column-famly level by hbase shell.
> Design spec is at 
> https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129135#comment-15129135
 ] 

Enis Soztutar commented on HBASE-15177:
---

Thanks. Can we just default to using whatever yetus version works in the 
hadoopqa job. We do not have to depend on a released version. 

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129127#comment-15129127
 ] 

Sean Busbey commented on HBASE-15177:
-

The build error for module ordering is YETUS-280. you can build with this 
change in place if you go to the precommit job and use "build with parameters" 
to check the USE_YETUS_PRERELEASE option.

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15194) TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky on trunk

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129107#comment-15129107
 ] 

stack commented on HBASE-15194:
---

Applied to branch-1 because it failed here: See 


> TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky 
> on trunk
> 
>
> Key: HBASE-15194
> URL: https://issues.apache.org/jira/browse/HBASE-15194
> Project: HBase
>  Issue Type: Sub-task
>  Components: flakey, test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: disable.patch
>
>
> Fails 25% of the time: 
> https://builds.apache.org/job/PreCommit-HBASE-Build/349/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/testRegionReplicationOnMidClusterSameHosts/history/
> I'm just going to disable till someone has time to dig in on the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15194) TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky on trunk

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15194:
--
Fix Version/s: 1.3.0
   2.0.0

> TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky 
> on trunk
> 
>
> Key: HBASE-15194
> URL: https://issues.apache.org/jira/browse/HBASE-15194
> Project: HBase
>  Issue Type: Sub-task
>  Components: flakey, test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: disable.patch
>
>
> Fails 25% of the time: 
> https://builds.apache.org/job/PreCommit-HBASE-Build/349/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/testRegionReplicationOnMidClusterSameHosts/history/
> I'm just going to disable till someone has time to dig in on the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129103#comment-15129103
 ] 

Enis Soztutar commented on HBASE-15177:
---

[~busbey] is yetus only compiling the relevant module without applying the 
patch to the dependent modules? It seems that it is what is happening. 

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15194) TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky on trunk

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129102#comment-15129102
 ] 

stack commented on HBASE-15194:
---

Applied to branch-1 too. It looks like it was already disabled on branch-1.2 as 
part of this:

{code}
kalashnikov:hbase.git.commit stack$ git show f2338be4
commit f2338be4d84d1cff1eb27e87c1302f9d896d4744
Author: stack 
Date:   Tue Dec 29 09:51:51 2015 -0800

 HBASE-15023 Reenable TestShell and TestStochasticLoadBalancer; Amendment 
Disable testRegionReplicationOnMidClusterSameHosts because it fails on 1.2 
branch

{code}

> TestStochasticLoadBalancer.testRegionReplicationOnMidClusterSameHosts flaky 
> on trunk
> 
>
> Key: HBASE-15194
> URL: https://issues.apache.org/jira/browse/HBASE-15194
> Project: HBase
>  Issue Type: Sub-task
>  Components: flakey, test
>Reporter: stack
>Assignee: stack
> Attachments: disable.patch
>
>
> Fails 25% of the time: 
> https://builds.apache.org/job/PreCommit-HBASE-Build/349/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/testRegionReplicationOnMidClusterSameHosts/history/
> I'm just going to disable till someone has time to dig in on the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15177) Reduce garbage created under high load

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129088#comment-15129088
 ] 

Hadoop QA commented on HBASE-15177:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m 
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 17s 
{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 31s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 27s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_66. 
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 27s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 29s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_91. 
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 19s 
{color} | {color:red} Patch generated 3 new checkstyle issues in hbase-common 
(total was 7, now 10). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
22m 18s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 55s 
{color} | {color:red} hbase-common introduced 1 new FindBugs issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 48s {color} 
| {color:red} hbase-client in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 36s 
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_66. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 20s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 53s {color} 
| {color:red} hbase-client in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 42s 
{color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_91. 
{color} |
| {color:red}-1{color} | {color:red} unit

[jira] [Commented] (HBASE-15184) SparkSQL Scan operation doesn't work on kerberos cluster

[Ari Rabkin (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129074#comment-15129074
 ] 

Ari Rabkin commented on HBASE-15184:


Hi [~ted.m]!  Can you describe in a few sentences what the problem was and what 
needed overriding? 

> SparkSQL Scan operation doesn't work on kerberos cluster
> 
>
> Key: HBASE-15184
> URL: https://issues.apache.org/jira/browse/HBASE-15184
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Malaska
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBaseSparkModule.zip
>
>
> I was using the HBase Spark Module at a client with Kerberos and I ran into 
> an issue with the Scan.  
> I made a fix for the client but we need to put it back into HBase.  I will 
> attach my solution, but it has a major problem.  I had to over ride a 
> protected class in spark.  I will need help to decover a better approach



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[huaxiang sun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129070#comment-15129070
 ] 

huaxiang sun commented on HBASE-15206:
--

Thanks [~j...@cloudera.com] for reviewing. For 1), I think it may be case by 
case. 

For 2), 
Even if catalog janitor wakes up and cleans up the parent region, hbase fsck 
still works (from the unittest result). The issue is that it still expects the 
parent region be in the metatable after hbase fsck, which could be cleaned up 
by the catalog janitor. Either we disable the CatalogJanitor or remove this 
assert (I do not understand why this assert for parent region is required after 
split).

  Result result = meta.get(get);
  assertNotNull(result);
  assertNotNull(MetaTableAccessor.getHRegionInfo(result)); // make sure the 
parent region info still exists in meta table, failed here.

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15122) Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129038#comment-15129038
 ] 

Sean Busbey commented on HBASE-15122:
-

You'll have to make sure it's not still getting pulled in via e.g. the hadoop 
dependency

> Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
> ---
>
> Key: HBASE-15122
> URL: https://issues.apache.org/jira/browse/HBASE-15122
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Attachments: HBASE-15122-v0-master, HBASE-15122.patch, 
> HBASE-15122_v1.patch
>
>
> In our JMXJsonServlet we are doing this:
> jsonpcb = request.getParameter(CALLBACK_PARAM);
> if (jsonpcb != null) {
>   response.setContentType("application/javascript; charset=utf8");
>   writer.write(jsonpcb + "(");
> ... 
> Findbugs complains rightly. There are other instances in our servlets and 
> then there are the pages generated by jamon excluded from findbugs checking 
> (and findbugs volunteers that it is dumb in this regard finding only the most 
> egregious of violations).
> We have no sanitizing tooling in hbase that I know of (correct me if I am 
> wrong). I started to pull on this thread and it runs deep. Our Jamon 
> templating (last updated in 2013 and before that, in 2011) engine doesn't 
> seem to have sanitizing means either and there seems to be outstanding XSS 
> complaint against jamon that goes unaddressed.
> Could pull in something like 
> https://www.owasp.org/index.php/OWASP_Java_Encoder_Project and run all 
> emissions via it or get a templating engine that has sanitizing built in. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129027#comment-15129027
 ] 

Jonathan Hsieh commented on HBASE-15206:


Related is HBASE-10196 -- online region merge case.

> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15122) Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings

[Samir Ahmic (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129014#comment-15129014
 ] 

Samir Ahmic commented on HBASE-15122:
-

I can add exclusion for ESAPI and check how things will look like.

> Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
> ---
>
> Key: HBASE-15122
> URL: https://issues.apache.org/jira/browse/HBASE-15122
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Attachments: HBASE-15122-v0-master, HBASE-15122.patch, 
> HBASE-15122_v1.patch
>
>
> In our JMXJsonServlet we are doing this:
> jsonpcb = request.getParameter(CALLBACK_PARAM);
> if (jsonpcb != null) {
>   response.setContentType("application/javascript; charset=utf8");
>   writer.write(jsonpcb + "(");
> ... 
> Findbugs complains rightly. There are other instances in our servlets and 
> then there are the pages generated by jamon excluded from findbugs checking 
> (and findbugs volunteers that it is dumb in this regard finding only the most 
> egregious of violations).
> We have no sanitizing tooling in hbase that I know of (correct me if I am 
> wrong). I started to pull on this thread and it runs deep. Our Jamon 
> templating (last updated in 2013 and before that, in 2011) engine doesn't 
> seem to have sanitizing means either and there seems to be outstanding XSS 
> complaint against jamon that goes unaddressed.
> Could pull in something like 
> https://www.owasp.org/index.php/OWASP_Java_Encoder_Project and run all 
> emissions via it or get a templating engine that has sanitizing built in. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15206) Flakey testSplitDaughtersNotInMeta test

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129015#comment-15129015
 ] 

Jonathan Hsieh commented on HBASE-15206:


interesting.  Some of the tests disable the catalog jantior and others do not.  
Would it make sense to do it in all?

Can you add a comment about the problem that can be triggered if not present? 
If i understand correctly "the catalog janitor must be disabled in this test 
because it could come and clean up the inconsistency that we are trying to 
repair."



> Flakey testSplitDaughtersNotInMeta test
> ---
>
> Key: HBASE-15206
> URL: https://issues.apache.org/jira/browse/HBASE-15206
> Project: HBase
>  Issue Type: Bug
>  Components: flakey
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-15206-001.patch
>
>
> Run into the following failure with hbase 1.0.0.
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1723)
> From the log, the ntp issue caused clock skew and it woke up CatalogJanitor 
> earlier. The CatalogJanitor cleaned up the parent region. It could happen 
> with master branch as well. The fix is to disable CatalogJanitor to make sure 
> this will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15187) Integrate CSRF prevention filter to REST gateway

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128963#comment-15128963
 ] 

Hadoop QA commented on HBASE-15187:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 43s 
{color} | {color:red} hbase-rest in master has 2 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s 
{color} | {color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s 
{color} | {color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 31s 
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-rest 
(total was 30, now 30). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
25m 25s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 3s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 29s 
{color} | {color:green} hbase-rest in the patch passed with JDK v1.8.0_66. 
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 28s 
{color} | {color:green} hbase-rest in the patch passed with JDK v1.7.0_91. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
10s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 55s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-02-02 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12785834/HBASE-15187.v6.patch |
| JIRA Issue | HBASE-15187 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux d162ce4f72da 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMP

[jira] [Commented] (HBASE-15198) RPC client not using Codec and CellBlock for puts by default

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128938#comment-15128938
 ] 

Enis Soztutar commented on HBASE-15198:
---

+1. We should bring this to branch-1, and maybe earlier as well, since we were 
supposed to do codecs by default. 

> RPC client not using Codec and CellBlock for puts by default
> 
>
> Key: HBASE-15198
> URL: https://issues.apache.org/jira/browse/HBASE-15198
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Critical
> Attachments: HBASE-15198.patch, HBASE-15198_V2.patch
>
>
> For puts we use MultiServerCallable. Here to decide whether to use cellBlock 
> we have
> {code}
> private boolean isCellBlock() {
> // This is not exact -- the configuration could have changed on us after 
> connection was set up
> // but it will do for now.
> HConnection connection = getConnection();
> if (connection == null) return true; // Default is to do cellblocks.
> Configuration configuration = connection.getConfiguration();
> if (configuration == null) return true;
> String codec = configuration.get(HConstants.RPC_CODEC_CONF_KEY, "");
> return codec != null && codec.length() > 0;
>   }
> {code}
> By default in hbase-default.xml, we dont have any Codec being specified.
> Where as in AbstractRpcClient we have
> {code}
> Codec getCodec() {
> // For NO CODEC, "hbase.client.rpc.codec" must be configured with empty 
> string AND
> // "hbase.client.default.rpc.codec" also -- because default is to do cell 
> block encoding.
> String className = conf.get(HConstants.RPC_CODEC_CONF_KEY, 
> getDefaultCodec(this.conf));
> if (className == null || className.length() == 0) return null;
> try {
>   return (Codec)Class.forName(className).newInstance();
> } catch (Exception e) {
>   throw new RuntimeException("Failed getting codec " + className, e);
> }
>   }
> .
> public static String getDefaultCodec(final Configuration c) {
> // If "hbase.client.default.rpc.codec" is empty string -- you can't set 
> it to null because
> // Configuration will complain -- then no default codec (and we'll pb 
> everything).  Else
> // default is KeyValueCodec
> return c.get(DEFAULT_CODEC_CLASS, KeyValueCodec.class.getCanonicalName());
>   }
> {code}
> Our aim is to by def use Codec and it is KeyValueCodec.  
> The codec finding in MultiServerCallable to be same way as in 
> AbstractRpcClient and then only we will be doing cellblock stuff.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15122) Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128931#comment-15128931
 ] 

Sean Busbey commented on HBASE-15122:
-

okay. I'm going ot make a ticket to exclude Xerces from our hadoop dependency 
(they have it for some offline image editor or similar thing we don't need).

Can someone see if ESAPI can avoid it?

> Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
> ---
>
> Key: HBASE-15122
> URL: https://issues.apache.org/jira/browse/HBASE-15122
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Attachments: HBASE-15122-v0-master, HBASE-15122.patch, 
> HBASE-15122_v1.patch
>
>
> In our JMXJsonServlet we are doing this:
> jsonpcb = request.getParameter(CALLBACK_PARAM);
> if (jsonpcb != null) {
>   response.setContentType("application/javascript; charset=utf8");
>   writer.write(jsonpcb + "(");
> ... 
> Findbugs complains rightly. There are other instances in our servlets and 
> then there are the pages generated by jamon excluded from findbugs checking 
> (and findbugs volunteers that it is dumb in this regard finding only the most 
> egregious of violations).
> We have no sanitizing tooling in hbase that I know of (correct me if I am 
> wrong). I started to pull on this thread and it runs deep. Our Jamon 
> templating (last updated in 2013 and before that, in 2011) engine doesn't 
> seem to have sanitizing means either and there seems to be outstanding XSS 
> complaint against jamon that goes unaddressed.
> Could pull in something like 
> https://www.owasp.org/index.php/OWASP_Java_Encoder_Project and run all 
> emissions via it or get a templating engine that has sanitizing built in. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15203) Reduce garbage created by path.toString() during Checksum verification

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128921#comment-15128921
 ] 

Enis Soztutar commented on HBASE-15203:
---

+1. Let's bring this to branch-1 as well. 

> Reduce garbage created by path.toString() during Checksum verification
> --
>
> Key: HBASE-15203
> URL: https://issues.apache.org/jira/browse/HBASE-15203
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15203.patch, HBASE-15203_1.patch
>
>
> When we try to read a block we do checksum verification for which we need the 
> file name in which the block belongs to. So we do Path.toString() every time. 
> This seems to create around 163MB of char[] that is garbage collected in a 
> simple scan run. It is also visible in writes but the impact is lesser. In 
> overall write/read profile the top 2 factors are byte[] and char[]. This 
> toString() can easily be avoided and reduce its share from the total. To make 
> it more precise in 1 min of profiling, among the 1.8G of garbage created by 
> StringBuilder.toString - this path.toString() was contributing around 3.5%. 
> After the patch this is totally not there. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15177) Reduce garbage created under high load

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-15177:
--
Attachment: hbase-15177_v2.patch

rebased. 

> Reduce garbage created under high load
> --
>
> Key: HBASE-15177
> URL: https://issues.apache.org/jira/browse/HBASE-15177
> Project: HBase
>  Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.0
>
> Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 
> 2016-01-26 at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, 
> Screen Shot 2016-01-26 at 10.15.15 PM.png, hbase-15177_v0.patch, 
> hbase-15177_v1.patch, hbase-15177_v2.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was 
> to follow up on HBASE-14490 and experiment with offheap IPC byte buffers and 
> byte buffer re-use. However, without changing the IPC byte buffers for now, 
> there are a couple of (easy) improvements that I've identified from 
> profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead 
> of byte[] and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 
> 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly 
> it allocates the regionName byte[] to get the table name. We already set the 
> priority for most of the operations (multi, get, increment, etc) but we are 
> only reading the priority in case of multi. We should use the priority from 
> the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer 
> re-use in HBASE-14490. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15207) Stuck balancer; logging ten lines a millisecond

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128889#comment-15128889
 ] 

Enis Soztutar commented on HBASE-15207:
---

Seems like a blocker if balancer got stuck. 

> Stuck balancer; logging ten lines a millisecond
> ---
>
> Key: HBASE-15207
> URL: https://issues.apache.org/jira/browse/HBASE-15207
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 1.2.0
>Reporter: stack
>
> Last night my logs filled with this (10x256MB log files):
> 
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer: 
> Lowest locality region server with non zero regions is 
> ve0542.halxg.cloudera.com with locality 0.0
> 2016-02-01 11:25:26,958 DEBUG 
> [B.defaultRpcServer.handler=9,queue=0,port=16000] balancer.BaseLoadBalancer:  
> Lowest locality region index is 0 and its region server contains 1 regions
> ...
> Added by this:
> commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
> Author: tedyu 
> Date:   Thu Aug 13 09:11:59 2015 -0700
> HBASE-13376 Improvements to Stochastic load balancer (Vandana 
> Ayyalasomayajula)
> Looks like balancer got stuck. Logging at ten lines a millisecond.
> Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.
> {code}
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,572 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Lowest locality region server with non zero 
> regions is ve0540.halxg.cloudera.com with locality 0.0
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
> server contains 1 regions
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
> 2016-01-28 05:56:22,573 DEBUG 
> [ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
> balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
> 
> {code}
> Nothing else is happening on this master
> Happens just after a Master joins cluster after being killed by a monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15207) Stuck balancer; logging ten lines a millisecond

[stack (JIRA)]

stack created HBASE-15207:
-

 Summary: Stuck balancer; logging ten lines a millisecond
 Key: HBASE-15207
 URL: https://issues.apache.org/jira/browse/HBASE-15207
 Project: HBase
  Issue Type: Bug
  Components: Balancer
Affects Versions: 1.2.0
Reporter: stack


Last night my logs filled with this (10x256MB log files):


2016-02-01 11:25:26,958 DEBUG [B.defaultRpcServer.handler=9,queue=0,port=16000] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0542.halxg.cloudera.com with locality 0.0
2016-02-01 11:25:26,958 DEBUG [B.defaultRpcServer.handler=9,queue=0,port=16000] 
balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
server contains 1 regions
...


Added by this:

commit 54028140f4f19a6af81c8c8f29dda0c52491a0c9
Author: tedyu 
Date:   Thu Aug 13 09:11:59 2015 -0700

HBASE-13376 Improvements to Stochastic load balancer (Vandana 
Ayyalasomayajula)

Looks like balancer got stuck. Logging at ten lines a millisecond.

Here is lead up. Nothing in particular jumps out. Rerun doesn't show this.

{code}
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.0
2016-01-28 05:56:22,572 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
server contains 1 regions
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Lowest locality region server with non zero regions 
is ve0540.halxg.cloudera.com with locality 0.0
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer:  Lowest locality region index is 0 and its region 
server contains 1 regions
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0526.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0532.halxg.cloudera.com had 0 regions.
2016-01-28 05:56:22,573 DEBUG 
[ve0524.halxg.cloudera.com,16000,1453988766013_ChoreService_1] 
balancer.BaseLoadBalancer: Server ve0538.halxg.cloudera.com had 0 regions.

{code}

Nothing else is happening on this master

Happens just after a Master joins cluster after being killed by a monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)