[jira] [Updated] (HBASE-19918) Promote TestAsyncClusterAdminApi to LargeTests

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-19918:
---
Description: 
https://builds.apache.org/job/HBase%20Nightly/job/branch-2/221/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncClusterAdminApi/org_apache_hadoop_hbase_client_TestAsyncClusterAdminApi/

org.junit.runners.model.TestTimedOutException: test timed out after 180 seconds

Found this timeout in our branch-2 nightly jobs. And this test run more than 
110 seconds on my local computer.

  was:
org.junit.runners.model.TestTimedOutException: test timed out after 180 seconds

Found this timeout in our branch-2 nightly jobs. And this test run more than 
110 seconds on my local computer.


> Promote TestAsyncClusterAdminApi to LargeTests
> --
>
> Key: HBASE-19918
> URL: https://issues.apache.org/jira/browse/HBASE-19918
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Affects Versions: 2.0.0-beta-1
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2/221/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncClusterAdminApi/org_apache_hadoop_hbase_client_TestAsyncClusterAdminApi/
> org.junit.runners.model.TestTimedOutException: test timed out after 180 
> seconds
> Found this timeout in our branch-2 nightly jobs. And this test run more than 
> 110 seconds on my local computer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-19918) Promote TestAsyncClusterAdminApi to LargeTests

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-19918:
--

 Summary: Promote TestAsyncClusterAdminApi to LargeTests
 Key: HBASE-19918
 URL: https://issues.apache.org/jira/browse/HBASE-19918
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 2.0.0-beta-1
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang


org.junit.runners.model.TestTimedOutException: test timed out after 180 seconds

Found this timeout in our branch-2 nightly jobs. And this test run more than 
110 seconds on my local computer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19782) Reject the replication request when peer is DA or A state

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349918#comment-16349918
 ] 

Hadoop QA commented on HBASE-19782:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  3s{color} 
| {color:red} HBASE-19782 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19782 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908933/HBASE-19064-HBASE-19782.v2.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11353/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Reject the replication request when peer is DA or A state
> -
>
> Key: HBASE-19782
> URL: https://issues.apache.org/jira/browse/HBASE-19782
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19064-HBASE-19782.v1.patch, 
> HBASE-19064-HBASE-19782.v2.patch
>
>
> According to the design doc,  we'll initialize  both of the cluster state to 
> DA  after added the bidirectional  replication path.   and when a cluster in 
> DA state, it'll reject replication request.   
> so for cluster A and B in state DA,  if any received replication entry whose 
> table or namespace match the peer,the cluster will  skip to apply into 
> its local rs.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19782) Reject the replication request when peer is DA or A state

[Zheng Hu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-19782:
-
Attachment: HBASE-19064-HBASE-19782.v2.patch

> Reject the replication request when peer is DA or A state
> -
>
> Key: HBASE-19782
> URL: https://issues.apache.org/jira/browse/HBASE-19782
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19064-HBASE-19782.v1.patch, 
> HBASE-19064-HBASE-19782.v2.patch
>
>
> According to the design doc,  we'll initialize  both of the cluster state to 
> DA  after added the bidirectional  replication path.   and when a cluster in 
> DA state, it'll reject replication request.   
> so for cluster A and B in state DA,  if any received replication entry whose 
> table or namespace match the peer,the cluster will  skip to apply into 
> its local rs.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19904) Break dependency of WAL constructor on Replication

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349867#comment-16349867
 ] 

Duo Zhang commented on HBASE-19904:
---

Pushed to master first.

There are several differences between master and branch-2 in the replication 
implementation(HBASE-19397), upload the patch for branch-2 to see the pre 
commit result first before committing.

> Break dependency of WAL constructor on Replication
> --
>
> Key: HBASE-19904
> URL: https://issues.apache.org/jira/browse/HBASE-19904
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19904-branch-2.patch, HBASE-19904-v3.patch, 
> HBASE-19904-v3.patch, HBASE-19904-v4.patch, HBASE-19904-v4.patch, 
> HBASE-19904-v5.patch
>
>
> When implementing synchronous replication, I found that we need to depend 
> more on replication in WAL so it is even more pain...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19855) Refactor RegionScannerImpl.nextInternal method

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-19855:
---
Attachment: HBASE-19855.master.003.patch

> Refactor RegionScannerImpl.nextInternal method
> --
>
> Key: HBASE-19855
> URL: https://issues.apache.org/jira/browse/HBASE-19855
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-19855.master.002.patch, 
> HBASE-19855.master.003.patch
>
>
> Now this method is too complicated and confusing...
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19904) Break dependency of WAL constructor on Replication

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19904:
--
Attachment: HBASE-19904-branch-2.patch

> Break dependency of WAL constructor on Replication
> --
>
> Key: HBASE-19904
> URL: https://issues.apache.org/jira/browse/HBASE-19904
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19904-branch-2.patch, HBASE-19904-v3.patch, 
> HBASE-19904-v3.patch, HBASE-19904-v4.patch, HBASE-19904-v4.patch, 
> HBASE-19904-v5.patch
>
>
> When implementing synchronous replication, I found that we need to depend 
> more on replication in WAL so it is even more pain...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19901) Up yetus proclimit on nightlies

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349857#comment-16349857
 ] 

stack commented on HBASE-19901:
---

I reverted all changes made against this issue, then put back the originals. 
With DEBUG I see that the hbase_nightly is not picking up defines in 
hbase-personality... So nightly was failing because docker mem set to ... ? 
Ditto on proclimit. So hardcoded values in hbase_nightly. Not nice.  Nightly 
seems to be working now though. To be revisited. Here is what is committed 
currently...

commit b2fe241089dddaf65253ad882fbbff36225dafd2
Author: Michael Stack 
Date:   Thu Feb 1 20:54:13 2018 -0800

HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies; AMENDMENT hardcode 
proclimit and docker memlimit in nightly script...

diff --git a/dev-support/hbase_nightly_yetus.sh 
b/dev-support/hbase_nightly_yetus.sh
index 651a2e22d7..27d11c8beb 100755
--- a/dev-support/hbase_nightly_yetus.sh
+++ b/dev-support/hbase_nightly_yetus.sh
@@ -67,8 +67,9 @@ 
YETUS_ARGS=("--whitespace-tabs-ignore-list=${WHITESPACE_IGNORE_LIST}" "${YETUS_A
 YETUS_ARGS=("--sentinel" "${YETUS_ARGS[@]}")
 YETUS_ARGS=("--branch=${BRANCH_NAME}" "${YETUS_ARGS[@]}")
 YETUS_ARGS=("--tests-filter=${TESTS_FILTER}" "${YETUS_ARGS[@]}")
-YETUS_ARGS=("--proclimit=${PROCLIMIT}" "${YETUS_ARGS[@]}")
-YETUS_ARGS=("--dockermemlimit=${DOCKERMEMLIMIT}" "${YETUS_ARGS[@]}")
+# Why are these not being picked up from hbase-personality?
+YETUS_ARGS=("--proclimit=1" "${YETUS_ARGS[@]}")
+YETUS_ARGS=("--dockermemlimit=20g" "${YETUS_ARGS[@]}")

 # Currently, flaky list is calculated only for master branch.
 UNDERSCORED_BRANCH_NAME=$(echo ${BRANCH_NAME} | tr '.-' '_')


The defines in hbase-personality:



  # Yetus 0.7.0 enforces limits. Default proclimit is 1000.
  # Up it. See HBASE-19902 for how we arrived at this number.
  PROCLIMIT=1

  # Set docker container to run with 20g. Default is 4g in yetus.
  # See HBASE-19902 for how we arrived at 20g.
  DOCKERMEMLIMIT=20g


... are what are not being picked up. I traced download of hbase-personality as 
personality.sh... but don't see how it picked up (though yetus describes 
picking up these globals...) TODO.

> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19904) Break dependency of WAL constructor on Replication

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349834#comment-16349834
 ] 

Duo Zhang commented on HBASE-19904:
---

Let me commit after fixing the javadoc issues.

> Break dependency of WAL constructor on Replication
> --
>
> Key: HBASE-19904
> URL: https://issues.apache.org/jira/browse/HBASE-19904
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19904-v3.patch, HBASE-19904-v3.patch, 
> HBASE-19904-v4.patch, HBASE-19904-v4.patch, HBASE-19904-v5.patch
>
>
> When implementing synchronous replication, I found that we need to depend 
> more on replication in WAL so it is even more pain...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19855) Refactor RegionScannerImpl.nextInternal method

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-19855:
---
Attachment: HBASE-19855.master.002.patch

> Refactor RegionScannerImpl.nextInternal method
> --
>
> Key: HBASE-19855
> URL: https://issues.apache.org/jira/browse/HBASE-19855
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-19855.master.002.patch
>
>
> Now this method is too complicated and confusing...
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19855) Refactor RegionScannerImpl.nextInternal method

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-19855:
---
Attachment: (was: HBASE-19855.master.001.patch)

> Refactor RegionScannerImpl.nextInternal method
> --
>
> Key: HBASE-19855
> URL: https://issues.apache.org/jira/browse/HBASE-19855
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
> Now this method is too complicated and confusing...
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19855) Refactor RegionScannerImpl.nextInternal method

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-19855:
---
Attachment: (was: HBASE-19855.master.001.patch)

> Refactor RegionScannerImpl.nextInternal method
> --
>
> Key: HBASE-19855
> URL: https://issues.apache.org/jira/browse/HBASE-19855
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
> Now this method is too complicated and confusing...
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19121) HBCK for AMv2

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19121:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> HBCK for AMv2
> -
>
> Key: HBASE-19121
> URL: https://issues.apache.org/jira/browse/HBASE-19121
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0
>
>
> We don't have an hbck for the new AM. Old hbck may actually do damage going 
> against AMv2.
> Fix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19121) HBCK for AMv2

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349801#comment-16349801
 ] 

stack commented on HBASE-19121:
---

Need this but we could ship a beta-2 w/o it so moving out.

> HBCK for AMv2
> -
>
> Key: HBASE-19121
> URL: https://issues.apache.org/jira/browse/HBASE-19121
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0
>
>
> We don't have an hbck for the new AM. Old hbck may actually do damage going 
> against AMv2.
> Fix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19904) Break dependency of WAL constructor on Replication

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349792#comment-16349792
 ] 

Hadoop QA commented on HBASE-19904:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 35 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
30s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch hbase-replication passed checkstyle 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
19s{color} | {color:green} hbase-server: The patch generated 0 new + 1130 
unchanged - 18 fixed = 1130 total (was 1148) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
17s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
46s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
18m 47s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 
30s{color} | {color:red} hbase-server generated 2 new + 2 unchanged - 0 fixed = 
4 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
21s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 
35s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m 
20s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
57s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 11s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19904 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908907/HBASE-19904-v5.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  

[jira] [Commented] (HBASE-19914) TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category annotation

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349793#comment-16349793
 ] 

Hadoop QA commented on HBASE-19914:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
13s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
3s{color} | {color:red} hbase-server: The patch generated 2 new + 1 unchanged - 
13 fixed = 3 total (was 14) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
40s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
18m 25s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
55s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 11s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
38s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}145m 32s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestFullLogReconstruction |
|   | 
hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService 
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19914 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908910/HBASE-19914-v1.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 32f75c167861 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / a11258599e |
| maven | version: Apache Maven 

[jira] [Commented] (HBASE-19886) Display maintenance mode in shell, web UI, JMX

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349790#comment-16349790
 ] 

stack commented on HBASE-19886:
---

Is it appropriate having this flag as a metric [~balazs.meszaros]? Thanks.

> Display maintenance mode in shell, web UI, JMX
> --
>
> Key: HBASE-19886
> URL: https://issues.apache.org/jira/browse/HBASE-19886
> Project: HBase
>  Issue Type: New Feature
>Affects Versions: 2.0.0, 3.0.0, 1.4.2
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.4.2
>
> Attachments: HBASE-19886.master.001.patch
>
>
> Maintenance mode was introduced in HBASE-16008. This mode is controlled by 
> hbck. Splitting an balancing is disabled in this mode. It would be useful to 
> present this information to users through shell, web UI, JMX.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-19896) Fix ScanInfo.customize() to update only the requested options

[Ankit Singhal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Singhal resolved HBASE-19896.
---
   Resolution: Fixed
Fix Version/s: 2.0.0-beta-2

> Fix ScanInfo.customize() to update only the requested options
> -
>
> Key: HBASE-19896
> URL: https://issues.apache.org/jira/browse/HBASE-19896
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> It seems that ScanInfo.customize() is changing 
> options(timeToPurgeDeletes,tableMaxRowSize,cellsPerTimeoutCheck,preadMaxBytes)
>  along with requested one's (maxVersions,ttl)
> FYI [~tedyu]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19896) Fix ScanInfo.customize() to update only the requested options

[Ankit Singhal (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349789#comment-16349789
 ] 

Ankit Singhal commented on HBASE-19896:
---

Fix for this is committed as part of HBASE-19895. 

> Fix ScanInfo.customize() to update only the requested options
> -
>
> Key: HBASE-19896
> URL: https://issues.apache.org/jira/browse/HBASE-19896
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> It seems that ScanInfo.customize() is changing 
> options(timeToPurgeDeletes,tableMaxRowSize,cellsPerTimeoutCheck,preadMaxBytes)
>  along with requested one's (maxVersions,ttl)
> FYI [~tedyu]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19895) Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks

[Ankit Singhal (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349787#comment-16349787
 ] 

Ankit Singhal commented on HBASE-19895:
---

Thanks [~yuzhih...@gmail.com] for committing it.

> Add keepDeletedCells option in ScanOptions for customizing scanInfo in 
> pre-hooks
> 
>
> Key: HBASE-19895
> URL: https://issues.apache.org/jira/browse/HBASE-19895
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19895.v2.txt, HBASE-19895.patch, HBASE-19895_v1.patch, 
> HBASE-19895_v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19912) The flag "writeToWAL" of Region#checkAndRowMutate is useless

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349786#comment-16349786
 ] 

stack commented on HBASE-19912:
---

Thank you [~kewang]

> The flag "writeToWAL" of Region#checkAndRowMutate is useless
> 
>
> Key: HBASE-19912
> URL: https://issues.apache.org/jira/browse/HBASE-19912
> Project: HBase
>  Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Mu Chun Wang
>Priority: Minor
>  Labels: beginner
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19912.branch-2.v0.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19901) Up yetus proclimit on nightlies

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349746#comment-16349746
 ] 

stack commented on HBASE-19901:
---

Arghh... After revert, nightlies got further... running unit tests... but 
failing in hbase-procedure... not saying why.

> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19897) RowMutations should follow the fluent pattern

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349739#comment-16349739
 ] 

Hudson commented on HBASE-19897:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4511 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4511/])
HBASE-19897 RowMutations should follow the fluent pattern (chia7712: rev 
adccbb7edf3986701d54b9ca93fcf5a8e99548fb)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RowMutations.java


> RowMutations should follow the fluent pattern
> -
>
> Key: HBASE-19897
> URL: https://issues.apache.org/jira/browse/HBASE-19897
> Project: HBase
>  Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19897.v0.patch, HBASE-19897.v1.patch, 
> HBASE-19897.v1.patch
>
>
> Other row ops, including {{Put}}, {{Delete}}, {{Get}}, {{Scan}}, do have the 
> fluent interface. Also, Changing the return type from {{Void}} to 
> {{RowMutations}} won't break the API BC (unless someone has interest in 
> {{Void}} object...)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19913) Split TestStochasticLoadBalancer2

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349742#comment-16349742
 ] 

Hudson commented on HBASE-19913:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4511 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4511/])
HBASE-19913 Split TestStochasticLoadBalancer2 (zhangduo: rev 
94dad844af239eafd09dac951bbc9ac2243fac82)
* (delete) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
* (add) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaLargeCluster.java
* (add) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaReplicationGreaterThanNumNodes.java
* (add) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaHighReplication.java
* (add) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase2.java
* (add) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaMidCluster.java


> Split TestStochasticLoadBalancer2
> -
>
> Key: HBASE-19913
> URL: https://issues.apache.org/jira/browse/HBASE-19913
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19913.patch, HBASE-19913.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19901) Up yetus proclimit on nightlies

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349740#comment-16349740
 ] 

Hudson commented on HBASE-19901:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4511 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4511/])
HBASE-19901 Up yetus proclimit on nightlies; ADDENDUM -- up proclimit (stack: 
rev 0db7db3cd1022f794c1cf6d83858e7c83964cd73)
* (edit) dev-support/hbase_nightly_yetus.sh
* (edit) dev-support/hbase-personality.sh
HBASE-19901 Up yetus proclimit on nightlies; ADDENDUM -- remove docker (stack: 
rev 5f5ddf55416a09c9675a0026b75dfc2aaf12632d)
* (edit) dev-support/hbase-personality.sh
* (edit) dev-support/hbase_nightly_yetus.sh


> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19147) All branch-2 unit tests pass

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349741#comment-16349741
 ] 

Hudson commented on HBASE-19147:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4511 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4511/])
HBASE-19147 TestCacheOnWrite Times Out (stack: rev 
3622bb0333ea1b7c05fd47e5e0fee61b82f08109)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java


> All branch-2 unit tests pass
> 
>
> Key: HBASE-19147
> URL: https://issues.apache.org/jira/browse/HBASE-19147
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19914) TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category annotation

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349695#comment-16349695
 ] 

Duo Zhang commented on HBASE-19914:
---

Fix the checkstyle issues.

> TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category 
> annotation
> ---
>
> Key: HBASE-19914
> URL: https://issues.apache.org/jira/browse/HBASE-19914
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19914-v1.patch, HBASE-19914.patch, 
> HBASE-19914.patch
>
>
> And both TestVisibilityLabelsOnNewVersionBehaviorTable and its parent class 
> run about 2 minutes, which is not safe to declared as MediumTests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19914) TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category annotation

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19914:
--
Attachment: HBASE-19914-v1.patch

> TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category 
> annotation
> ---
>
> Key: HBASE-19914
> URL: https://issues.apache.org/jira/browse/HBASE-19914
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19914-v1.patch, HBASE-19914.patch, 
> HBASE-19914.patch
>
>
> And both TestVisibilityLabelsOnNewVersionBehaviorTable and its parent class 
> run about 2 minutes, which is not safe to declared as MediumTests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19895) Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19895:
---
Attachment: 19895.v2.txt

> Add keepDeletedCells option in ScanOptions for customizing scanInfo in 
> pre-hooks
> 
>
> Key: HBASE-19895
> URL: https://issues.apache.org/jira/browse/HBASE-19895
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19895.v2.txt, HBASE-19895.patch, HBASE-19895_v1.patch, 
> HBASE-19895_v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19895) Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19895:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Addressed checkstyle warnings and pushed to branch-2 and master.

Thanks for the patch, Ankit

> Add keepDeletedCells option in ScanOptions for customizing scanInfo in 
> pre-hooks
> 
>
> Key: HBASE-19895
> URL: https://issues.apache.org/jira/browse/HBASE-19895
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19895.v2.txt, HBASE-19895.patch, HBASE-19895_v1.patch, 
> HBASE-19895_v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19904) Break dependency of WAL constructor on Replication

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19904:
--
Attachment: HBASE-19904-v5.patch

> Break dependency of WAL constructor on Replication
> --
>
> Key: HBASE-19904
> URL: https://issues.apache.org/jira/browse/HBASE-19904
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19904-v3.patch, HBASE-19904-v3.patch, 
> HBASE-19904-v4.patch, HBASE-19904-v4.patch, HBASE-19904-v5.patch
>
>
> When implementing synchronous replication, I found that we need to depend 
> more on replication in WAL so it is even more pain...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19904) Break dependency of WAL constructor on Replication

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349687#comment-16349687
 ] 

Duo Zhang commented on HBASE-19904:
---

Address the comments on rb.

> Break dependency of WAL constructor on Replication
> --
>
> Key: HBASE-19904
> URL: https://issues.apache.org/jira/browse/HBASE-19904
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19904-v3.patch, HBASE-19904-v3.patch, 
> HBASE-19904-v4.patch, HBASE-19904-v4.patch, HBASE-19904-v5.patch
>
>
> When implementing synchronous replication, I found that we need to depend 
> more on replication in WAL so it is even more pain...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19895) Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19895:
---
Summary: Add keepDeletedCells option in ScanOptions for customizing 
scanInfo in pre-hooks  (was: Add keepDeletedCells option in ScanOptions for 
customize scanInfo in pre-hooks)

> Add keepDeletedCells option in ScanOptions for customizing scanInfo in 
> pre-hooks
> 
>
> Key: HBASE-19895
> URL: https://issues.apache.org/jira/browse/HBASE-19895
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19895.patch, HBASE-19895_v1.patch, 
> HBASE-19895_v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-19841:
--
Release Note: HBaseTestingUtility now assumes that all clusters will use 
local storage until a MiniDFSCluster is started or assigned.

> Tests against hadoop3 fail with StreamLacksCapabilityException
> --
>
> Key: HBASE-19841
> URL: https://issues.apache.org/jira/browse/HBASE-19841
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19841.007.patch, 19841.06.patch, 19841.v0.txt, 
> 19841.v1.txt, HBASE-19841.v10.patch, HBASE-19841.v11.patch, 
> HBASE-19841.v11.patch, HBASE-19841.v2.patch, HBASE-19841.v3.patch, 
> HBASE-19841.v4.patch, HBASE-19841.v5.patch, HBASE-19841.v7.patch, 
> HBASE-19841.v8.patch, HBASE-19841.v8.patch, HBASE-19841.v8.patch, 
> HBASE-19841.v9.patch
>
>
> The following can be observed running against hadoop3:
> {code}
> java.io.IOException: cannot get log writer
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> {code}
> This was due to hbase-server/src/test/resources/hbase-site.xml not being 
> picked up by Configuration object. Among the configs from this file, the 
> value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence 
> of hflush and hsync. Without this config entry,  
> StreamLacksCapabilityException is thrown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-19841:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Pushed to master and branch-2. Thanks for reviews, [~elserj], [~stack], [~tedyu]

> Tests against hadoop3 fail with StreamLacksCapabilityException
> --
>
> Key: HBASE-19841
> URL: https://issues.apache.org/jira/browse/HBASE-19841
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19841.007.patch, 19841.06.patch, 19841.v0.txt, 
> 19841.v1.txt, HBASE-19841.v10.patch, HBASE-19841.v11.patch, 
> HBASE-19841.v11.patch, HBASE-19841.v2.patch, HBASE-19841.v3.patch, 
> HBASE-19841.v4.patch, HBASE-19841.v5.patch, HBASE-19841.v7.patch, 
> HBASE-19841.v8.patch, HBASE-19841.v8.patch, HBASE-19841.v8.patch, 
> HBASE-19841.v9.patch
>
>
> The following can be observed running against hadoop3:
> {code}
> java.io.IOException: cannot get log writer
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> {code}
> This was due to hbase-server/src/test/resources/hbase-site.xml not being 
> picked up by Configuration object. Among the configs from this file, the 
> value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence 
> of hflush and hsync. Without this config entry,  
> StreamLacksCapabilityException is thrown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19912) The flag "writeToWAL" of Region#checkAndRowMutate is useless

[Mu Chun Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349683#comment-16349683
 ] 

Mu Chun Wang commented on HBASE-19912:
--

Thanks a lot, it is my first contribution at HBase. XDDD

> The flag "writeToWAL" of Region#checkAndRowMutate is useless
> 
>
> Key: HBASE-19912
> URL: https://issues.apache.org/jira/browse/HBASE-19912
> Project: HBase
>  Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Mu Chun Wang
>Priority: Minor
>  Labels: beginner
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19912.branch-2.v0.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19917) Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient

[Xiang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Li updated HBASE-19917:
-
Description: 
{code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}
{code}

filterServers is to return the union of servers and onlineServers. The current 
implementation has time complexity as O(m*n) (2 loops), could be in O(m + n) if 
HashSet is used. The trade-off is space complexity is increased.

Another point which could be improved: filterServers() is only called in 
filterOfflineServers(). filterOfflineServers calls filterServers(Set, List). 
The current filterServers(Collection, Collection) seems could be improved.

  was:
{code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}
{code}

filterServers is to return the union of servers and onlineServers. The current 
implementation has time complexity as O(m*n) (2 loops), could be in O(m + n) if 
HashSet is used.

Another point which could be improved: filterServers() is only called in 
filterOfflineServers(). filterOfflineServers calls filterServers(Set, List). 
The current filterServers(Collection, Collection) seems could be improved.


> Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient
> -
>
> Key: HBASE-19917
> URL: https://issues.apache.org/jira/browse/HBASE-19917
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
> private List filterServers(Collection servers,
> Collection onlineServers) {
>   ArrayList finalList = new ArrayList();
>   for (Address server : servers) {
> for(ServerName curr: onlineServers) {
>   if(curr.getAddress().equals(server)) {
> finalList.add(curr);
>   }
> }
>   }
>   return finalList;
> }
> {code}
> filterServers is to return the union of servers and onlineServers. The 
> current implementation has time complexity as O(m*n) (2 loops), could be in 
> O(m + n) if HashSet is used. The trade-off is space complexity is increased.
> Another point which could be improved: filterServers() is only called in 
> filterOfflineServers(). filterOfflineServers calls filterServers(Set, List). 
> The current filterServers(Collection, Collection) seems could be improved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19901) Up yetus proclimit on nightlies

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349667#comment-16349667
 ] 

stack commented on HBASE-19901:
---

Ok. Reverted original patch too. Back to square one. I can't get nightlies to 
build. Failing cryptically. Starting over.

> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19899) Dump ulimit -a, fd count, and free output at end of build into system dir

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349643#comment-16349643
 ] 

stack commented on HBASE-19899:
---

Sorry about that [~apurtell] Thanks.

> Dump ulimit -a, fd count, and free output at end of build into system dir
> -
>
> Key: HBASE-19899
> URL: https://issues.apache.org/jira/browse/HBASE-19899
> Project: HBase
>  Issue Type: Improvement
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 
> 0001-HBASE-19899-Dump-ulimit-a-fd-count-and-free-output-a.patch
>
>
> We're OOME'ing unable to create threads. I added ulimit -l, free -h, and 
> count of open fds to hadoopqa just now. Add them to the script used by 
> JenkinsFile too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19904) Break dependency of WAL constructor on Replication

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-19904:
-
Summary: Break dependency of WAL constructor on Replication  (was: Find a 
better way to deal with the cyclic dependency when initializing WAL and 
Replication)

> Break dependency of WAL constructor on Replication
> --
>
> Key: HBASE-19904
> URL: https://issues.apache.org/jira/browse/HBASE-19904
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19904-v3.patch, HBASE-19904-v3.patch, 
> HBASE-19904-v4.patch, HBASE-19904-v4.patch
>
>
> When implementing synchronous replication, I found that we need to depend 
> more on replication in WAL so it is even more pain...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19858) Backport HBASE-14061 (Support CF-level Storage Policy) to branch-1

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19858:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to branch-1

> Backport HBASE-14061 (Support CF-level Storage Policy) to branch-1
> --
>
> Key: HBASE-19858
> URL: https://issues.apache.org/jira/browse/HBASE-19858
> Project: HBase
>  Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-19858-branch-1.patch
>
>
> Backport the following commits to branch-1:
>  * HBASE-14061 Support CF-level Storage Policy
>  * HBASE-14061 Support CF-level Storage Policy (addendum)
>  * HBASE-14061 Support CF-level Storage Policy (addendum2)
>  * HBASE-15172 Support setting storage policy in bulkload
>  * HBASE-17538 HDFS.setStoragePolicy() logs errors on local fs
>  * HBASE-18015 Storage class aware block placement for procedure v2 WALs
>  * HBASE-18017 Reduce frequency of setStoragePolicy failure warnings
>  * HBASE-19016 Coordinate storage policy property name for table schema and 
> bulkload
>  
> Fix
>  * Default storage policy if not configured cannot be "NONE"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19917) Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient

[Xiang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Li updated HBASE-19917:
-
Description: 
{code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}
{code}

filterServers is to return the union of servers and onlineServers. The current 
implementation has time complexity as O(m*n) (2 loops), could be in O(m + n) if 
HashSet is used.

Another point which could be improved: filterServers() is only called in 
filterOfflineServers(). filterOfflineServers calls filterServers(Set, List). 
The current filterServers(Collection, Collection) seems could be improved.

  was:
{code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}
{code}

filterServers is to return the union of servers and onlineServers. The current 
implementation has time complexity as O(m*n) (2 loops), could be in O(m + n) if 
HashSet is used.

One more,  filterServers() is only called in filterOfflineServers(). 
filterOfflineServers calls filterServers(Set, List). The current 
filterServers(Collection, Collection) seems could be improved.


> Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient
> -
>
> Key: HBASE-19917
> URL: https://issues.apache.org/jira/browse/HBASE-19917
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
> private List filterServers(Collection servers,
> Collection onlineServers) {
>   ArrayList finalList = new ArrayList();
>   for (Address server : servers) {
> for(ServerName curr: onlineServers) {
>   if(curr.getAddress().equals(server)) {
> finalList.add(curr);
>   }
> }
>   }
>   return finalList;
> }
> {code}
> filterServers is to return the union of servers and onlineServers. The 
> current implementation has time complexity as O(m*n) (2 loops), could be in 
> O(m + n) if HashSet is used.
> Another point which could be improved: filterServers() is only called in 
> filterOfflineServers(). filterOfflineServers calls filterServers(Set, List). 
> The current filterServers(Collection, Collection) seems could be improved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19917) Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient

[Xiang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Li updated HBASE-19917:
-
Description: 
{code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}
{code}

filterServers is to return the union of servers and onlineServers. The current 
implementation has time complexity as O(m*n) (2 loops), could be in O(m + n) if 
HashSet is used.

One more,  filterServers() is only called in filterOfflineServers(). 
filterOfflineServers calls filterServers(Set, List). The current 
filterServers(Collection, Collection) seems could be improved.

  was:
{code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}
{code}


> Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient
> -
>
> Key: HBASE-19917
> URL: https://issues.apache.org/jira/browse/HBASE-19917
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
> private List filterServers(Collection servers,
> Collection onlineServers) {
>   ArrayList finalList = new ArrayList();
>   for (Address server : servers) {
> for(ServerName curr: onlineServers) {
>   if(curr.getAddress().equals(server)) {
> finalList.add(curr);
>   }
> }
>   }
>   return finalList;
> }
> {code}
> filterServers is to return the union of servers and onlineServers. The 
> current implementation has time complexity as O(m*n) (2 loops), could be in 
> O(m + n) if HashSet is used.
> One more,  filterServers() is only called in filterOfflineServers(). 
> filterOfflineServers calls filterServers(Set, List). The current 
> filterServers(Collection, Collection) seems could be improved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19917) Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient

[Xiang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiang Li updated HBASE-19917:
-
Description: 
{code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}
{code}

  was:
private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}


> Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient
> -
>
> Key: HBASE-19917
> URL: https://issues.apache.org/jira/browse/HBASE-19917
> Project: HBase
>  Issue Type: Improvement
>  Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java|borderStyle=solid}
> private List filterServers(Collection servers,
> Collection onlineServers) {
>   ArrayList finalList = new ArrayList();
>   for (Address server : servers) {
> for(ServerName curr: onlineServers) {
>   if(curr.getAddress().equals(server)) {
> finalList.add(curr);
>   }
> }
>   }
>   return finalList;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19899) Dump ulimit -a, fd count, and free output at end of build into system dir

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349598#comment-16349598
 ] 

Andrew Purtell commented on HBASE-19899:


Wasn't reverted on branch-1. Now reverted there too

> Dump ulimit -a, fd count, and free output at end of build into system dir
> -
>
> Key: HBASE-19899
> URL: https://issues.apache.org/jira/browse/HBASE-19899
> Project: HBase
>  Issue Type: Improvement
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 
> 0001-HBASE-19899-Dump-ulimit-a-fd-count-and-free-output-a.patch
>
>
> We're OOME'ing unable to create threads. I added ulimit -l, free -h, and 
> count of open fds to hadoopqa just now. Add them to the script used by 
> JenkinsFile too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19728) Add lock to filesCompacting in all place.

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19728:
---
Fix Version/s: 1.4.2

> Add lock to filesCompacting in all place.
> -
>
> Key: HBASE-19728
> URL: https://issues.apache.org/jira/browse/HBASE-19728
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0, 3.0.0, 1.5.0
>Reporter: binlijin
>Assignee: binlijin
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.5.0, 1.4.2
>
> Attachments: HBASE-19728.branch-1.001.patch, 
> HBASE-19728.master.001.patch, HBASE-19728.master.002.patch, 
> HBASE-19728.master.002.patch
>
>
> We find regionserver abort with the following exception:
> 2017-05-09 17:40:06,369 FATAL 
> [regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275]
>  regionserver.HRegionServer: ABORTING region server 
> hadoop0349.et2.tbsite.net,16020,1493026637177: 
> Thread[regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275,5,main]
>  throw uncaught exception
> java.lang.ArrayIndexOutOfBoundsException
>         at java.lang.System.arraycopy(Native Method)
>         at java.util.ArrayList.batchRemove(ArrayList.java:726)
>         at java.util.ArrayList.removeAll(ArrayList.java:690)
>         at 
> org.apache.hadoop.hbase.regionserver.HStore.finishCompactionRequest(HStore.java:1666)
>         at 
> org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1656)
>         at 
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:504)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
>         at java.lang.Thread.run(Thread.java:834)
> 2017-05-08 21:15:31,979 FATAL 
> [regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978]
>  regionserver.HRegionServer: ABORTING region server 
> hadoop1191.et2.tbsite.net,16020,1493196567798: 
> Thread[regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978,5,main]
>  throw uncaught exception
> java.lang.IllegalArgumentException
>         at 
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:76)
>         at 
> org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.getCurrentEligibleFiles(RatioBasedCompactionPolicy.java:64)
>         at 
> org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.preSelectCompactionForCoprocessor(RatioBasedCompactionPolicy.java:72)
>         at 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.preSelect(DefaultStoreEngine.java:117)
>         at 
> org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1542)
>         at 
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.selectCompaction(CompactSplitThread.java:362)
>         at 
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.access$200(CompactSplitThread.java:58)
>         at 
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:491)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
>         at java.lang.Thread.run(Thread.java:834)
> HStore#finishCompactionRequest do not require any HStore#lock's lock so 
> HStore.replaceStoreFiles need to synchronized on filesCompacting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-19917) Improve RSGroupBasedLoadBalancer#filterServers() to be more efficient

[Xiang Li (JIRA)]

Xiang Li created HBASE-19917:


 Summary: Improve RSGroupBasedLoadBalancer#filterServers() to be 
more efficient
 Key: HBASE-19917
 URL: https://issues.apache.org/jira/browse/HBASE-19917
 Project: HBase
  Issue Type: Improvement
  Components: rsgroup
Reporter: Xiang Li
Assignee: Xiang Li


private List filterServers(Collection servers,
Collection onlineServers) {
  ArrayList finalList = new ArrayList();
  for (Address server : servers) {
for(ServerName curr: onlineServers) {
  if(curr.getAddress().equals(server)) {
finalList.add(curr);
  }
}
  }
  return finalList;
}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19901) Up yetus proclimit on nightlies

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349597#comment-16349597
 ] 

stack commented on HBASE-19901:
---

Pushed a new addendum that removes docker limit for the moment. Having trouble 
getting nightlies to launch. This is what I pushed as addendum number two.

commit 5f5ddf55416a09c9675a0026b75dfc2aaf12632d
Author: Michael Stack 
Date:   Thu Feb 1 16:38:50 2018 -0800

HBASE-19901 Up yetus proclimit on nightlies; ADDENDUM -- remove docker mem 
sizing for the moment...

diff --git a/dev-support/hbase-personality.sh b/dev-support/hbase-personality.sh
index 5526e09d59..ddd0df4d2b 100755
--- a/dev-support/hbase-personality.sh
+++ b/dev-support/hbase-personality.sh
@@ -71,7 +71,7 @@ function personality_globals

   # Set docker container to run with 20g. Default is 4g in yetus.
   # See HBASE-19902 for how we arrived at 20g.
-  DOCKERMEMLIMIT=20g
+  # UNUSED AT MOMENT DOCKERMEMLIMIT=20g
 }

 ## @description  Parse extra arguments required by personalities, if any.
diff --git a/dev-support/hbase_nightly_yetus.sh 
b/dev-support/hbase_nightly_yetus.sh
index 651a2e22d7..3e6fc7c97d 100755
--- a/dev-support/hbase_nightly_yetus.sh
+++ b/dev-support/hbase_nightly_yetus.sh
@@ -68,7 +68,6 @@ YETUS_ARGS=("--sentinel" "${YETUS_ARGS[@]}")
 YETUS_ARGS=("--branch=${BRANCH_NAME}" "${YETUS_ARGS[@]}")
 YETUS_ARGS=("--tests-filter=${TESTS_FILTER}" "${YETUS_ARGS[@]}")
 YETUS_ARGS=("--proclimit=${PROCLIMIT}" "${YETUS_ARGS[@]}")
-YETUS_ARGS=("--dockermemlimit=${DOCKERMEMLIMIT}" "${YETUS_ARGS[@]}")

 # Currently, flaky list is calculated only for master branch.
 UNDERSCORED_BRANCH_NAME=$(echo ${BRANCH_NAME} | tr '.-' '_')

> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349591#comment-16349591
 ] 

Hadoop QA commented on HBASE-19841:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  4m  
2s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 9 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
 8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  8m 
 7s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
11s{color} | {color:green} hbase-server: The patch generated 0 new + 360 
unchanged - 1 fixed = 360 total (was 361) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
31s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
21m 46s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
37s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}132m 18s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
37s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}187m 21s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService 
|
|   | hadoop.hbase.regionserver.TestMajorCompaction |
|   | hadoop.hbase.io.hfile.TestCacheOnWrite |
|   | hadoop.hbase.namespace.TestNamespaceAuditor |
|   | hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19841 |
| JIRA Patch URL | 

[jira] [Updated] (HBASE-19913) Split TestStochasticLoadBalancer2

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19913:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to master and branch-2.

Thanks [~stack] for reviewing.

> Split TestStochasticLoadBalancer2
> -
>
> Key: HBASE-19913
> URL: https://issues.apache.org/jira/browse/HBASE-19913
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19913.patch, HBASE-19913.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19915) From split/ merge procedures daughter/ merged regions get created in OFFLINE state

[Umesh Agashe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Umesh Agashe updated HBASE-19915:
-
Summary: From split/ merge procedures daughter/ merged regions get created 
in OFFLINE state  (was: From SplitTableRegionProcedure daughter regions get 
created in OFFLINE state)

> From split/ merge procedures daughter/ merged regions get created in OFFLINE 
> state
> --
>
> Key: HBASE-19915
> URL: https://issues.apache.org/jira/browse/HBASE-19915
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0-beta-1
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> See HBASE-19530. When regions are created initial state should be CLOSED. Bug 
> was discovered while debugging flaky test 
> TestSplitTableRegionProcedure#testRollbackAndDoubleExecution with numOfSteps 
> set to 4. After updating daughter regions in meta when master is restarted, 
> startup sequence of master assigns all OFFLINE regions. As daughter regions 
> are stored with OFFLINE state, daughter regions are assigned. This is 
> followed by re-assignment of daughter regions from resumed 
> SplitTableRegionProcedure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19913) Split TestStochasticLoadBalancer2

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349573#comment-16349573
 ] 

stack commented on HBASE-19913:
---

Ok.

+1

> Split TestStochasticLoadBalancer2
> -
>
> Key: HBASE-19913
> URL: https://issues.apache.org/jira/browse/HBASE-19913
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19913.patch, HBASE-19913.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19914) TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category annotation

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349570#comment-16349570
 ] 

Duo Zhang commented on HBASE-19914:
---

It is the new mvcc sensitive version behavior introduced in HBASE-15968.

And since there are only 4 methods in the new 
TestVisibilityLabelsOnNewVersionBehaviorTable  I think it is OK to declared it 
as MediumTests. Will check later why we only run 4 methods.

Thanks.

> TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category 
> annotation
> ---
>
> Key: HBASE-19914
> URL: https://issues.apache.org/jira/browse/HBASE-19914
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19914.patch, HBASE-19914.patch
>
>
> And both TestVisibilityLabelsOnNewVersionBehaviorTable and its parent class 
> run about 2 minutes, which is not safe to declared as MediumTests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19839) Flakey TestMergeTableRegionsProcedure & TestSplitTableRegionProcedure

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349567#comment-16349567
 ] 

Hudson commented on HBASE-19839:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4510 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4510/])
HBASE-19839 Fixed flakey tests (stack: rev 
57911d02c65b80eb198530b2a7d1fa39dba11a2c)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestMergeTableRegionsProcedure.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestSplitTableRegionProcedure.java


> Flakey TestMergeTableRegionsProcedure & TestSplitTableRegionProcedure
> -
>
> Key: HBASE-19839
> URL: https://issues.apache.org/jira/browse/HBASE-19839
> Project: HBase
>  Issue Type: Sub-task
>  Components: flakey, test
>Reporter: stack
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 0hbase-19839.master.004.patch, 
> 0hbase-19839.master.004.patch, 0hbase-19839.master.004.patch, 
> 0hbase-19839.master.004.patch, hbase-19839.master.001.patch, 
> hbase-19839.master.002.patch, hbase-19839.master.003.patch, 
> hbase-19839.master.003.patch, hbase-19839.master.003.patch, 
> hbase-19839.master.004.patch, hbase-19839.master.004.patch, 
> hbase-19839.master.004.patch, hbase-19839.master.004.patch, 
> hbase-19839.master.004.patch, hbase-19839.master.004.patch, 
> hbase-19839.master.004.patch
>
>
> Failing about 10% of the time: 
> [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/590/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure.txt]
> Its a good one. Probably an issue down deep.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19906) TestZooKeeper Timeout

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349566#comment-16349566
 ] 

Hudson commented on HBASE-19906:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4510 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4510/])
HBASE-19906 TestZooKeeper Timeout Includes move of TestQoSFunction from (stack: 
rev b21b8bfb91662bc65327f924504975997e0946da)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQosFunction.java


> TestZooKeeper Timeout
> -
>
> Key: HBASE-19906
> URL: https://issues.apache.org/jira/browse/HBASE-19906
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19906.branch-2.001.patch, 
> HBASE-19906.branch-2.002.patch, HBASE-19906.branch-2.003.patch, 
> HBASE-19906.branch-2.003.patch, HBASE-19906.branch-2.004.patch, 
> HBASE-19906.branch-2.005.patch, HBASE-19906.branch-2.005.patch
>
>
> TestZooKeeper is timing out causing hbase2 failures and breaking 
> HBASE-Flaky-Tests-branch2.0.0.
> ---
> Test set: org.apache.hadoop.hbase.TestZooKeeper
> ---
> Tests run: 6, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 600.8 s <<< 
> FAILURE! - in org.apache.hadoop.hbase.TestZooKeeper
> org.apache.hadoop.hbase.TestZooKeeper  Time elapsed: 551.041 s  <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 600 
> seconds
>   at org.apache.hadoop.hbase.TestZooKeeper.after(TestZooKeeper.java:103)
> org.apache.hadoop.hbase.TestZooKeeper  Time elapsed: 551.046 s  <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread 
> NIOServerCxn.Factory:0.0.0.0/0.0.0.0:59935
> Not always though.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19915) From SplitTableRegionProcedure daughter regions get created in OFFLINE state

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349572#comment-16349572
 ] 

Umesh Agashe commented on HBASE-19915:
--

{color:#33}In case of merge, merged region has state null which is 
interpreted as OFFLINE as noted in HBASE-19529.{color}

> From SplitTableRegionProcedure daughter regions get created in OFFLINE state
> 
>
> Key: HBASE-19915
> URL: https://issues.apache.org/jira/browse/HBASE-19915
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0-beta-1
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> See HBASE-19530. When regions are created initial state should be CLOSED. Bug 
> was discovered while debugging flaky test 
> TestSplitTableRegionProcedure#testRollbackAndDoubleExecution with numOfSteps 
> set to 4. After updating daughter regions in meta when master is restarted, 
> startup sequence of master assigns all OFFLINE regions. As daughter regions 
> are stored with OFFLINE state, daughter regions are assigned. This is 
> followed by re-assignment of daughter regions from resumed 
> SplitTableRegionProcedure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19912) The flag "writeToWAL" of Region#checkAndRowMutate is useless

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349568#comment-16349568
 ] 

Hudson commented on HBASE-19912:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4510 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4510/])
HBASE-19912 Remove useless 'writeToWAL' flag of Region#checkAndRowMutate 
(stack: rev 38c8144a065bc6d330330f611ec8beaa1477a884)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> The flag "writeToWAL" of Region#checkAndRowMutate is useless
> 
>
> Key: HBASE-19912
> URL: https://issues.apache.org/jira/browse/HBASE-19912
> Project: HBase
>  Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Mu Chun Wang
>Priority: Minor
>  Labels: beginner
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19912.branch-2.v0.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19884) BucketEntryGroup's equals, hashCode and compareTo methods are not consistent

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349569#comment-16349569
 ] 

Hudson commented on HBASE-19884:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4510 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4510/])
HBASE-19884 BucketEntryGroup's equals, hashCode and compareTo methods (stack: 
rev d47242247552bbad2df578c89501a6c6789c89c8)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java


> BucketEntryGroup's equals, hashCode and compareTo methods are not consistent
> 
>
> Key: HBASE-19884
> URL: https://issues.apache.org/jira/browse/HBASE-19884
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19884.master.001.patch, 
> HBASE-19884.master.001.patch, HBASE-19884.master.001.patch, 
> HBASE-19884.master.002.patch, HBASE-19884.master.003.patch
>
>
> BucketEntryGroup currently uses different fields to calculate compareTo, 
> equals and hasCode.
> In some cases !a.equals(b) but a.compareTo(b) == 0. Javadoc of Comparator 
> recommends that natural orderings be consistent with equals.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19913) Split TestStochasticLoadBalancer2

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349565#comment-16349565
 ] 

Duo Zhang commented on HBASE-19913:
---

{quote}
Ok making this Medium when it was Large ... 
TestStochasticLoadBalancerRegionReplica
{quote}
The work is done by me in HBASE-19867. When implementing this issue I found 
that it runs less than 1 minute so I think it is OK to declared it as 
MediumTests? The time consuming methods are moved to separated classes.

Thanks.

> Split TestStochasticLoadBalancer2
> -
>
> Key: HBASE-19913
> URL: https://issues.apache.org/jira/browse/HBASE-19913
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19913.patch, HBASE-19913.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19913) Split TestStochasticLoadBalancer2

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349564#comment-16349564
 ] 

stack commented on HBASE-19913:
---

HBASE-19916 for TestCacheOnWrite failure.

> Split TestStochasticLoadBalancer2
> -
>
> Key: HBASE-19913
> URL: https://issues.apache.org/jira/browse/HBASE-19913
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19913.patch, HBASE-19913.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19916) TestCacheOnWrite Times Out

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349561#comment-16349561
 ] 

stack commented on HBASE-19916:
---

.001 is what I pushed on master and branch-2. Sets test to be Large from Medium.

> TestCacheOnWrite Times Out
> --
>
> Key: HBASE-19916
> URL: https://issues.apache.org/jira/browse/HBASE-19916
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19916.master.001.patch
>
>
> All day it has been timing out. Its a medium test. There is a bit in the 
> middle where we are hung up for a minute or more:
> 2018-02-01 23:01:02,471 DEBUG [Time-limited test] 
> hfile.HFile$WriterFactory(336): Unable to set drop behind on 
> /testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e636faf5cf02d2d
> 2018-02-01 23:01:03,059 DEBUG [Time-limited test] 
> regionserver.HRegionFileSystem(463): Committing store file 
> /testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e63
> ...[truncated 1865657 bytes]...
> b663/myCF/61386855036d4facb75ce7eca2059661, entries=15000, sequenceid=1005, 
> filesize=85.3 K
> 2018-02-01 23:03:50,591 INFO  [Time-limited test] regionserver.HRegion(2713): 
> Finished memstore flush of ~1.73 MB/1814000, currentsize=0 B/0 for region 
> CompactionCacheOnWrite,,1517526229883.3b579f93f196a847ed1489e71585b663. in 
> 175ms, sequenceid=1005, compaction requested=false
> 2018-02-01 23:03:50,799 INFO  [Time-limited test] regionserver.HRegion(2517): 
> Flushing 1/1 column families, memstore=1.80 MB
> ...
> I've seen this a few times. The test takes 100seconds locally.
> Let me try changing it to type. If that doesn't work, will be back.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19916) TestCacheOnWrite Times Out

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19916:
--
Attachment: HBASE-19916.master.001.patch

> TestCacheOnWrite Times Out
> --
>
> Key: HBASE-19916
> URL: https://issues.apache.org/jira/browse/HBASE-19916
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19916.master.001.patch
>
>
> All day it has been timing out. Its a medium test. There is a bit in the 
> middle where we are hung up for a minute or more:
> 2018-02-01 23:01:02,471 DEBUG [Time-limited test] 
> hfile.HFile$WriterFactory(336): Unable to set drop behind on 
> /testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e636faf5cf02d2d
> 2018-02-01 23:01:03,059 DEBUG [Time-limited test] 
> regionserver.HRegionFileSystem(463): Committing store file 
> /testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e63
> ...[truncated 1865657 bytes]...
> b663/myCF/61386855036d4facb75ce7eca2059661, entries=15000, sequenceid=1005, 
> filesize=85.3 K
> 2018-02-01 23:03:50,591 INFO  [Time-limited test] regionserver.HRegion(2713): 
> Finished memstore flush of ~1.73 MB/1814000, currentsize=0 B/0 for region 
> CompactionCacheOnWrite,,1517526229883.3b579f93f196a847ed1489e71585b663. in 
> 175ms, sequenceid=1005, compaction requested=false
> 2018-02-01 23:03:50,799 INFO  [Time-limited test] regionserver.HRegion(2517): 
> Flushing 1/1 column families, memstore=1.80 MB
> ...
> I've seen this a few times. The test takes 100seconds locally.
> Let me try changing it to type. If that doesn't work, will be back.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349557#comment-16349557
 ] 

Hadoop QA commented on HBASE-19658:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  7m 
21s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
41s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
18m 26s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}119m 
55s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 36s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19658 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908871/HBASE-19658.09.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux a4f988e8766b 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / d472422475 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11342/testReport/ |
| Max. process+thread count | 5138 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11342/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Fix and reenable 
> TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
> 

[jira] [Assigned] (HBASE-19916) TestCacheOnWrite Times Out

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack reassigned HBASE-19916:
-

 Assignee: stack
Fix Version/s: 2.0.0-beta-2

> TestCacheOnWrite Times Out
> --
>
> Key: HBASE-19916
> URL: https://issues.apache.org/jira/browse/HBASE-19916
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> All day it has been timing out. Its a medium test. There is a bit in the 
> middle where we are hung up for a minute or more:
> 2018-02-01 23:01:02,471 DEBUG [Time-limited test] 
> hfile.HFile$WriterFactory(336): Unable to set drop behind on 
> /testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e636faf5cf02d2d
> 2018-02-01 23:01:03,059 DEBUG [Time-limited test] 
> regionserver.HRegionFileSystem(463): Committing store file 
> /testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e63
> ...[truncated 1865657 bytes]...
> b663/myCF/61386855036d4facb75ce7eca2059661, entries=15000, sequenceid=1005, 
> filesize=85.3 K
> 2018-02-01 23:03:50,591 INFO  [Time-limited test] regionserver.HRegion(2713): 
> Finished memstore flush of ~1.73 MB/1814000, currentsize=0 B/0 for region 
> CompactionCacheOnWrite,,1517526229883.3b579f93f196a847ed1489e71585b663. in 
> 175ms, sequenceid=1005, compaction requested=false
> 2018-02-01 23:03:50,799 INFO  [Time-limited test] regionserver.HRegion(2517): 
> Flushing 1/1 column families, memstore=1.80 MB
> ...
> I've seen this a few times. The test takes 100seconds locally.
> Let me try changing it to type. If that doesn't work, will be back.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-19916) TestCacheOnWrite Times Out

[stack (JIRA)]

stack created HBASE-19916:
-

 Summary: TestCacheOnWrite Times Out
 Key: HBASE-19916
 URL: https://issues.apache.org/jira/browse/HBASE-19916
 Project: HBase
  Issue Type: Sub-task
Reporter: stack


All day it has been timing out. Its a medium test. There is a bit in the middle 
where we are hung up for a minute or more:


2018-02-01 23:01:02,471 DEBUG [Time-limited test] 
hfile.HFile$WriterFactory(336): Unable to set drop behind on 
/testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e636faf5cf02d2d
2018-02-01 23:01:03,059 DEBUG [Time-limited test] 
regionserver.HRegionFileSystem(463): Committing store file 
/testptch/hbase/hbase-server/target/test-data/6a153924-7f81-4008-ac7e-d0e69384655e/data/default/CompactionCacheOnWrite/6dd6ed35f3b6090bd8d04ed21d687424/.tmp/myCF/c0387b09f82840ab9e63
...[truncated 1865657 bytes]...
b663/myCF/61386855036d4facb75ce7eca2059661, entries=15000, sequenceid=1005, 
filesize=85.3 K
2018-02-01 23:03:50,591 INFO  [Time-limited test] regionserver.HRegion(2713): 
Finished memstore flush of ~1.73 MB/1814000, currentsize=0 B/0 for region 
CompactionCacheOnWrite,,1517526229883.3b579f93f196a847ed1489e71585b663. in 
175ms, sequenceid=1005, compaction requested=false
2018-02-01 23:03:50,799 INFO  [Time-limited test] regionserver.HRegion(2517): 
Flushing 1/1 column families, memstore=1.80 MB

...

I've seen this a few times. The test takes 100seconds locally.

Let me try changing it to type. If that doesn't work, will be back.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19915) From SplitTableRegionProcedure daughter regions get created in OFFLINE state

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349547#comment-16349547
 ] 

Umesh Agashe commented on HBASE-19915:
--

good question [~stack]! FWICS, merge seems to have same problem. Let me test 
and patch merge too. Thanks!

> From SplitTableRegionProcedure daughter regions get created in OFFLINE state
> 
>
> Key: HBASE-19915
> URL: https://issues.apache.org/jira/browse/HBASE-19915
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0-beta-1
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> See HBASE-19530. When regions are created initial state should be CLOSED. Bug 
> was discovered while debugging flaky test 
> TestSplitTableRegionProcedure#testRollbackAndDoubleExecution with numOfSteps 
> set to 4. After updating daughter regions in meta when master is restarted, 
> startup sequence of master assigns all OFFLINE regions. As daughter regions 
> are stored with OFFLINE state, daughter regions are assigned. This is 
> followed by re-assignment of daughter regions from resumed 
> SplitTableRegionProcedure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19895) Add keepDeletedCells option in ScanOptions for customize scanInfo in pre-hooks

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349538#comment-16349538
 ] 

Ted Yu commented on HBASE-19895:


Ankit:
Please take a look at the checkstyle warnings w.r.t. TestMajorCompaction.java

> Add keepDeletedCells option in ScanOptions for customize scanInfo in pre-hooks
> --
>
> Key: HBASE-19895
> URL: https://issues.apache.org/jira/browse/HBASE-19895
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19895.patch, HBASE-19895_v1.patch, 
> HBASE-19895_v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19913) Split TestStochasticLoadBalancer2

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349536#comment-16349536
 ] 

Hadoop QA commented on HBASE-19913:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 7 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
 4s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
54s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
19m 19s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 48s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
23s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}157m 43s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.hfile.TestCacheOnWrite |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19913 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908867/HBASE-19913.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 5b2bfe67932d 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 38c8144a06 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11339/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11339/testReport/ |
| Max. process+thread count | 5276 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11339/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was 

[jira] [Comment Edited] (HBASE-19529) Handle null states in AM

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349534#comment-16349534
 ] 

stack edited comment on HBASE-19529 at 2/1/18 11:56 PM:


Umesh found that when we create daughter regions, we were adding them with null 
which became OFFLINE. Linked HBASE-19915


was (Author: stack):
Umesh found that when we create daughter regions, we were adding them with null 
which became OFFLINE.

> Handle null states in AM
> 
>
> Key: HBASE-19529
> URL: https://issues.apache.org/jira/browse/HBASE-19529
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> From debugging in HBASE-19457, found some questions that need concrete 
> answers:
> 1) What does a region state of null in meta means? Currently AM treats it as 
> OFFLINE
> 2) What does a table state of null in meta means? Currently TSM treats it as 
> ENABLED.
> More importantly, we need to fix holes in AM so that our state machine is 
> well defined and doesn't end up in these uncertainties.
> Figuring out answers to above questions will help in that direction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19529) Handle null states in AM

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349534#comment-16349534
 ] 

stack commented on HBASE-19529:
---

Umesh found that when we create daughter regions, we were adding them with null 
which became OFFLINE.

> Handle null states in AM
> 
>
> Key: HBASE-19529
> URL: https://issues.apache.org/jira/browse/HBASE-19529
> Project: HBase
>  Issue Type: Bug
>Reporter: Appy
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> From debugging in HBASE-19457, found some questions that need concrete 
> answers:
> 1) What does a region state of null in meta means? Currently AM treats it as 
> OFFLINE
> 2) What does a table state of null in meta means? Currently TSM treats it as 
> ENABLED.
> More importantly, we need to fix holes in AM so that our state machine is 
> well defined and doesn't end up in these uncertainties.
> Figuring out answers to above questions will help in that direction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19849) MetaEntry in HBaseFsck has inconsistent equals and hashCode

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19849:
--
Fix Version/s: (was: 2.0.0-beta-2)
   2.0.0

> MetaEntry in HBaseFsck has inconsistent equals and hashCode
> ---
>
> Key: HBASE-19849
> URL: https://issues.apache.org/jira/browse/HBASE-19849
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Peter Somogyi
>Priority: Major
> Fix For: 2.0.0
>
>
> [~balazs.meszaros] noticed that MetaEntry uses different fields in equals and 
> hashCode. The used fields should be unified.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19849) MetaEntry in HBaseFsck has inconsistent equals and hashCode

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349527#comment-16349527
 ] 

stack commented on HBASE-19849:
---

Moved out of beta-2. There is no hbck for hbase2 (smile).

> MetaEntry in HBaseFsck has inconsistent equals and hashCode
> ---
>
> Key: HBASE-19849
> URL: https://issues.apache.org/jira/browse/HBASE-19849
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Peter Somogyi
>Priority: Major
> Fix For: 2.0.0
>
>
> [~balazs.meszaros] noticed that MetaEntry uses different fields in equals and 
> hashCode. The used fields should be unified.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19915) From SplitTableRegionProcedure daughter regions get created in OFFLINE state

[Umesh Agashe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Umesh Agashe updated HBASE-19915:
-
Description: See HBASE-19530. When regions are created initial state should 
be CLOSED. Bug was discovered while debugging flaky test 
TestSplitTableRegionProcedure#testRollbackAndDoubleExecution with numOfSteps 
set to 4. After updating daughter regions in meta when master is restarted, 
startup sequence of master assigns all OFFLINE regions. As daughter regions are 
stored with OFFLINE state, daughter regions are assigned. This is followed by 
re-assignment of daughter regions from resumed SplitTableRegionProcedure.  
(was: See HBASE-19530. When regions are created initial state should be CLOSED. 
Bug was discovered while debugging flaky test 
TestSplitTableRegionProcedure#testRollbackAndDoubleExecution numOfSteps set to 
4. After updating daughter regions in meta when master is restarted, startup 
sequence of master assigns all OFFLINE regions. As daughter regions are stored 
with OFFLINE state, daughter regions are assigned. This is followed by 
re-assignment of daughter regions from resumed SplitTableRegionProcedure.)

> From SplitTableRegionProcedure daughter regions get created in OFFLINE state
> 
>
> Key: HBASE-19915
> URL: https://issues.apache.org/jira/browse/HBASE-19915
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0-beta-1
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> See HBASE-19530. When regions are created initial state should be CLOSED. Bug 
> was discovered while debugging flaky test 
> TestSplitTableRegionProcedure#testRollbackAndDoubleExecution with numOfSteps 
> set to 4. After updating daughter regions in meta when master is restarted, 
> startup sequence of master assigns all OFFLINE regions. As daughter regions 
> are stored with OFFLINE state, daughter regions are assigned. This is 
> followed by re-assignment of daughter regions from resumed 
> SplitTableRegionProcedure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19914) TestVisibilityLabelsOnNewVersionBehaviorTable does not have Category annotation

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349524#comment-16349524
 ] 

Hadoop QA commented on HBASE-19914:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
20s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
5s{color} | {color:red} hbase-server: The patch generated 36 new + 1 unchanged 
- 13 fixed = 37 total (was 14) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
47s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
19m 18s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
59s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}114m  
3s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
57s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}159m  1s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19914 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908866/HBASE-19914.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 4f92bba20f62 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 38c8144a06 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle | 

[jira] [Commented] (HBASE-19904) Find a better way to deal with the cyclic dependency when initializing WAL and Replication

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349523#comment-16349523
 ] 

Hadoop QA commented on HBASE-19904:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 35 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
20s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 1s{color} | {color:green} hbase-server: The patch generated 0 new + 1131 
unchanged - 17 fixed = 1131 total (was 1148) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 5s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
16m 18s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}100m 
14s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m  
1s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
41s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}147m 41s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19904 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908868/HBASE-19904-v4.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 817538f28283 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 38c8144a06 |
| maven | version: Apache 

[jira] [Commented] (HBASE-19915) From SplitTableRegionProcedure daughter regions get created in OFFLINE state

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349518#comment-16349518
 ] 

stack commented on HBASE-19915:
---

Good one [~uagashe]...  What about Merge? Do we create the Merge target in 
CLOSED state?

> From SplitTableRegionProcedure daughter regions get created in OFFLINE state
> 
>
> Key: HBASE-19915
> URL: https://issues.apache.org/jira/browse/HBASE-19915
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0-beta-1
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0-beta-2
>
>
> See HBASE-19530. When regions are created initial state should be CLOSED. Bug 
> was discovered while debugging flaky test 
> TestSplitTableRegionProcedure#testRollbackAndDoubleExecution numOfSteps set 
> to 4. After updating daughter regions in meta when master is restarted, 
> startup sequence of master assigns all OFFLINE regions. As daughter regions 
> are stored with OFFLINE state, daughter regions are assigned. This is 
> followed by re-assignment of daughter regions from resumed 
> SplitTableRegionProcedure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19528) Major Compaction Tool

[churro morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

churro morales updated HBASE-19528:
---
Status: Open  (was: Patch Available)

> Major Compaction Tool 
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/jira/browse/HBASE-19528
> Project: HBase
>  Issue Type: New Feature
>Reporter: churro morales
>Assignee: churro morales
>Priority: Major
> Fix For: 3.0.0, 2.0.0-beta-2
>
> Attachments: 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, 
> HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, 
> HBASE-19528.v1.patch, HBASE-19528.v2.branch-1.patch, 
> HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch
>
>
> The basic overview of how this tool works is:
> Parameters:
> Table
> Stores
> ClusterConcurrency
> Timestamp
> So you input a table, desired concurrency and the list of stores you wish to 
> major compact.  The tool first checks the filesystem to see which stores need 
> compaction based on the timestamp you provide (default is current time).  It 
> takes that list of stores that require compaction and executes those requests 
> concurrently with at most N distinct RegionServers compacting at a given 
> time.  Each thread waits for the compaction to complete before moving to the 
> next queue.  If a region split, merge or move happens this tool ensures those 
> regions get major compacted as well. 
> This helps us in two ways, we can limit how much I/O bandwidth we are using 
> for major compaction cluster wide and we are guaranteed after the tool 
> completes that all requested compactions complete regardless of moves, merges 
> and splits. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19528) Major Compaction Tool

[churro morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

churro morales updated HBASE-19528:
---
Status: Patch Available  (was: Open)

> Major Compaction Tool 
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/jira/browse/HBASE-19528
> Project: HBase
>  Issue Type: New Feature
>Reporter: churro morales
>Assignee: churro morales
>Priority: Major
> Fix For: 3.0.0, 2.0.0-beta-2
>
> Attachments: 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, 
> HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, 
> HBASE-19528.v1.patch, HBASE-19528.v2.branch-1.patch, 
> HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch
>
>
> The basic overview of how this tool works is:
> Parameters:
> Table
> Stores
> ClusterConcurrency
> Timestamp
> So you input a table, desired concurrency and the list of stores you wish to 
> major compact.  The tool first checks the filesystem to see which stores need 
> compaction based on the timestamp you provide (default is current time).  It 
> takes that list of stores that require compaction and executes those requests 
> concurrently with at most N distinct RegionServers compacting at a given 
> time.  Each thread waits for the compaction to complete before moving to the 
> next queue.  If a region split, merge or move happens this tool ensures those 
> regions get major compacted as well. 
> This helps us in two ways, we can limit how much I/O bandwidth we are using 
> for major compaction cluster wide and we are guaranteed after the tool 
> completes that all requested compactions complete regardless of moves, merges 
> and splits. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-19915) From SplitTableRegionProcedure daughter regions get created in OFFLINE state

[Umesh Agashe (JIRA)]

Umesh Agashe created HBASE-19915:


 Summary: From SplitTableRegionProcedure daughter regions get 
created in OFFLINE state
 Key: HBASE-19915
 URL: https://issues.apache.org/jira/browse/HBASE-19915
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.0.0-beta-1
Reporter: Umesh Agashe
Assignee: Umesh Agashe
 Fix For: 2.0.0-beta-2


See HBASE-19530. When regions are created initial state should be CLOSED. Bug 
was discovered while debugging flaky test 
TestSplitTableRegionProcedure#testRollbackAndDoubleExecution numOfSteps set to 
4. After updating daughter regions in meta when master is restarted, startup 
sequence of master assigns all OFFLINE regions. As daughter regions are stored 
with OFFLINE state, daughter regions are assigned. This is followed by 
re-assignment of daughter regions from resumed SplitTableRegionProcedure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19895) Add keepDeletedCells option in ScanOptions for customize scanInfo in pre-hooks

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349499#comment-16349499
 ] 

Hadoop QA commented on HBASE-19895:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
52s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
53s{color} | {color:red} hbase-server: The patch generated 12 new + 18 
unchanged - 2 fixed = 30 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 1s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
16m  3s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}100m 
30s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 46s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19895 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908869/HBASE-19895_v1.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 5bd5628b818c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 38c8144a06 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11341/artifact/patchprocess/diff-checkstyle-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11341/testReport/ |
| Max. process+thread count | 5753 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11341/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was 

[jira] [Commented] (HBASE-19720) Rename WALKey#getTabnename to WALKey#getTableName

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349462#comment-16349462
 ] 

stack commented on HBASE-19720:
---

+1

> Rename WALKey#getTabnename to WALKey#getTableName
> -
>
> Key: HBASE-19720
> URL: https://issues.apache.org/jira/browse/HBASE-19720
> Project: HBase
>  Issue Type: Task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-19720.v0.patch
>
>
> WALKey is denoted as LP so its naming should obey the common rule in our 
> codebase.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19901) Up yetus proclimit on nightlies

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349362#comment-16349362
 ] 

Hadoop QA commented on HBASE-19901:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  3s{color} 
| {color:red} HBASE-19901 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19901 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908875/HBASE-19901.master.002.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11347/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349361#comment-16349361
 ] 

stack commented on HBASE-19841:
---

bq. I found them based on which tests were failing!

Excellent.

+1 on commit though I suppose no harm waiting on build to come in...

> Tests against hadoop3 fail with StreamLacksCapabilityException
> --
>
> Key: HBASE-19841
> URL: https://issues.apache.org/jira/browse/HBASE-19841
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19841.007.patch, 19841.06.patch, 19841.v0.txt, 
> 19841.v1.txt, HBASE-19841.v10.patch, HBASE-19841.v11.patch, 
> HBASE-19841.v11.patch, HBASE-19841.v2.patch, HBASE-19841.v3.patch, 
> HBASE-19841.v4.patch, HBASE-19841.v5.patch, HBASE-19841.v7.patch, 
> HBASE-19841.v8.patch, HBASE-19841.v8.patch, HBASE-19841.v8.patch, 
> HBASE-19841.v9.patch
>
>
> The following can be observed running against hadoop3:
> {code}
> java.io.IOException: cannot get log writer
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> {code}
> This was due to hbase-server/src/test/resources/hbase-site.xml not being 
> picked up by Configuration object. Among the configs from this file, the 
> value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence 
> of hflush and hsync. Without this config entry,  
> StreamLacksCapabilityException is thrown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19901) Up yetus proclimit on nightlies

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349356#comment-16349356
 ] 

stack commented on HBASE-19901:
---

.002 is an addendum applied to branch-2 and master.

It ups proclimit to 10k for now until we figure out why so many procs during 
our test runs... and it adds a docker containter 20g limit which ups default of 
4g from yetus.

> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19901) Up yetus proclimit on nightlies

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19901:
--
Attachment: HBASE-19901.master.002.patch

> Up yetus proclimit on nightlies
> ---
>
> Key: HBASE-19901
> URL: https://issues.apache.org/jira/browse/HBASE-19901
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19901.master.001.patch, 
> HBASE-19901.master.002.patch
>
>
> We're on 0.7.0 now which enforces limits meant to protect against runaway 
> processes. Default is 1000 procs. HBase test runs seem to consume almost 4k. 
> Up our proclimit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19093) Check Admin/Table to ensure all operations go via AccessControl

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349348#comment-16349348
 ] 

stack commented on HBASE-19093:
---

This is stalled again [~balazs.meszaros] ? Thanks.

> Check Admin/Table to ensure all operations go via AccessControl
> ---
>
> Key: HBASE-19093
> URL: https://issues.apache.org/jira/browse/HBASE-19093
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19093.master.001.patch, 
> HBASE-19093.master.002.patch, RegionObserver.txt
>
>
> A cursory review of Admin Interface has a bunch of methods as open, with out 
> AccessControl checks. For example, procedure executor has not check on it.
> This issue is about given the Admin and Table Interfaces a once-over to see 
> what is missing and to fill in access control where missing.
> This is a follow-on from work over in HBASE-19048



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349332#comment-16349332
 ] 

Hadoop QA commented on HBASE-19876:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  4s{color} 
| {color:red} HBASE-19876 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908874/HBASE-19876.v3.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11346/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> The exception happening in converting pb mutation to hbase.mutation messes up 
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch, 
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch, 
> HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO  [hconnection-0x3291b443-shared-pool11-t6] 
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3, 
> attempt=6/16 failed=20ops, last 
> exception=org.apache.hadoop.hbase.client.WrongRowIOException: 
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't 
> match the original one aaa
>   at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
>   at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or 
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language: 
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada, 
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, 
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, 
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml  for 
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
>   mutation = ProtobufUtil.toPut(m, cells);
>   batchContainsPuts = true;
> } else {
>   mutation = ProtobufUtil.toDelete(m, cells);
>   batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
>   }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the 
> failed cells.
> {code:java}
> } catch (IOException ie) {
>   if (atomic) {
> throw ie;
>   }
>   for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie, 
> mutation.getIndex()));
>   }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they 
> refer to invalid cells.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349333#comment-16349333
 ] 

Hadoop QA commented on HBASE-18294:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  4s{color} 
| {color:red} HBASE-18294 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18294 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908873/HBASE-18294.01.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11345/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, 
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, 
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, 
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, 
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, 
> HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, 
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19528) Major Compaction Tool

[churro morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

churro morales updated HBASE-19528:
---
Status: Open  (was: Patch Available)

> Major Compaction Tool 
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/jira/browse/HBASE-19528
> Project: HBase
>  Issue Type: New Feature
>Reporter: churro morales
>Assignee: churro morales
>Priority: Major
> Fix For: 3.0.0, 2.0.0-beta-2
>
> Attachments: 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, 
> HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, 
> HBASE-19528.v1.patch, HBASE-19528.v2.branch-1.patch, 
> HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch
>
>
> The basic overview of how this tool works is:
> Parameters:
> Table
> Stores
> ClusterConcurrency
> Timestamp
> So you input a table, desired concurrency and the list of stores you wish to 
> major compact.  The tool first checks the filesystem to see which stores need 
> compaction based on the timestamp you provide (default is current time).  It 
> takes that list of stores that require compaction and executes those requests 
> concurrently with at most N distinct RegionServers compacting at a given 
> time.  Each thread waits for the compaction to complete before moving to the 
> next queue.  If a region split, merge or move happens this tool ensures those 
> regions get major compacted as well. 
> This helps us in two ways, we can limit how much I/O bandwidth we are using 
> for major compaction cluster wide and we are guaranteed after the tool 
> completes that all requested compactions complete regardless of moves, merges 
> and splits. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19528) Major Compaction Tool

[churro morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

churro morales updated HBASE-19528:
---
Status: Patch Available  (was: Open)

> Major Compaction Tool 
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/jira/browse/HBASE-19528
> Project: HBase
>  Issue Type: New Feature
>Reporter: churro morales
>Assignee: churro morales
>Priority: Major
> Fix For: 3.0.0, 2.0.0-beta-2
>
> Attachments: 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, 
> HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, 
> HBASE-19528.v1.patch, HBASE-19528.v2.branch-1.patch, 
> HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch
>
>
> The basic overview of how this tool works is:
> Parameters:
> Table
> Stores
> ClusterConcurrency
> Timestamp
> So you input a table, desired concurrency and the list of stores you wish to 
> major compact.  The tool first checks the filesystem to see which stores need 
> compaction based on the timestamp you provide (default is current time).  It 
> takes that list of stores that require compaction and executes those requests 
> concurrently with at most N distinct RegionServers compacting at a given 
> time.  Each thread waits for the compaction to complete before moving to the 
> next queue.  If a region split, merge or move happens this tool ensures those 
> regions get major compacted as well. 
> This helps us in two ways, we can limit how much I/O bandwidth we are using 
> for major compaction cluster wide and we are guaranteed after the tool 
> completes that all requested compactions complete regardless of moves, merges 
> and splits. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19876:
--
Attachment: HBASE-19876.v3.patch

> The exception happening in converting pb mutation to hbase.mutation messes up 
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.v0.patch, HBASE-19876.v1.patch, 
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch, 
> HBASE-19876.v3.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO  [hconnection-0x3291b443-shared-pool11-t6] 
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3, 
> attempt=6/16 failed=20ops, last 
> exception=org.apache.hadoop.hbase.client.WrongRowIOException: 
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't 
> match the original one aaa
>   at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
>   at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or 
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language: 
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada, 
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, 
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, 
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml  for 
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
>   mutation = ProtobufUtil.toPut(m, cells);
>   batchContainsPuts = true;
> } else {
>   mutation = ProtobufUtil.toDelete(m, cells);
>   batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
>   }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the 
> failed cells.
> {code:java}
> } catch (IOException ie) {
>   if (atomic) {
> throw ie;
>   }
>   for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie, 
> mutation.getIndex()));
>   }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they 
> refer to invalid cells.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349327#comment-16349327
 ] 

stack commented on HBASE-18294:
---

[~eshcar] Looks like there is a blocking item up on rbfrom our Anoop?

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, 
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, 
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, 
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, 
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, 
> HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, 
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18294:
--
Attachment: HBASE-18294.01.patch

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, 
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, 
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, 
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, 
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, 
> HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, 
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19528) Major Compaction Tool

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349322#comment-16349322
 ] 

stack commented on HBASE-19528:
---

Failed to launch Docker is an infra issue.. See 
https://issues.apache.org/jira/browse/HBASE-19902?focusedCommentId=16349025=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16349025

Retry...

> Major Compaction Tool 
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/jira/browse/HBASE-19528
> Project: HBase
>  Issue Type: New Feature
>Reporter: churro morales
>Assignee: churro morales
>Priority: Major
> Fix For: 3.0.0, 2.0.0-beta-2
>
> Attachments: 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, 
> HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, 
> HBASE-19528.v1.patch, HBASE-19528.v2.branch-1.patch, 
> HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch
>
>
> The basic overview of how this tool works is:
> Parameters:
> Table
> Stores
> ClusterConcurrency
> Timestamp
> So you input a table, desired concurrency and the list of stores you wish to 
> major compact.  The tool first checks the filesystem to see which stores need 
> compaction based on the timestamp you provide (default is current time).  It 
> takes that list of stores that require compaction and executes those requests 
> concurrently with at most N distinct RegionServers compacting at a given 
> time.  Each thread waits for the compaction to complete before moving to the 
> next queue.  If a region split, merge or move happens this tool ensures those 
> regions get major compacted as well. 
> This helps us in two ways, we can limit how much I/O bandwidth we are using 
> for major compaction cluster wide and we are guaranteed after the tool 
> completes that all requested compactions complete regardless of moves, merges 
> and splits. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349319#comment-16349319
 ] 

Mike Drob commented on HBASE-19841:
---

I found them based on which tests were failing!

> Tests against hadoop3 fail with StreamLacksCapabilityException
> --
>
> Key: HBASE-19841
> URL: https://issues.apache.org/jira/browse/HBASE-19841
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19841.007.patch, 19841.06.patch, 19841.v0.txt, 
> 19841.v1.txt, HBASE-19841.v10.patch, HBASE-19841.v11.patch, 
> HBASE-19841.v11.patch, HBASE-19841.v2.patch, HBASE-19841.v3.patch, 
> HBASE-19841.v4.patch, HBASE-19841.v5.patch, HBASE-19841.v7.patch, 
> HBASE-19841.v8.patch, HBASE-19841.v8.patch, HBASE-19841.v8.patch, 
> HBASE-19841.v9.patch
>
>
> The following can be observed running against hadoop3:
> {code}
> java.io.IOException: cannot get log writer
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
>   at 
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> {code}
> This was due to hbase-server/src/test/resources/hbase-site.xml not being 
> picked up by Configuration object. Among the configs from this file, the 
> value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence 
> of hflush and hsync. Without this config entry,  
> StreamLacksCapabilityException is thrown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19528) Major Compaction Tool

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349320#comment-16349320
 ] 

Hadoop QA commented on HBASE-19528:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red}  0m  
4s{color} | {color:red} Docker failed to build yetus/hbase:36a7029. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19528 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12908872/HBASE-19528.v2.branch-1.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11344/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Major Compaction Tool 
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/jira/browse/HBASE-19528
> Project: HBase
>  Issue Type: New Feature
>Reporter: churro morales
>Assignee: churro morales
>Priority: Major
> Fix For: 3.0.0, 2.0.0-beta-2
>
> Attachments: 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, 
> HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, 
> HBASE-19528.v1.patch, HBASE-19528.v2.branch-1.patch, 
> HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch
>
>
> The basic overview of how this tool works is:
> Parameters:
> Table
> Stores
> ClusterConcurrency
> Timestamp
> So you input a table, desired concurrency and the list of stores you wish to 
> major compact.  The tool first checks the filesystem to see which stores need 
> compaction based on the timestamp you provide (default is current time).  It 
> takes that list of stores that require compaction and executes those requests 
> concurrently with at most N distinct RegionServers compacting at a given 
> time.  Each thread waits for the compaction to complete before moving to the 
> next queue.  If a region split, merge or move happens this tool ensures those 
> regions get major compacted as well. 
> This helps us in two ways, we can limit how much I/O bandwidth we are using 
> for major compaction cluster wide and we are guaranteed after the tool 
> completes that all requested compactions complete regardless of moves, merges 
> and splits. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19897) RowMutations should follow the fluent pattern

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349318#comment-16349318
 ] 

Chia-Ping Tsai commented on HBASE-19897:


{quote}Until the old method is removed I suppose?
{quote}
yep.

Push it to branch-2 and master. Thanks [~stack] for reviewing.

> RowMutations should follow the fluent pattern
> -
>
> Key: HBASE-19897
> URL: https://issues.apache.org/jira/browse/HBASE-19897
> Project: HBase
>  Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19897.v0.patch, HBASE-19897.v1.patch, 
> HBASE-19897.v1.patch
>
>
> Other row ops, including {{Put}}, {{Delete}}, {{Get}}, {{Scan}}, do have the 
> fluent interface. Also, Changing the return type from {{Void}} to 
> {{RowMutations}} won't break the API BC (unless someone has interest in 
> {{Void}} object...)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19897) RowMutations should follow the fluent pattern

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-19897:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

> RowMutations should follow the fluent pattern
> -
>
> Key: HBASE-19897
> URL: https://issues.apache.org/jira/browse/HBASE-19897
> Project: HBase
>  Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19897.v0.patch, HBASE-19897.v1.patch, 
> HBASE-19897.v1.patch
>
>
> Other row ops, including {{Put}}, {{Delete}}, {{Get}}, {{Scan}}, do have the 
> fluent interface. Also, Changing the return type from {{Void}} to 
> {{RowMutations}} won't break the API BC (unless someone has interest in 
> {{Void}} object...)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19528) Major Compaction Tool

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19528:
--
Attachment: HBASE-19528.v2.branch-1.patch

> Major Compaction Tool 
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/jira/browse/HBASE-19528
> Project: HBase
>  Issue Type: New Feature
>Reporter: churro morales
>Assignee: churro morales
>Priority: Major
> Fix For: 3.0.0, 2.0.0-beta-2
>
> Attachments: 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, 
> HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, 
> HBASE-19528.v1.patch, HBASE-19528.v2.branch-1.patch, 
> HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch
>
>
> The basic overview of how this tool works is:
> Parameters:
> Table
> Stores
> ClusterConcurrency
> Timestamp
> So you input a table, desired concurrency and the list of stores you wish to 
> major compact.  The tool first checks the filesystem to see which stores need 
> compaction based on the timestamp you provide (default is current time).  It 
> takes that list of stores that require compaction and executes those requests 
> concurrently with at most N distinct RegionServers compacting at a given 
> time.  Each thread waits for the compaction to complete before moving to the 
> next queue.  If a region split, merge or move happens this tool ensures those 
> regions get major compacted as well. 
> This helps us in two ways, we can limit how much I/O bandwidth we are using 
> for major compaction cluster wide and we are guaranteed after the tool 
> completes that all requested compactions complete regardless of moves, merges 
> and splits. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19897) RowMutations should follow the fluent pattern

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349313#comment-16349313
 ] 

stack commented on HBASE-19897:
---

Thanks [~chia7712] for explaination.

bq. ... If users want to try the fluent way, they must to write 
rm.add((Mutation) new Put).

Until the old method is removed I suppose?

Thanks.

> RowMutations should follow the fluent pattern
> -
>
> Key: HBASE-19897
> URL: https://issues.apache.org/jira/browse/HBASE-19897
> Project: HBase
>  Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19897.v0.patch, HBASE-19897.v1.patch, 
> HBASE-19897.v1.patch
>
>
> Other row ops, including {{Put}}, {{Delete}}, {{Get}}, {{Scan}}, do have the 
> fluent interface. Also, Changing the return type from {{Void}} to 
> {{RowMutations}} won't break the API BC (unless someone has interest in 
> {{Void}} object...)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)