[jira] [Updated] (HBASE-18972) Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-18972:
--
Attachment: HBASE-18972-v3.patch

Fix typo. Will commit shortly. Thanks [~appy] for reviewing.

> Use Builder pattern to remove nullable parameters for coprocessor methods in 
> RawAsyncTable interface
> 
>
> Key: HBASE-18972
> URL: https://issues.apache.org/jira/browse/HBASE-18972
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18972-v1.patch, HBASE-18972-v2.patch, 
> HBASE-18972-v2.patch, HBASE-18972-v3.patch, HBASE-18972.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19131) Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235223#comment-16235223
 ] 

stack commented on HBASE-19131:
---

Is it ok changing the return from Collection to List? ClusterStatus is public. 
We are changing the signature.

 public List getDeadServerNames() {

Is listDeadServers an API that was in branch-1?

Skimmed patch. Looks great. Thanks.



> Add the ClusterStatus hook and cleanup other hooks which can be replaced by 
> ClusterStatus hook
> --
>
> Key: HBASE-19131
> URL: https://issues.apache.org/jira/browse/HBASE-19131
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19131.v0.patch, HBASE-19131.v1.patch
>
>
> This issue will add the hook for {{ClusterStatus}} and cleanup other hooks 
> which can be replaced by {{ClusterStatus}} hook.
> # {{ListDeadServers}} -  introduced by HBASE-18131



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19141) [compat 1-2] getClusterStatus always return empty ClusterStatus

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235214#comment-16235214
 ] 

stack commented on HBASE-19141:
---

+1 on patch. What does this mean for hbase1 clients [~chia7712]? Did you see 
the [~reidchan] review notes above? Could reopen to apply sir? Good stuff.

>  [compat 1-2] getClusterStatus always return empty ClusterStatus
> 
>
> Key: HBASE-19141
> URL: https://issues.apache.org/jira/browse/HBASE-19141
> Project: HBase
>  Issue Type: Sub-task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19141.v0.patch, HBASE-19141.v1.patch
>
>
> We are able to limit the scope to get part of {{ClusterStatus}} in 2.0. 
> However the request sent by 1.x client has no specific scope info to retrieve 
> any information from {{ClusterStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19075) Task tabs on master UI cause page scroll

[Sahil Aggarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235204#comment-16235204
 ] 

Sahil Aggarwal commented on HBASE-19075:


Hi Mike,

Other tables uses href="#div" which just changes the div on tab click but this 
table changes the query param and rely on page refresh to show the requested 
tab. Shall we change its impl to work in similar way as others or we just add 
the scipt to scroll to the selected anchor?

> Task tabs on master UI cause page scroll
> 
>
> Key: HBASE-19075
> URL: https://issues.apache.org/jira/browse/HBASE-19075
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: Mike Drob
>Priority: Major
>  Labels: beginner
> Fix For: 2.0.0
>
>
> On the master info page, the clicking the tabs under Tasks causes the page to 
> scroll back to the top of the page.
> {noformat}
> Tasks
> Show All Monitored Tasks Show non-RPC Tasks Show All RPC Handler Tasks Show 
> Active RPC Calls Show Client Operations View as JSON
> {noformat}
> ^^ Any of those
> The other tab-like links on the page keep the scroll in the same location.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18972) Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235206#comment-16235206
 ] 

Appy commented on HBASE-18972:
--

+1 and small fix-on-commit on RB.

> Use Builder pattern to remove nullable parameters for coprocessor methods in 
> RawAsyncTable interface
> 
>
> Key: HBASE-18972
> URL: https://issues.apache.org/jira/browse/HBASE-18972
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18972-v1.patch, HBASE-18972-v2.patch, 
> HBASE-18972-v2.patch, HBASE-18972.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235200#comment-16235200
 ] 

stack commented on HBASE-19152:
---

.005 adds notes around the profiles -Prelease and -Papache-release saying where 
they come from and why apply them. Fixed the nice shellcheck complaint stuff 
above (all but one).

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch, 
> HBASE-19152.master.005.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19152:
--
Attachment: HBASE-19152.master.005.patch

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch, 
> HBASE-19152.master.005.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18827) Site build takes 1.5 hours; it looks like it is stuck...

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235193#comment-16235193
 ] 

stack commented on HBASE-18827:
---

Running site a few times I get:

real26m35.453s
user43m21.790s
sys 3m5.750s

and

real27m13.986s
user43m24.673s
sys 3m13.040s



> Site build takes 1.5 hours; it looks like it is stuck...
> 
>
> Key: HBASE-18827
> URL: https://issues.apache.org/jira/browse/HBASE-18827
> Project: HBase
>  Issue Type: Bug
>  Components: site
>Reporter: stack
>Priority: Minor
>
> I'm trying to make a release. I'm in a tizzy as is usual around these times*. 
> 1.5 hours seems totally over-the-top. I think [~misty]'s lovely automation 
> has hidden this fact from us but needs digging on why we are taking so long. 
> The cycle seems to be provoked by hbase-archetypes module but I got 'mvn 
> log glaze disease' as soon as I tried digging in.
> Filing issue in case someone else wants to have a go at this before I. Also 
> filing if only to note that the thing does eventually finish in case I 
> forget
> * I go to build the doc/site and the build never seems to end. First there is 
> the issue HBASE-18821 where we were NPE'ing after a bunch of time had passed. 
> My fix for HBASE-18821 then got me further but after what seemed hours, it 
> failed with a cryptic message (Thanks [~Apache9] for figuring that I'd made 
> an incorrect fix over in HBASE-18821).  Next up I'm looking at a cycle that 
> never seems to end... only it eventually does after 90minutes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18232) Add variable size chunks to the MSLAB

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235183#comment-16235183
 ] 

ramkrishna.s.vasudevan commented on HBASE-18232:


So this is committed to trunk only? So you can push this to branch-2 also and 
let this one and the other Sub JIRAs to make this complete land in 2.0 beta? Is 
that fine?

> Add variable size chunks to the MSLAB
> -
>
> Key: HBASE-18232
> URL: https://issues.apache.org/jira/browse/HBASE-18232
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
>Priority: Major
> Attachments: HBASE-18232-V03.patch, HBASE-18232-V05.patch, 
> HBASE-18232-V06.patch, HBASE-18232-V07.patch, HBASE-18232-V08.patch, 
> HBASE-18232-V09.patch, HBASE-18232-V10.patch
>
>
> Add possibility to create a variable size chunks of memory, so any cell (of 
> any size) can reside on a chunk.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19147) All branch-2 unit tests pass

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235178#comment-16235178
 ] 

stack commented on HBASE-19147:
---

Thanks [~elserj] I'm starting to go through them...

> All branch-2 unit tests pass
> 
>
> Key: HBASE-19147
> URL: https://issues.apache.org/jira/browse/HBASE-19147
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235172#comment-16235172
 ] 

Josh Elser commented on HBASE-19118:


[~reidchan], can you take a look at branch-1.2 and branch-1.3 and put a patch 
together for those branches, please? (my hunch is that one patch will do it). 
I've already pushed to the branches that correspond to the fixVersions I just 
set.

Thanks for your work!

> Use SaslUtil to set Sasl.QOP in 'Thrift'
> 
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-beta-1
>
> Attachments: HBASE-19118.master.001.patch, 
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift 
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says 
> "set the property {{hbase.thrift.security.qop}} to one of the following three 
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to 
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}}, 
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated HBASE-19118:
---
Fix Version/s: 2.0.0-beta-1
   1.5.0
   1.4.0
   3.0.0

> Use SaslUtil to set Sasl.QOP in 'Thrift'
> 
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-beta-1
>
> Attachments: HBASE-19118.master.001.patch, 
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift 
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says 
> "set the property {{hbase.thrift.security.qop}} to one of the following three 
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to 
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}}, 
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19124) Move HBase-Nightly source artifact creation test from JenkinsFile to a script in dev-support

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235166#comment-16235166
 ] 

Sean Busbey commented on HBASE-19124:
-

here's a build on that branch that shows the working output and the archived 
logs: 
https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/HBASE-19124/7/

> Move HBase-Nightly source artifact creation test from JenkinsFile to a script 
> in dev-support
> 
>
> Key: HBASE-19124
> URL: https://issues.apache.org/jira/browse/HBASE-19124
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-19124.0.patch, HBASE-19124.1.patch
>
>
> ref: 
> https://issues.apache.org/jira/browse/HBASE-19089?focusedCommentId=16221551=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16221551
> Moving that jenkins test to script will make it easier to test changes to 
> hbase-assembly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19147) All branch-2 unit tests pass

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235162#comment-16235162
 ] 

Josh Elser commented on HBASE-19147:


TestNamespacesInstanceResource, TestReplicaWithCluster, TestReplicationAdmin, 
and TestRegionServerHostname are failing for me locally as of alpha-4 rc0

> All branch-2 unit tests pass
> 
>
> Key: HBASE-19147
> URL: https://issues.apache.org/jira/browse/HBASE-19147
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18972) Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235155#comment-16235155
 ] 

Hadoop QA commented on HBASE-18972:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
33s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} hbase-client: The patch generated 0 new + 2 
unchanged - 1 fixed = 2 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} hbase-endpoint: The patch generated 0 new + 11 
unchanged - 1 fixed = 11 total (was 12) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
50s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
49m 10s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
41s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
36s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 10s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18972 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12895334/HBASE-18972-v2.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux da022b87b870 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 GNU/Linux |
| 

[jira] [Commented] (HBASE-19118) Use SaslUtil to set Sasl.QOP in 'Thrift'

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235147#comment-16235147
 ] 

Josh Elser commented on HBASE-19118:


Nope, fine by me. Running through cherry-picking this to all the branches. 
Slight conflicts, but easy to fix.

> Use SaslUtil to set Sasl.QOP in 'Thrift'
> 
>
> Key: HBASE-19118
> URL: https://issues.apache.org/jira/browse/HBASE-19118
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-19118.master.001.patch, 
> HBASE-19118.master.002.patch, HBASE-19118.master.003.patch
>
>
> In [Configure the Thrift 
> Gateway|http://hbase.apache.org/book.html#security.client.thrift], it says 
> "set the property {{hbase.thrift.security.qop}} to one of the following three 
> values: {{privacy}}, {{integrity}}, {{authentication}}", which would lead to 
> failure of starting up a thrift server.
> In fact, the value of {{hbase.thrift.security.qop}} should be {{auth}}, 
> {{auth-int}}, {{auth-conf}}, according to the documentation of {{Sasl.QOP}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19122) preCompact and preFlush can bypass by returning null scanner; shut it down

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235142#comment-16235142
 ] 

stack commented on HBASE-19122:
---

If CP returns null, throw a CPException (wrapping a NPE?). The NonNull won't 
catch runtime CPs returning null? (I should add the annotation though as hint 
to user). Thanks [~Apache9]

> preCompact and preFlush can bypass by returning null scanner; shut it down
> --
>
> Key: HBASE-19122
> URL: https://issues.apache.org/jira/browse/HBASE-19122
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors, Scanners
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> Noticed by [~anoop.hbase] during review of HBASE-18770, preCompact and 
> preFlush can bypass normal processing by returning null. They are not 
> bypasable by ordained route. We should shut down this avenue.
> The preCompact at least may be new coming in with:
> {code}
> tree dbf13093842f85a713f023d7219caccf8f4eb05f
> parent a4dcf51415616772e462091ce93622f070ea8810
> author zhangduo  Sat Apr 9 16:18:08 2016 +0800
> committer zhangduo  Sun Apr 10 09:26:28 2016 +0800
> HBASE-15527 Refactor Compactor related classes
> {code}
> Would have to dig in more to figure for sure.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-19141) [compat 1-2] getClusterStatus always return empty ClusterStatus

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235135#comment-16235135
 ] 

Reid Chan edited comment on HBASE-19141 at 11/2/17 3:58 AM:


 I recommend to add following codes 
{code}
// given that hbase1 can't submit the request with Option,
// we return all information to client if the list of Option is empty.
if (options.isEmpty()) {
   options = EnumSet.allOf(Option.class);
}
{code}
in method {{EnumSet toOptions(List 
options)}} in {{ProtobufUtil}}, instead of in {{HMaster}}
{code}
public static EnumSet toOptions(List 
options) {
// given that hbase1 can't submit the request with Option,
// we return all information to client if the list of Option is empty.
if (options.isEmpty()) {
   return EnumSet.allOf(Option.class);
}
...
}
{code}
though it is committed. : p


was (Author: reidchan):
 I recommend to add following codes 
{code}
// given that hbase1 can't submit the request with Option,
// we return all information to client if the list of Option is empty.
if (options.isEmpty()) {
   options = EnumSet.allOf(Option.class);
}
{code}
in method {{EnumSet toOptions(List 
options)}} in {{ProtobufUtil}}, 
{code}
public static EnumSet toOptions(List 
options) {
// given that hbase1 can't submit the request with Option,
// we return all information to client if the list of Option is empty.
if (options.isEmpty()) {
   return EnumSet.allOf(Option.class);
}
...
}
{code}
though it is committed. : p

>  [compat 1-2] getClusterStatus always return empty ClusterStatus
> 
>
> Key: HBASE-19141
> URL: https://issues.apache.org/jira/browse/HBASE-19141
> Project: HBase
>  Issue Type: Sub-task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19141.v0.patch, HBASE-19141.v1.patch
>
>
> We are able to limit the scope to get part of {{ClusterStatus}} in 2.0. 
> However the request sent by 1.x client has no specific scope info to retrieve 
> any information from {{ClusterStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235136#comment-16235136
 ] 

Hadoop QA commented on HBASE-19144:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m 
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
12s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
14s{color} | {color:green} branch-1 passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
15s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
41s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
36s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
10s{color} | {color:green} branch-1 passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
28s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 40s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3. 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
59s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 8s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m  0s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:4bf71c3 |
| JIRA Issue | HBASE-19144 |
| JIRA Patch URL | 

[jira] [Commented] (HBASE-19141) [compat 1-2] getClusterStatus always return empty ClusterStatus

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235135#comment-16235135
 ] 

Reid Chan commented on HBASE-19141:
---

 I recommend to add following codes 
{code}
// given that hbase1 can't submit the request with Option,
// we return all information to client if the list of Option is empty.
if (options.isEmpty()) {
   options = EnumSet.allOf(Option.class);
}
{code}
in method {{EnumSet toOptions(List 
options)}} in {{ProtobufUtil}}, 
{code}
public static EnumSet toOptions(List 
options) {
// given that hbase1 can't submit the request with Option,
// we return all information to client if the list of Option is empty.
if (options.isEmpty()) {
   return EnumSet.allOf(Option.class);
}
...
}
{code}
though it is committed. : p

>  [compat 1-2] getClusterStatus always return empty ClusterStatus
> 
>
> Key: HBASE-19141
> URL: https://issues.apache.org/jira/browse/HBASE-19141
> Project: HBase
>  Issue Type: Sub-task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19141.v0.patch, HBASE-19141.v1.patch
>
>
> We are able to limit the scope to get part of {{ClusterStatus}} in 2.0. 
> However the request sent by 1.x client has no specific scope info to retrieve 
> any information from {{ClusterStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235133#comment-16235133
 ] 

stack commented on HBASE-19152:
---

Duh. -Prelease is a profile from our pom.xml. Trying build with both apache 
apache-release and our release profiles in place.

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18972) Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235115#comment-16235115
 ] 

Duo Zhang commented on HBASE-18972:
---

[~appy] Any other concerns for the new patch? Thanks.

> Use Builder pattern to remove nullable parameters for coprocessor methods in 
> RawAsyncTable interface
> 
>
> Key: HBASE-18972
> URL: https://issues.apache.org/jira/browse/HBASE-18972
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18972-v1.patch, HBASE-18972-v2.patch, 
> HBASE-18972-v2.patch, HBASE-18972.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19122) preCompact and preFlush can bypass by returning null scanner; shut it down

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235114#comment-16235114
 ] 

Duo Zhang commented on HBASE-19122:
---

How do we fix this? The cp hooks for in memory compaction has the same problem. 
In HBASE-19095, I added a NonNull annotation to the method in RegionObserver 
and did not do null check for the return value.

Thanks.

> preCompact and preFlush can bypass by returning null scanner; shut it down
> --
>
> Key: HBASE-19122
> URL: https://issues.apache.org/jira/browse/HBASE-19122
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors, Scanners
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> Noticed by [~anoop.hbase] during review of HBASE-18770, preCompact and 
> preFlush can bypass normal processing by returning null. They are not 
> bypasable by ordained route. We should shut down this avenue.
> The preCompact at least may be new coming in with:
> {code}
> tree dbf13093842f85a713f023d7219caccf8f4eb05f
> parent a4dcf51415616772e462091ce93622f070ea8810
> author zhangduo  Sat Apr 9 16:18:08 2016 +0800
> committer zhangduo  Sun Apr 10 09:26:28 2016 +0800
> HBASE-15527 Refactor Compactor related classes
> {code}
> Would have to dig in more to figure for sure.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19095) Add CP hooks in RegionObserver for in memory compaction

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19095:
--
Attachment: HBASE-19095-v2.patch

Address the comments on rb.

> Add CP hooks in RegionObserver for in memory compaction
> ---
>
> Key: HBASE-19095
> URL: https://issues.apache.org/jira/browse/HBASE-19095
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19095-v1.patch, HBASE-19095-v2.patch, 
> HBASE-19095.patch
>
>
> This is a hole in our CP hooks.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19131) Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235106#comment-16235106
 ] 

Guanghao Zhang commented on HBASE-19131:


The listDeadServers() can be implemented as a default method in 
Admin/AsyncAdmin interface.
The patch didn't upload to review board, right? I can help to review. :-)

> Add the ClusterStatus hook and cleanup other hooks which can be replaced by 
> ClusterStatus hook
> --
>
> Key: HBASE-19131
> URL: https://issues.apache.org/jira/browse/HBASE-19131
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19131.v0.patch, HBASE-19131.v1.patch
>
>
> This issue will add the hook for {{ClusterStatus}} and cleanup other hooks 
> which can be replaced by {{ClusterStatus}} hook.
> # {{ListDeadServers}} -  introduced by HBASE-18131



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18972) Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-18972:
--
Attachment: HBASE-18972-v2.patch

Retry.

> Use Builder pattern to remove nullable parameters for coprocessor methods in 
> RawAsyncTable interface
> 
>
> Key: HBASE-18972
> URL: https://issues.apache.org/jira/browse/HBASE-18972
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18972-v1.patch, HBASE-18972-v2.patch, 
> HBASE-18972-v2.patch, HBASE-18972.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19146) Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)

[Yuqi Gu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235097#comment-16235097
 ] 

Yuqi Gu commented on HBASE-19146:
-

Yes of course, I'd like to provide the patch for this issue.

> Hbase3.0  protobuf-maven-plugin do not support Arm64(only for x86)
> --
>
> Key: HBASE-19146
> URL: https://issues.apache.org/jira/browse/HBASE-19146
> Project: HBase
>  Issue Type: Bug
>  Components: build, pom
>Affects Versions: 3.0.0
> Environment: OS:  Ubuntu 16.04.3 
> OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
> Hw platform:  AARCH64
>Reporter: Yuqi Gu
>Priority: Major
>
> We are building the HBase 3.0.0-SNAPSHOT on AARCH64.
> It is noted that 'protobuf-maven-plugin' only support x86 shown as follows:
> {code:java}
>  
>org.xolstice.maven.plugins
>protobuf-maven-plugin
>${protobuf.plugin.version}
>
>   com.google.protobuf:protoc:${external.protobuf.version}:
> exe:${os.detected.classifier}
> 
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
> 
>false
>true
>   
> 
> {code}
> So the build is failed.
> {code:java}
> [INFO] --- protobuf-maven-plugin:0.5.0:compile (compile-protoc) @ 
> hbase-protocol-shaded ---
> [INFO] Compiling 32 proto file(s) to 
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> Failed to execute goal 
> org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile 
> (compile-protoc) on project hbase-protocol-shaded: Missing:
> {code}
> Then I installed aarch64 protobuf 2.5.0 on the host and modify the pom:
> {code:java}
> -   ${basedir}/src/main/protobuf/
> +  /usr/local/bin/protoc
> {code}
>  The build is also failed:
> {code:java}
> [INFO] Compiling 32 proto file(s) to 
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> [ERROR] PROTOC FAILED: google/protobuf/any.proto:31:10: Unrecognized syntax 
> identifier "proto3".  This parser only recognizes "proto2".
> {code}
> It seems that "internal.protobuf.version" in "hbase-protocol-shaded" is 3.3.0.
> How to fix it? Thanks!
>   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18624) Added support for clearing BlockCache based on table name

[Zach York (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zach York updated HBASE-18624:
--
Status: In Progress  (was: Patch Available)

> Added support for clearing BlockCache based on table name
> -
>
> Key: HBASE-18624
> URL: https://issues.apache.org/jira/browse/HBASE-18624
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Ajay Jadhav
>Assignee: Zach York
>Priority: Major
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18624.branch-1.001.patch, 
> HBASE-18624.master.001.patch, HBASE-18624.master.002.patch, 
> HBASE-18624.master.003.patch, HBASE-18624.master.004.patch, 
> HBASE-18624.master.005.patch, HBASE-18624.master.006.patch, 
> HBASE-18624.master.007.patch, HBASE-18624.master.008.patch, 
> HBASE-18624.master.009.patch
>
>
> Bulk loading the primary HBase cluster triggers a lot of compactions 
> resulting in archival/ creation
> of multiple HFiles. This process will cause a lot of items to become stale in 
> replica’s BlockCache.
> This patch will help users to clear the block cache for a given table by 
> either using shell or API.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18624) Added support for clearing BlockCache based on table name

[Zach York (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zach York updated HBASE-18624:
--
Status: Patch Available  (was: In Progress)

> Added support for clearing BlockCache based on table name
> -
>
> Key: HBASE-18624
> URL: https://issues.apache.org/jira/browse/HBASE-18624
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Ajay Jadhav
>Assignee: Zach York
>Priority: Major
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18624.branch-1.001.patch, 
> HBASE-18624.master.001.patch, HBASE-18624.master.002.patch, 
> HBASE-18624.master.003.patch, HBASE-18624.master.004.patch, 
> HBASE-18624.master.005.patch, HBASE-18624.master.006.patch, 
> HBASE-18624.master.007.patch, HBASE-18624.master.008.patch, 
> HBASE-18624.master.009.patch
>
>
> Bulk loading the primary HBase cluster triggers a lot of compactions 
> resulting in archival/ creation
> of multiple HFiles. This process will cause a lot of items to become stale in 
> replica’s BlockCache.
> This patch will help users to clear the block cache for a given table by 
> either using shell or API.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18624) Added support for clearing BlockCache based on table name

[Zach York (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zach York updated HBASE-18624:
--
Attachment: HBASE-18624.master.009.patch

> Added support for clearing BlockCache based on table name
> -
>
> Key: HBASE-18624
> URL: https://issues.apache.org/jira/browse/HBASE-18624
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Ajay Jadhav
>Assignee: Zach York
>Priority: Major
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18624.branch-1.001.patch, 
> HBASE-18624.master.001.patch, HBASE-18624.master.002.patch, 
> HBASE-18624.master.003.patch, HBASE-18624.master.004.patch, 
> HBASE-18624.master.005.patch, HBASE-18624.master.006.patch, 
> HBASE-18624.master.007.patch, HBASE-18624.master.008.patch, 
> HBASE-18624.master.009.patch
>
>
> Bulk loading the primary HBase cluster triggers a lot of compactions 
> resulting in archival/ creation
> of multiple HFiles. This process will cause a lot of items to become stale in 
> replica’s BlockCache.
> This patch will help users to clear the block cache for a given table by 
> either using shell or API.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19150) TestSnapshotWithAcl is flaky

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235073#comment-16235073
 ] 

Mike Drob commented on HBASE-19150:
---

Other than that, great patch - I was looking at this failure last week but got 
distracted before I could figure out what was going on. Thanks, 
[~churromorales]!

> TestSnapshotWithAcl is flaky
> 
>
> Key: HBASE-19150
> URL: https://issues.apache.org/jira/browse/HBASE-19150
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: churro morales
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19150.patch
>
>
> testRestoreSnapshot(org.apache.hadoop.hbase.client.TestSnapshotWithAcl) Time 
> elapsed: 1.425 sec <<< FAILURE!
> java.lang.AssertionError: Expected action to pass for user 'owner' but was 
> denied
> at org.junit.Assert.fail(Assert.java:88)
> at 
> org.apache.hadoop.hbase.security.access.SecureTestUtil.verifyAllowed(SecureTestUtil.java:185)
> at 
> org.apache.hadoop.hbase.security.access.SecureTestUtil.verifyAllowed(SecureTestUtil.java:193)
> at 
> org.apache.hadoop.hbase.client.TestSnapshotWithAcl.testRestoreSnapshot(TestSnapshotWithAcl.java:188)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19150) TestSnapshotWithAcl is flaky

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235072#comment-16235072
 ] 

Mike Drob commented on HBASE-19150:
---

UUID.randomUUID uses the SecureRandom generator and uses up system entropy. Can 
we switch to a different generator?

> TestSnapshotWithAcl is flaky
> 
>
> Key: HBASE-19150
> URL: https://issues.apache.org/jira/browse/HBASE-19150
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: churro morales
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19150.patch
>
>
> testRestoreSnapshot(org.apache.hadoop.hbase.client.TestSnapshotWithAcl) Time 
> elapsed: 1.425 sec <<< FAILURE!
> java.lang.AssertionError: Expected action to pass for user 'owner' but was 
> denied
> at org.junit.Assert.fail(Assert.java:88)
> at 
> org.apache.hadoop.hbase.security.access.SecureTestUtil.verifyAllowed(SecureTestUtil.java:185)
> at 
> org.apache.hadoop.hbase.security.access.SecureTestUtil.verifyAllowed(SecureTestUtil.java:193)
> at 
> org.apache.hadoop.hbase.client.TestSnapshotWithAcl.testRestoreSnapshot(TestSnapshotWithAcl.java:188)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19153) LruBlockCache cache too big blocks logic error

[Zhang Quanjin (JIRA)]

Zhang Quanjin created HBASE-19153:
-

 Summary: LruBlockCache cache too big blocks logic error
 Key: HBASE-19153
 URL: https://issues.apache.org/jira/browse/HBASE-19153
 Project: HBase
  Issue Type: Bug
  Components: BlockCache
Affects Versions: 2.0.0-alpha-3
Reporter: Zhang Quanjin


The latest version of LruBolckCache, I found the code logic of cache too big 
bolcks is inconsistent with annotation.
If follow the notes, the code should look like this:

if (buf.heapSize() > maxBlockSize) {
  // If there are a lot of blocks that are too
  // big this can make the logs way too noisy.
  // So we log 2%
  if (stats.failInsert() % 50 != 0) {
return;
  }
  LOG.warn("Trying to cache too large a block "
+ cacheKey.getHfileName() + " @ "
+ cacheKey.getOffset()
+ " is " + buf.heapSize()
+ " which is larger than " + maxBlockSize);
  
}





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235058#comment-16235058
 ] 

Andrew Purtell commented on HBASE-19144:


WDYT? [~lhofhansl] [~churromorales]

> [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the 
> cluster
> ---
>
> Key: HBASE-19144
> URL: https://issues.apache.org/jira/browse/HBASE-19144
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19144-branch-1.patch, HBASE-19144.patch
>
>
> After all servers in the RSgroup are down the regions cannot be opened 
> anywhere and transition rapidly into FAILED_OPEN state.
>  
> 2017-10-31 21:06:25,449 INFO [ProcedureExecutor-13] master.RegionStates: 
> Transition {c6c8150c9f4b8df25ba32073f25a5143 state=OFFLINE, ts=1509483985448, 
> server=node-5.cluster,16020,1509482700768} to 
> {c6c8150c9f4b8df25ba32073f25a5143 state=FAILED_OPEN, ts=1509483985449, 
> server=node-5.cluster,16020,1509482700768}
> 2017-10-31 21:06:25,449 WARN [ProcedureExecutor-13] master.RegionStates: 
> Failed to open/close d4e2f173e31ffad6aac140f4bd7b02bc on 
> node-5.cluster,16020,1509482700768, set to FAILED_OPEN
>  
> Any region in FAILED_OPEN state has to be manually reassigned, or the master 
> can be restarted and this will also cause reattempt of assignment of any 
> regions in FAILED_OPEN state. This is not unexpected but is an operational 
> headache. It would be better if the RSGroupInfoManager could automatically 
> kick reassignments of regions in FAILED_OPEN state when servers rejoin the 
> cluster. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19150) TestSnapshotWithAcl is flaky

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19150:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to branch-1.4 and up

> TestSnapshotWithAcl is flaky
> 
>
> Key: HBASE-19150
> URL: https://issues.apache.org/jira/browse/HBASE-19150
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: churro morales
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19150.patch
>
>
> testRestoreSnapshot(org.apache.hadoop.hbase.client.TestSnapshotWithAcl) Time 
> elapsed: 1.425 sec <<< FAILURE!
> java.lang.AssertionError: Expected action to pass for user 'owner' but was 
> denied
> at org.junit.Assert.fail(Assert.java:88)
> at 
> org.apache.hadoop.hbase.security.access.SecureTestUtil.verifyAllowed(SecureTestUtil.java:185)
> at 
> org.apache.hadoop.hbase.security.access.SecureTestUtil.verifyAllowed(SecureTestUtil.java:193)
> at 
> org.apache.hadoop.hbase.client.TestSnapshotWithAcl.testRestoreSnapshot(TestSnapshotWithAcl.java:188)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18925) Need updated mockito for using java optional

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235046#comment-16235046
 ] 

Hudson commented on HBASE-18925:


FAILURE: Integrated in Jenkins build HBase-2.0 #783 (See 
[https://builds.apache.org/job/HBase-2.0/783/])
HBASE-18925 Update mockito dependency from mockito-all:1.10.19 to (appy: rev 
d69570a485de9c857139394c03aa43827b01e37d)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestTableQuotaViolationStore.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestSpaceQuotaViolationPolicyRefresherChore.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBulkLoad.java
* (edit) hbase-metrics-api/pom.xml
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/MockMasterServices.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java
* (edit) 
hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestHttpServer.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCompactor.java
* (edit) 
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRSGroupBasedLoadBalancer.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestDateTieredCompactor.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALSplit.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestIdentityTableMap.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestDriver.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionArchiveIOException.java
* (edit) hbase-endpoint/pom.xml
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestMasterSpaceQuotaObserverWithMocks.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStore.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALPlayer.java
* (edit) hbase-http/pom.xml
* (edit) src/main/asciidoc/_chapters/unit_testing.adoc
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
* (edit) hbase-shaded/hbase-shaded-check-invariants/pom.xml
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeStoreEngine.java
* (edit) hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestLoadIncrementalHFilesSplitRecovery.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestRowCounter.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/errorhandling/TestForeignExceptionDispatcher.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormatImpl.java
* (edit) 
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteHTableRetries.java
* (edit) hbase-common/pom.xml
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactor.java
* (edit) pom.xml
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseTestingUtility.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableLocator.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/StatefulStoreMockMaker.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java
* (edit) 
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestGroupingTableMapper.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/errorhandling/TestTimeoutExceptionInjector.java
* (edit) 
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientScanner.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerRegionSpaceUseReport.java
* (edit) 
hbase-client/src/test/java/org/apache/hadoop/hbase/security/TestHBaseSaslRpcClient.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/example/TestZooKeeperTableArchiveClient.java
* (edit) hbase-metrics/pom.xml
* (edit) 

[jira] [Commented] (HBASE-19140) hbase-cleanup.sh uses deprecated call to remove files in hdfs

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235045#comment-16235045
 ] 

Hudson commented on HBASE-19140:


FAILURE: Integrated in Jenkins build HBase-2.0 #783 (See 
[https://builds.apache.org/job/HBase-2.0/783/])
HBASE-19140 hbase-cleanup.sh uses deprecated call to remove files in 
(jan.hentschel: rev d407e37cf4584773046fa6def41c0c722015356b)
* (edit) bin/hbase-cleanup.sh


> hbase-cleanup.sh uses deprecated call to remove files in hdfs
> -
>
> Key: HBASE-19140
> URL: https://issues.apache.org/jira/browse/HBASE-19140
> Project: HBase
>  Issue Type: Improvement
>  Components: scripts
>Affects Versions: 2.0.0, 1.3.1, 1.2.6, 1.4.1, 1.1.12
> Environment: MVN: 3.5.2
> JDK: 1.8.0.144
>Reporter: Artem Ervits
>Assignee: Artem Ervits
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13
>
> Attachments: HBASE-19140-00.patch
>
>
> steps to reproduce:
> {code}
> bin/hbase-cleanup.sh --cleanAll
> {code}
> {code}
> rmr: DEPRECATED: Please use 'rm -r' instead.
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19140) hbase-cleanup.sh uses deprecated call to remove files in hdfs

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235034#comment-16235034
 ] 

Hudson commented on HBASE-19140:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3983 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3983/])
HBASE-19140 hbase-cleanup.sh uses deprecated call to remove files in (tedyu: 
rev b3e438b9c5be750694b3cec1f6f754635240d474)
* (edit) bin/hbase-cleanup.sh


> hbase-cleanup.sh uses deprecated call to remove files in hdfs
> -
>
> Key: HBASE-19140
> URL: https://issues.apache.org/jira/browse/HBASE-19140
> Project: HBase
>  Issue Type: Improvement
>  Components: scripts
>Affects Versions: 2.0.0, 1.3.1, 1.2.6, 1.4.1, 1.1.12
> Environment: MVN: 3.5.2
> JDK: 1.8.0.144
>Reporter: Artem Ervits
>Assignee: Artem Ervits
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13
>
> Attachments: HBASE-19140-00.patch
>
>
> steps to reproduce:
> {code}
> bin/hbase-cleanup.sh --cleanAll
> {code}
> {code}
> rmr: DEPRECATED: Please use 'rm -r' instead.
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19144:
---
Attachment: HBASE-19144-branch-1.patch
HBASE-19144.patch

> [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the 
> cluster
> ---
>
> Key: HBASE-19144
> URL: https://issues.apache.org/jira/browse/HBASE-19144
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19144-branch-1.patch, HBASE-19144.patch
>
>
> After all servers in the RSgroup are down the regions cannot be opened 
> anywhere and transition rapidly into FAILED_OPEN state.
>  
> 2017-10-31 21:06:25,449 INFO [ProcedureExecutor-13] master.RegionStates: 
> Transition {c6c8150c9f4b8df25ba32073f25a5143 state=OFFLINE, ts=1509483985448, 
> server=node-5.cluster,16020,1509482700768} to 
> {c6c8150c9f4b8df25ba32073f25a5143 state=FAILED_OPEN, ts=1509483985449, 
> server=node-5.cluster,16020,1509482700768}
> 2017-10-31 21:06:25,449 WARN [ProcedureExecutor-13] master.RegionStates: 
> Failed to open/close d4e2f173e31ffad6aac140f4bd7b02bc on 
> node-5.cluster,16020,1509482700768, set to FAILED_OPEN
>  
> Any region in FAILED_OPEN state has to be manually reassigned, or the master 
> can be restarted and this will also cause reattempt of assignment of any 
> regions in FAILED_OPEN state. This is not unexpected but is an operational 
> headache. It would be better if the RSGroupInfoManager could automatically 
> kick reassignments of regions in FAILED_OPEN state when servers rejoin the 
> cluster. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19144:
---
Attachment: (was: HBASE-19144-branch-1.patch)

> [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the 
> cluster
> ---
>
> Key: HBASE-19144
> URL: https://issues.apache.org/jira/browse/HBASE-19144
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19144-branch-1.patch, HBASE-19144.patch
>
>
> After all servers in the RSgroup are down the regions cannot be opened 
> anywhere and transition rapidly into FAILED_OPEN state.
>  
> 2017-10-31 21:06:25,449 INFO [ProcedureExecutor-13] master.RegionStates: 
> Transition {c6c8150c9f4b8df25ba32073f25a5143 state=OFFLINE, ts=1509483985448, 
> server=node-5.cluster,16020,1509482700768} to 
> {c6c8150c9f4b8df25ba32073f25a5143 state=FAILED_OPEN, ts=1509483985449, 
> server=node-5.cluster,16020,1509482700768}
> 2017-10-31 21:06:25,449 WARN [ProcedureExecutor-13] master.RegionStates: 
> Failed to open/close d4e2f173e31ffad6aac140f4bd7b02bc on 
> node-5.cluster,16020,1509482700768, set to FAILED_OPEN
>  
> Any region in FAILED_OPEN state has to be manually reassigned, or the master 
> can be restarted and this will also cause reattempt of assignment of any 
> regions in FAILED_OPEN state. This is not unexpected but is an operational 
> headache. It would be better if the RSGroupInfoManager could automatically 
> kick reassignments of regions in FAILED_OPEN state when servers rejoin the 
> cluster. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19144:
---
Attachment: (was: HBASE-19144-branch-1.patch)

> [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the 
> cluster
> ---
>
> Key: HBASE-19144
> URL: https://issues.apache.org/jira/browse/HBASE-19144
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19144-branch-1.patch, HBASE-19144.patch
>
>
> After all servers in the RSgroup are down the regions cannot be opened 
> anywhere and transition rapidly into FAILED_OPEN state.
>  
> 2017-10-31 21:06:25,449 INFO [ProcedureExecutor-13] master.RegionStates: 
> Transition {c6c8150c9f4b8df25ba32073f25a5143 state=OFFLINE, ts=1509483985448, 
> server=node-5.cluster,16020,1509482700768} to 
> {c6c8150c9f4b8df25ba32073f25a5143 state=FAILED_OPEN, ts=1509483985449, 
> server=node-5.cluster,16020,1509482700768}
> 2017-10-31 21:06:25,449 WARN [ProcedureExecutor-13] master.RegionStates: 
> Failed to open/close d4e2f173e31ffad6aac140f4bd7b02bc on 
> node-5.cluster,16020,1509482700768, set to FAILED_OPEN
>  
> Any region in FAILED_OPEN state has to be manually reassigned, or the master 
> can be restarted and this will also cause reattempt of assignment of any 
> regions in FAILED_OPEN state. This is not unexpected but is an operational 
> headache. It would be better if the RSGroupInfoManager could automatically 
> kick reassignments of regions in FAILED_OPEN state when servers rejoin the 
> cluster. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235009#comment-16235009
 ] 

Hadoop QA commented on HBASE-19152:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue}  0m  
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
 5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red}  0m  
0s{color} | {color:red} The patch generated 3 new + 8 unchanged - 6 fixed = 11 
total (was 14) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
0s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}146m 53s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
38s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m  8s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19152 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12895274/HBASE-19152.master.004.patch
 |
| Optional Tests |  asflicense  shellcheck  shelldocs  javac  javadoc  unit  
xml  |
| uname | Linux 1baaed1d2205 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 
18:04:35 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 71a55dcd64 |
| Default Java | 1.8.0_141 |
| shellcheck | v0.4.4 |
| shellcheck | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9566/artifact/patchprocess/diff-patch-shellcheck.txt
 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9566/artifact/patchprocess/whitespace-eol.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9566/artifact/patchprocess/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9566/testReport/ |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9566/console |
| Powered by | Apache Yetus 0.5.0   http://yetus.apache.org |


This message was automatically generated.



> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 

[jira] [Comment Edited] (HBASE-12091) Optionally ignore edits for dropped tables for replication.

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234989#comment-16234989
 ] 

Lars Hofhansl edited comment on HBASE-12091 at 11/2/17 12:19 AM:
-

-v6 works.

The tests verify the default behavior, and the optional behavior with and 
without namespaces.

It's ugly, though. The code needs to parse out the table name from the 
exception, it double checks that it got a valid table-name, and check back for 
a local table... So it errs of the side of being conservative and does not drop 
the edits. Still it's ugly and fragile. At least there is a test now making 
sure that nobody messes with the exception message.


was (Author: lhofhansl):
-v6 works.

The tests verify the default behavior, and the optional behavior with and 
without namespaces.

It's ugly, though. The code needs to parse out the table name from the 
exception, it double checks that it got a valid table-name, and check back for 
a local table... So it errs of the side of being conservative and does not drop 
the edits. Still it's ugly and fragile.

> Optionally ignore edits for dropped tables for replication.
> ---
>
> Key: HBASE-12091
> URL: https://issues.apache.org/jira/browse/HBASE-12091
> Project: HBase
>  Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: 12091-v2-branch-1.txt, 12091-v3-branch-1.txt, 
> 12091-v4-branch-1.txt, 12091-v5-branch-1.txt, 12091-v6-branch-1.txt, 12091.txt
>
>
> We just ran into a scenario where we dropped a table from both the source and 
> the sink, but the source still has outstanding edits that now it could not 
> get rid of. Now all replication is backed up behind these unreplicatable 
> edits.
> We should have an option to ignore edits for tables dropped at the source.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234998#comment-16234998
 ] 

Andrew Purtell commented on HBASE-19144:


Patch which allows configuration of the wait interval after we detect added 
servers before reassignment attempts. Set the default to 10 seconds. Tested on 
a 5 node cluster. 

- Create a test table. Move the test table and two servers into a new RSGroup. 
- Populate the table with 1M rows, count them, confirmed 1M rows present and 
reachable.
- Shut down the two servers in the RSGroup. Note test table regions transition 
to FAILED_OPEN.
- Restart the two servers in the RSGroup. Note after 10 seconds the regions are 
redeployed.
- Count the test table, confirmed 1M rows present and reachable.

> [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the 
> cluster
> ---
>
> Key: HBASE-19144
> URL: https://issues.apache.org/jira/browse/HBASE-19144
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19144-branch-1.patch, HBASE-19144-branch-1.patch
>
>
> After all servers in the RSgroup are down the regions cannot be opened 
> anywhere and transition rapidly into FAILED_OPEN state.
>  
> 2017-10-31 21:06:25,449 INFO [ProcedureExecutor-13] master.RegionStates: 
> Transition {c6c8150c9f4b8df25ba32073f25a5143 state=OFFLINE, ts=1509483985448, 
> server=node-5.cluster,16020,1509482700768} to 
> {c6c8150c9f4b8df25ba32073f25a5143 state=FAILED_OPEN, ts=1509483985449, 
> server=node-5.cluster,16020,1509482700768}
> 2017-10-31 21:06:25,449 WARN [ProcedureExecutor-13] master.RegionStates: 
> Failed to open/close d4e2f173e31ffad6aac140f4bd7b02bc on 
> node-5.cluster,16020,1509482700768, set to FAILED_OPEN
>  
> Any region in FAILED_OPEN state has to be manually reassigned, or the master 
> can be restarted and this will also cause reattempt of assignment of any 
> regions in FAILED_OPEN state. This is not unexpected but is an operational 
> headache. It would be better if the RSGroupInfoManager could automatically 
> kick reassignments of regions in FAILED_OPEN state when servers rejoin the 
> cluster. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19144:
---
Attachment: HBASE-19144-branch-1.patch

> [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the 
> cluster
> ---
>
> Key: HBASE-19144
> URL: https://issues.apache.org/jira/browse/HBASE-19144
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-19144-branch-1.patch, HBASE-19144-branch-1.patch
>
>
> After all servers in the RSgroup are down the regions cannot be opened 
> anywhere and transition rapidly into FAILED_OPEN state.
>  
> 2017-10-31 21:06:25,449 INFO [ProcedureExecutor-13] master.RegionStates: 
> Transition {c6c8150c9f4b8df25ba32073f25a5143 state=OFFLINE, ts=1509483985448, 
> server=node-5.cluster,16020,1509482700768} to 
> {c6c8150c9f4b8df25ba32073f25a5143 state=FAILED_OPEN, ts=1509483985449, 
> server=node-5.cluster,16020,1509482700768}
> 2017-10-31 21:06:25,449 WARN [ProcedureExecutor-13] master.RegionStates: 
> Failed to open/close d4e2f173e31ffad6aac140f4bd7b02bc on 
> node-5.cluster,16020,1509482700768, set to FAILED_OPEN
>  
> Any region in FAILED_OPEN state has to be manually reassigned, or the master 
> can be restarted and this will also cause reattempt of assignment of any 
> regions in FAILED_OPEN state. This is not unexpected but is an operational 
> headache. It would be better if the RSGroupInfoManager could automatically 
> kick reassignments of regions in FAILED_OPEN state when servers rejoin the 
> cluster. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-12091) Optionally ignore edits for dropped tables for replication.

[Lars Hofhansl (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-12091:
--
Attachment: 12091-v6-branch-1.txt

-v6 works.

The tests verify the default behavior, and the optional behavior with and 
without namespaces.

It's ugly, though. The code needs to parse out the table name from the 
exception, it double checks that it got a valid table-name, and check back for 
a local table... So it errs of the side of being conservative and does not drop 
the edits. Still it's ugly and fragile.

> Optionally ignore edits for dropped tables for replication.
> ---
>
> Key: HBASE-12091
> URL: https://issues.apache.org/jira/browse/HBASE-12091
> Project: HBase
>  Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: 12091-v2-branch-1.txt, 12091-v3-branch-1.txt, 
> 12091-v4-branch-1.txt, 12091-v5-branch-1.txt, 12091-v6-branch-1.txt, 12091.txt
>
>
> We just ran into a scenario where we dropped a table from both the source and 
> the sink, but the source still has outstanding edits that now it could not 
> get rid of. Now all replication is backed up behind these unreplicatable 
> edits.
> We should have an option to ignore edits for tables dropped at the source.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-12125) Add Hbck option to check and fix WAL's from replication queue

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-12125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234982#comment-16234982
 ] 

Ted Yu commented on HBASE-12125:


Running new test against hadoop3 beta1, I got:
{code}
testFixMissingReplicationWAL(org.apache.hadoop.hbase.util.TestHBaseFsckReplication)
  Time elapsed: 49.211 sec  <<< ERROR!
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.util.TestHBaseFsckReplication.testFixMissingReplicationWAL(TestHBaseFsckReplication.java:184)
{code}
See if the above can be reproduced on hadoop2

> Add Hbck option to check and fix WAL's from replication queue
> -
>
> Key: HBASE-12125
> URL: https://issues.apache.org/jira/browse/HBASE-12125
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Affects Versions: 3.0.0
>Reporter: Virag Kothari
>Assignee: Vincent Poon
>Priority: Major
> Attachments: HBASE-12125.v1.master.patch, HBASE-12125.v2.master.patch
>
>
> The replication source will discard the WAL file in many cases when it 
> encounters an exception reading it . This can cause data loss
> and the underlying reason of failed read remains hidden.  Only in certain 
> scenarios, the replication source should dump the current WAL and move to the 
> next one. 
> This JIRA aims to have an hbck option to check the WAL files of replication 
> queues for any inconsistencies and also provide an option to fix it.
> The fix can be to remove the file from replication queue in zk and from the 
> memory of replication source manager and replication sources. 
> A region server endpoint call from the hbck client to region server can be 
> used to achieve this.
> Hbck can be configured with the following options:
> -softCheckReplicationWAL : Tries to open only the oldest WAL (the WAL 
> currently read by replication source) from replication queue. If there is a 
> position associated, it also seeks to that position and reads an entry from 
> there
> -hardCheckReplicationWAL:  Check all WAL paths from replication queues by 
> reading them completely to make sure they are ok.
> -fixMissingReplicationWAL: Remove the WAL's from replication queues which are 
> not present on hdfs
> -fixCorruptedReplicationWAL:  Remove the WAL's from replication queues which 
> are corrupted (based on the findings from softCheck/hardCheck). Also the 
> WAL's are moved to a quarantine dir
> -rollAndFixCorruptedReplicationWAL - If the current WAL is corrupted, it is 
> first rolled over and then deals with it in the same way as 
> -fixCorruptedReplicationWAL option



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19150) TestSnapshotWithAcl is flaky

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234983#comment-16234983
 ] 

Hadoop QA commented on HBASE-19150:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
44s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
2s{color} | {color:red} hbase-server: The patch generated 1 new + 1 unchanged - 
1 fixed = 2 total (was 2) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
42s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
47m 56s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 
49s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
18s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 38s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19150 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12895263/HBASE-19150.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 0e852a866222 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / b3e438b9c5 |
| Default Java | 1.8.0_141 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9565/artifact/patchprocess/diff-checkstyle-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9565/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/9565/console |
| Powered by | Apache Yetus 0.5.0   http://yetus.apache.org |


This message was automatically generated.



> TestSnapshotWithAcl is flaky
> 
>
> Key: HBASE-19150
>  

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234908#comment-16234908
 ] 

Vladimir Rodionov commented on HBASE-17852:
---

[~elserj], I just wanted to point out, that, in a current implementation, the 
code is deadlock - safe. Timeouts are possible, as always in HBase, but this 
will fail bulk loader (this happens sometimes for other reasons). One can not 
put a finger on this and say - boooh , bad approach :)

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234903#comment-16234903
 ] 

Josh Elser commented on HBASE-17852:


bq. I do not see hot this can result in a distributed deadlock.

I think Andrew is just trying to warn about the kind of problem that can be 
observed with Cross-RS RPCs. It's possible that some lock in one RS might 
accidentally preclude some other RPC from completing. Or, if one RS executes 
enough RPCs that saturate the handlers in another RS, we could soft-lock 
(degrade and appear to be stuck). I don't think he's trying to tell you that he 
believes there to be a concrete problem that exists in the implementation, just 
giving context to why "Cross-RS RPCs" are oft considered smells and raise the 
eyebrows they have raised.

Sorry if I'm putting words in your mouth (fingers), Andrew.

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-12125) Add Hbck option to check and fix WAL's from replication queue

[Vincent Poon (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vincent Poon updated HBASE-12125:
-
Attachment: HBASE-12125.v2.master.patch

> Add Hbck option to check and fix WAL's from replication queue
> -
>
> Key: HBASE-12125
> URL: https://issues.apache.org/jira/browse/HBASE-12125
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Affects Versions: 3.0.0
>Reporter: Virag Kothari
>Assignee: Vincent Poon
>Priority: Major
> Attachments: HBASE-12125.v1.master.patch, HBASE-12125.v2.master.patch
>
>
> The replication source will discard the WAL file in many cases when it 
> encounters an exception reading it . This can cause data loss
> and the underlying reason of failed read remains hidden.  Only in certain 
> scenarios, the replication source should dump the current WAL and move to the 
> next one. 
> This JIRA aims to have an hbck option to check the WAL files of replication 
> queues for any inconsistencies and also provide an option to fix it.
> The fix can be to remove the file from replication queue in zk and from the 
> memory of replication source manager and replication sources. 
> A region server endpoint call from the hbck client to region server can be 
> used to achieve this.
> Hbck can be configured with the following options:
> -softCheckReplicationWAL : Tries to open only the oldest WAL (the WAL 
> currently read by replication source) from replication queue. If there is a 
> position associated, it also seeks to that position and reads an entry from 
> there
> -hardCheckReplicationWAL:  Check all WAL paths from replication queues by 
> reading them completely to make sure they are ok.
> -fixMissingReplicationWAL: Remove the WAL's from replication queues which are 
> not present on hdfs
> -fixCorruptedReplicationWAL:  Remove the WAL's from replication queues which 
> are corrupted (based on the findings from softCheck/hardCheck). Also the 
> WAL's are moved to a quarantine dir
> -rollAndFixCorruptedReplicationWAL - If the current WAL is corrupted, it is 
> first rolled over and then deals with it in the same way as 
> -fixCorruptedReplicationWAL option



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18784) Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234896#comment-16234896
 ] 

Hadoop QA commented on HBASE-18784:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 23 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
30s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
19s{color} | {color:red} hbase-common: The patch generated 81 new + 0 unchanged 
- 0 fixed = 81 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} hbase-procedure: The patch generated 1 new + 118 
unchanged - 2 fixed = 119 total (was 120) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
56s{color} | {color:red} hbase-server: The patch generated 29 new + 384 
unchanged - 41 fixed = 413 total (was 425) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 7s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
41m 32s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 
13s{color} | {color:red} hbase-common generated 1 new + 1 unchanged - 0 fixed = 
2 total (was 1) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m  
6s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
51s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m 
24s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
44s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 24s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18784 |
| JIRA Patch URL | 

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234895#comment-16234895
 ] 

Vladimir Rodionov commented on HBASE-17852:
---

We do *single* RPC call from one RS (A) to another (B). This RPC call 
*terminates* at B. I do not see hot this can result in a distributed deadlock.

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19131) Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-19131:
---
Status: Patch Available  (was: Open)

> Add the ClusterStatus hook and cleanup other hooks which can be replaced by 
> ClusterStatus hook
> --
>
> Key: HBASE-19131
> URL: https://issues.apache.org/jira/browse/HBASE-19131
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19131.v0.patch, HBASE-19131.v1.patch
>
>
> This issue will add the hook for {{ClusterStatus}} and cleanup other hooks 
> which can be replaced by {{ClusterStatus}} hook.
> # {{ListDeadServers}} -  introduced by HBASE-18131



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19131) Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-19131:
---
Attachment: HBASE-19131.v1.patch

v1
# add test for {{ClusterStatus}} hook
# {{ClusterStatus}} hook is executed by normal rpc call from client. On the 
other hand, the {{ClusterStatus}} sync, master web, and balancer won't call the 
{{ClusterStatus}} hook.

> Add the ClusterStatus hook and cleanup other hooks which can be replaced by 
> ClusterStatus hook
> --
>
> Key: HBASE-19131
> URL: https://issues.apache.org/jira/browse/HBASE-19131
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19131.v0.patch, HBASE-19131.v1.patch
>
>
> This issue will add the hook for {{ClusterStatus}} and cleanup other hooks 
> which can be replaced by {{ClusterStatus}} hook.
> # {{ListDeadServers}} -  introduced by HBASE-18131



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19140) hbase-cleanup.sh uses deprecated call to remove files in hdfs

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234871#comment-16234871
 ] 

Hudson commented on HBASE-19140:


SUCCESS: Integrated in Jenkins build HBase-1.5 #127 (See 
[https://builds.apache.org/job/HBase-1.5/127/])
HBASE-19140 hbase-cleanup.sh uses deprecated call to remove files in 
(jan.hentschel: rev b0ff1dd5cb920a11e3d8aab8313a1326284ee922)
* (edit) bin/hbase-cleanup.sh


> hbase-cleanup.sh uses deprecated call to remove files in hdfs
> -
>
> Key: HBASE-19140
> URL: https://issues.apache.org/jira/browse/HBASE-19140
> Project: HBase
>  Issue Type: Improvement
>  Components: scripts
>Affects Versions: 2.0.0, 1.3.1, 1.2.6, 1.4.1, 1.1.12
> Environment: MVN: 3.5.2
> JDK: 1.8.0.144
>Reporter: Artem Ervits
>Assignee: Artem Ervits
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13
>
> Attachments: HBASE-19140-00.patch
>
>
> steps to reproduce:
> {code}
> bin/hbase-cleanup.sh --cleanAll
> {code}
> {code}
> rmr: DEPRECATED: Please use 'rm -r' instead.
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19137) Nightly test should make junit reports optional rather than attempt archive after reporting.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234853#comment-16234853
 ] 

Hudson commented on HBASE-19137:


FAILURE: Integrated in Jenkins build HBase-2.0 #782 (See 
[https://builds.apache.org/job/HBase-2.0/782/])
HBASE-19137 Nightly test should make junit reports optional rather than 
(busbey: rev 5a941d8f802a00944f1babf021498cdf14935576)
* (edit) dev-support/Jenkinsfile


> Nightly test should make junit reports optional rather than attempt archive 
> after reporting.
> 
>
> Key: HBASE-19137
> URL: https://issues.apache.org/jira/browse/HBASE-19137
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1, 1.1.13
>
> Attachments: HBASE-19137.0.patch
>
>
> HBASE-19030 "nightly runs should attempt to log tests results after 
> archiving" was reopened because its fix was causing failures (reported by our 
> @busbey). This issue is about fixing the new failures (originally to be done 
> against reopened HBASE-19030 but time passed).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19141) [compat 1-2] getClusterStatus always return empty ClusterStatus

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234857#comment-16234857
 ] 

Hudson commented on HBASE-19141:


FAILURE: Integrated in Jenkins build HBase-2.0 #782 (See 
[https://builds.apache.org/job/HBase-2.0/782/])
HBASE-19141 [compat 1-2] getClusterStatus always return empty (chia7712: rev 
261cb8a7e4b3fa2092099e5364e7f846d09ea6fd)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientClusterStatus.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


>  [compat 1-2] getClusterStatus always return empty ClusterStatus
> 
>
> Key: HBASE-19141
> URL: https://issues.apache.org/jira/browse/HBASE-19141
> Project: HBase
>  Issue Type: Sub-task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19141.v0.patch, HBASE-19141.v1.patch
>
>
> We are able to limit the scope to get part of {{ClusterStatus}} in 2.0. 
> However the request sent by 1.x client has no specific scope info to retrieve 
> any information from {{ClusterStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234859#comment-16234859
 ] 

Andrew Purtell commented on HBASE-17852:


So my point is, be careful.

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234852#comment-16234852
 ] 

Hudson commented on HBASE-16417:


FAILURE: Integrated in Jenkins build HBase-2.0 #782 (See 
[https://builds.apache.org/job/HBase-2.0/782/])
HBASE-16417: In-memory MemStore Policy for Flattening and Compactions (eshcar: 
rev 526d2826f5cce4f3b21326657d3c17a651aa6975)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CellArrayImmutableSegment.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CellChunkImmutableSegment.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/VersionedSegmentsList.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/MemoryCompactionPolicy.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
* (add) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/BasicMemStoreCompactionStrategy.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SegmentFactory.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableSegment.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactor.java
* (add) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AdaptiveMemStoreCompactionStrategy.java
* (add) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/EagerMemStoreCompactionStrategy.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStore.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWalAndCompactingMemStoreFlush.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CellSet.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingMemStore.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionPipeline.java
* (add) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestIOFencing.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java


> In-Memory MemStore Policy for Flattening and Compactions
> 
>
> Key: HBASE-16417
> URL: https://issues.apache.org/jira/browse/HBASE-16417
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-16417 - Adaptive Compaction Policy - 20171001.pdf, 
> HBASE-16417 - parameter tuning - 20171001.pdf, HBASE-16417-V01.patch, 
> HBASE-16417-benchmarkresults-20161101.pdf, 
> HBASE-16417-benchmarkresults-20161110.pdf, 
> HBASE-16417-benchmarkresults-20161123.pdf, 
> HBASE-16417-benchmarkresults-20161205.pdf, 
> HBASE-16417-benchmarkresults-20170309.pdf, 
> HBASE-16417-benchmarkresults-20170317.pdf, HBASE-16417.01.patch, 
> HBASE-16417.02.patch, HBASE-16417.03.patch, HBASE-16417.04.patch, 
> HBASE-16417.05.patch, HBASE-16417.06.patch, HBASE-16417.07.patch, 
> HBASE-16417.07.patch, HBASE-16417.08.patch, HBASE-16417.09.patch, 
> HBASE-16417.10.patch, HBASE-16417.11.patch, HBASE-16417.12.patch, 
> HBASE-16417.13.patch, HBASE-16417.14.patch, HBASE-16417.15.patch, 
> HBASE-16417.16.patch, HBASE-16417.17.patch, HBASE-16417.17.patch, 
> HBASE-16417.17.patch, HBASE-16417.17.patch, HBASE-16417.17.patch, 
> HBASE-16417.18.patch, HBASE-16417.branch-2.17.patch
>
>
> This Jira explores the performance of different memstore compaction policies.
> It presents the result of write-only workload evaluation as well as read 
> performance in read-write workloads.
> We investigate several settings of hardware (SSD, HDD), key distribution 
> (Zipf, uniform), with multiple settings of the system, and compare measures 
> like write throughput, read latency, write volume, total gc time, etc.
> The submitted patch sets some system properties at the values yielding 
> optimal performance. In addition we suggest a new Adaptive memstore 
> compaction policy that shows good tradeoffs between write throughput and 
> write volume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19065) HRegion#bulkLoadHFiles() should wait for concurrent Region#flush() to finish

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234855#comment-16234855
 ] 

Hudson commented on HBASE-19065:


FAILURE: Integrated in Jenkins build HBase-2.0 #782 (See 
[https://builds.apache.org/job/HBase-2.0/782/])
HBASE-19065 HRegion#bulkLoadHFiles() should wait for concurrent (tedyu: rev 
f66afa5227ad92cb812fa1947c9dc4fa0d33dbfc)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> HRegion#bulkLoadHFiles() should wait for concurrent Region#flush() to finish
> 
>
> Key: HBASE-19065
> URL: https://issues.apache.org/jira/browse/HBASE-19065
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: 19065.v1.txt, 19065.v2.txt, 19065.v2.txt
>
>
> When I was debugging bulk load failure, I saw the following in region server 
> log:
> {code}
> 2017-10-17 23:05:28,795 DEBUG 
> [B.defaultRpcServer.handler=0,queue=0,port=16020] regionserver.HRegion: NOT 
> flushing memstore for region mx_, 
> f449669a8b0341e4edbd2ebdacc72094f449669a8b0341e4edbd2ebdacc7209420150711,1504909319142.52d496ba39036e0c2cc9522895ad438f.,
>  flushing=true, writesEnabled=true
> 2017-10-17 23:05:28,796 ERROR 
> [B.defaultRpcServer.handler=0,queue=0,port=16020] 
> access.SecureBulkLoadEndpoint: Failed to complete bulk load
> java.io.IOException: Could not bulk load with an assigned sequential ID 
> because the flush didn't run. Reason for not flushing: Not flushing since 
> already flushing
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:5282)
>   at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:292)
>   at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:275)
> {code}
> There was concurrent flush which got misinterpreted by bulkLoadHFiles().
> HRegion#bulkLoadHFiles() should wait for the concurrent flush to complete.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19120) IllegalArgumentException from ZNodeClearer when master shuts down

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234856#comment-16234856
 ] 

Hudson commented on HBASE-19120:


FAILURE: Integrated in Jenkins build HBase-2.0 #782 (See 
[https://builds.apache.org/job/HBase-2.0/782/])
HBASE-19120 IllegalArgumentException from ZNodeClearer when master shuts 
(tedyu: rev 39e8c16fa81624f36042e30242561efccc66589f)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ZNodeClearer.java


> IllegalArgumentException from ZNodeClearer when master shuts down
> -
>
> Key: HBASE-19120
> URL: https://issues.apache.org/jira/browse/HBASE-19120
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: 19120.v1.txt
>
>
> Found the following in master log (build as of commit eee3b0) :
> {code}
> 2017-10-30 15:40:24,383 ERROR [main] util.ServerCommandLine: Failed to run
> java.lang.IllegalArgumentException: Path must start with / character
> at 
> org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:851)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:182)
> at 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1266)
> at 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1258)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:186)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2873)
> {code}
> Looking at ZNodeClearer, it seems that intention was to remove znode under 
> /rs subtree.
> However, the znode name was passed without path.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19100) Missing break in catch block of InterruptedException in HRegion#waitForFlushesAndCompactions

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234854#comment-16234854
 ] 

Hudson commented on HBASE-19100:


FAILURE: Integrated in Jenkins build HBase-2.0 #782 (See 
[https://builds.apache.org/job/HBase-2.0/782/])
HBASE-19100 Missing break in catch block of InterruptedException in (tedyu: rev 
fc4110a7a92cc8cf0ba388c0bbfeac42abe2cee3)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Missing break in catch block of InterruptedException in 
> HRegion#waitForFlushesAndCompactions
> 
>
> Key: HBASE-19100
> URL: https://issues.apache.org/jira/browse/HBASE-19100
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: 19100.v1.txt, 19100.v1.txt
>
>
> Over in HBASE-19072, Anoop reminded me that there was a missing break in 
> HRegion#waitForFlushesAndCompactions as well.
> This issue would fix the defect.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17065) Perform more effective sorting for RPC Handler Tasks

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234858#comment-16234858
 ] 

Hudson commented on HBASE-17065:


FAILURE: Integrated in Jenkins build HBase-2.0 #782 (See 
[https://builds.apache.org/job/HBase-2.0/782/])
HBASE-17065 Perform more effective sorting for RPC Handler Tasks (tedyu: rev 
8a3db6eaaf9a593d9b1171c2bc73e276776b9c43)
* (edit) 
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/common/TaskMonitorTmpl.jamon


> Perform more effective sorting for RPC Handler Tasks
> 
>
> Key: HBASE-17065
> URL: https://issues.apache.org/jira/browse/HBASE-17065
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
>Priority: Minor
>  Labels: rpc, ui
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17065.master.001.patch, Screen Shot 2017-11-01 at 
> 5.38.53 PM.png, rpc-hdlr-tasks.png
>
>
> Under the 'Show All RPC Handler Tasks' tab, the tasks are not sorted 
> according to the duration they have been in the current state (see picture).
> In TaskMonitorTmpl.jamon :
> {code}
> long now = System.currentTimeMillis();
> Collections.reverse(tasks);
> {code}
> The underlying tasks are backed by CircularFifoBuffer, leading to ineffective 
> sorting.
> We should display tasks with the longest duration (in current state) first.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234850#comment-16234850
 ] 

Andrew Purtell commented on HBASE-17852:


bq. Heavy writing during index updates what brings down clusters in Phoenix- 
not cross RS RPC calls.

You are wrong. I am right. I was there, you were not. :-) There were 
implementation bugs and unfortunate/accidental complications that contributed 
to the problem, but we were brought down by cross RS RPC calls. 

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19124) Move HBase-Nightly source artifact creation test from JenkinsFile to a script in dev-support

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234848#comment-16234848
 ] 

Sean Busbey commented on HBASE-19124:
-

updated the branch to skip the unit tests. just gotta get builds.a.o to respond 
to a build request.

> Move HBase-Nightly source artifact creation test from JenkinsFile to a script 
> in dev-support
> 
>
> Key: HBASE-19124
> URL: https://issues.apache.org/jira/browse/HBASE-19124
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-19124.0.patch, HBASE-19124.1.patch
>
>
> ref: 
> https://issues.apache.org/jira/browse/HBASE-19089?focusedCommentId=16221551=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16221551
> Moving that jenkins test to script will make it easier to test changes to 
> hbase-assembly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234847#comment-16234847
 ] 

Vladimir Rodionov commented on HBASE-17852:
---

{quote}
This has brought down big clusters where I work. I'm not saying don't do it, 
but whatever depends on cross-server RPC should learn from the Phoenix example:
{quote}

Heavy writing during index updates what brings down clusters in Phoenix- not 
cross RS RPC calls. This is not the case for backup - we do not do any heavy 
writing at all.  Just record file names.



> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234845#comment-16234845
 ] 

stack commented on HBASE-19152:
---

Dunno. Looks like old knowledge. Probably from before this time when it looks 
like it was added (when it was probably just recast from elsewhere)

tree 23345eae23ca7b51f11c451ccbe8ef49f641e5c1
parent 95f1fe52ed486fea21587739af7a18583d667b31
author Nick Dimiduk  Mon May 11 14:43:58 2015 -0700
committer Nick Dimiduk  Mon May 11 15:26:41 2015 -0700

HBASE-13665 Fix docs and site building on branch-1

make_rc.sh has both on the command-line.  

Up here in 
https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom it 
calls it the 'release-profile'

In maven super pom, there is a release-profile Profile.

Trying w/ apache-release only.

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234841#comment-16234841
 ] 

Andrew Purtell edited comment on HBASE-17852 at 11/1/17 10:16 PM:
--

bq. This is what Phoenix does for indexing AFAIK
This has brought down big clusters where I work. I'm not saying don't do it, 
but whatever depends on cross-server RPC should learn from the Phoenix example:
- Never block on those remote RPCs in critical sections, holding locks, 
especially if you are running in a RPC handler already
- Don't expect the remote resource to be available
- Fail as quickly as possible and retry/clean up later

Let me just assume this stuff is handled, but a walk through of what happens 
when the backup table goes away in different scenarios would be good. 

We are also going to have these same issues when we migrate replication 
tracking state away from ZooKeeper into a system table. At some point we have 
to communicate off server for resilient state tracking, no ways around it. 
Maybe we can build up some kind of library for cross server RPC. 


was (Author: apurtell):
bq. This is what Phoenix does for indexing AFAIK
This has brought down big clusters where I work. I'm not saying don't do it, 
but whatever depends on cross-server RPC should learn from the Phoenix example:
- Never block on those remote RPCs in critical sections, holding locks, 
especially if you are running in a RPC handler already
- Don't expect the remote resource to be available
- Fail as quickly as possible and retry/clean up later
Let me just assume this stuff is handled, but a walk through of what happens 
when the backup table goes away in different scenarios would be good. 

We are also going to have these same issues when we migrate replication 
tracking state away from ZooKeeper into a system table. At some point we have 
to communicate off server for resilient state tracking, no ways around it. 
Maybe we can build up some kind of library for cross server RPC. 

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234841#comment-16234841
 ] 

Andrew Purtell commented on HBASE-17852:


bq. This is what Phoenix does for indexing AFAIK
This has brought down big clusters where I work. I'm not saying don't do it, 
but whatever depends on cross-server RPC should learn from the Phoenix example:
- Never block on those remote RPCs in critical sections, holding locks, 
especially if you are running in a RPC handler already
- Don't expect the remote resource to be available
- Fail as quickly as possible and retry/clean up later
Let me just assume this stuff is handled, but a walk through of what happens 
when the backup table goes away in different scenarios would be good. 

We are also going to have these same issues when we migrate replication 
tracking state away from ZooKeeper into a system table. At some point we have 
to communicate off server for resilient state tracking, no ways around it. 
Maybe we can build up some kind of library for cross server RPC. 

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental 
> backup)
> 
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, 
> HBASE-17852-v3.patch, HBASE-17852-v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19131) Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-19131:
---
Status: Open  (was: Patch Available)

> Add the ClusterStatus hook and cleanup other hooks which can be replaced by 
> ClusterStatus hook
> --
>
> Key: HBASE-19131
> URL: https://issues.apache.org/jira/browse/HBASE-19131
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19131.v0.patch
>
>
> This issue will add the hook for {{ClusterStatus}} and cleanup other hooks 
> which can be replaced by {{ClusterStatus}} hook.
> # {{ListDeadServers}} -  introduced by HBASE-18131



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19124) Move HBase-Nightly source artifact creation test from JenkinsFile to a script in dev-support

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234833#comment-16234833
 ] 

Sean Busbey commented on HBASE-19124:
-

oof. my test branch failed on jdk8 unit tests. :(

> Move HBase-Nightly source artifact creation test from JenkinsFile to a script 
> in dev-support
> 
>
> Key: HBASE-19124
> URL: https://issues.apache.org/jira/browse/HBASE-19124
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Appy
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-19124.0.patch, HBASE-19124.1.patch
>
>
> ref: 
> https://issues.apache.org/jira/browse/HBASE-19089?focusedCommentId=16221551=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16221551
> Moving that jenkins test to script will make it easier to test changes to 
> hbase-assembly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19140) hbase-cleanup.sh uses deprecated call to remove files in hdfs

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234817#comment-16234817
 ] 

Hudson commented on HBASE-19140:


FAILURE: Integrated in Jenkins build HBase-1.4 #984 (See 
[https://builds.apache.org/job/HBase-1.4/984/])
HBASE-19140 hbase-cleanup.sh uses deprecated call to remove files in 
(jan.hentschel: rev 9780ea872988000bcde6d4323c400511307cab87)
* (edit) bin/hbase-cleanup.sh


> hbase-cleanup.sh uses deprecated call to remove files in hdfs
> -
>
> Key: HBASE-19140
> URL: https://issues.apache.org/jira/browse/HBASE-19140
> Project: HBase
>  Issue Type: Improvement
>  Components: scripts
>Affects Versions: 2.0.0, 1.3.1, 1.2.6, 1.4.1, 1.1.12
> Environment: MVN: 3.5.2
> JDK: 1.8.0.144
>Reporter: Artem Ervits
>Assignee: Artem Ervits
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13
>
> Attachments: HBASE-19140-00.patch
>
>
> steps to reproduce:
> {code}
> bin/hbase-cleanup.sh --cleanAll
> {code}
> {code}
> rmr: DEPRECATED: Please use 'rm -r' instead.
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18925) Need updated mockito for using java optional

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-18925:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Need updated mockito for using java optional
> 
>
> Key: HBASE-18925
> URL: https://issues.apache.org/jira/browse/HBASE-18925
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18925.master.001.patch, 
> HBASE-18925.master.002.patch, HBASE-18925.master.002.patch, 
> HBASE-18925.master.003.patch, HBASE-18925.master.004.patch, 
> HBASE-18925.master.005.patch, HBASE-18925.master.006.patch, 
> HBASE-18925.master.007.patch, HBASE-18925.master.008.patch, 
> HBASE-18925.master.009.patch
>
>
> Came up when i was trying to test HBASE-18878.
> It kept failing because mock of RpcCall returned null where return type was 
> Optional.
> Instead, we want it to return Optional.empty(). 
> New mockito versions support this (and other java8 things) - 
> https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2
> We use mockito-all which was last released in Dec2014. However, mockito-core 
> has had more than 50 releases after that 
> (https://mvnrepository.com/artifact/org.mockito/mockito-core). 
> We need to change our deps from mockito-all to mockito-core.
> However that comes with fair breakages, so this is not a simple task of 
> changing pom files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19144) [RSgroups] Retry assignments in FAILED_OPEN state when servers (re)join the cluster

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234794#comment-16234794
 ] 

Hadoop QA commented on HBASE-19144:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 27m  
4s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
37s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
13s{color} | {color:green} branch-1 passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
14s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
52s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
32s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
17s{color} | {color:green} branch-1 passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
30s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 46s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3. 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed with JDK v1.8.0_141 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
13s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m  
1s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 71m  8s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:4bf71c3 |
| JIRA Issue | HBASE-19144 |
| JIRA Patch URL | 

[jira] [Updated] (HBASE-17365) Data at rest encryption support (cloud backups)

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17365:
--
Fix Version/s: 2.1.0

> Data at rest encryption support (cloud backups)
> ---
>
> Key: HBASE-17365
> URL: https://issues.apache.org/jira/browse/HBASE-17365
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.1.0
>
>
> Support encryption of data at rest in cloud backup destinations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19104) Add filtering during restore in HBase backups.

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-19104:
--
Fix Version/s: 2.1.0

> Add filtering during restore in HBase backups.
> --
>
> Key: HBASE-19104
> URL: https://issues.apache.org/jira/browse/HBASE-19104
> Project: HBase
>  Issue Type: New Feature
>  Components: backup
>Reporter: Amit Kabra
>Priority: Major
> Fix For: 2.1.0
>
>
> When we deal with large amount of data, it would be great , if we can do data 
> restore from backups based on tenant , based on time range , etc , so that if 
> finishes faster and we restore only what's required.
> Currently restore take backup id as input and restore all the data will that 
> backup id time stamp. We may not need to restore all data in a given backup 
> id.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17364) Point in time (PIT) restore

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17364:
--
Fix Version/s: 2.1.0

> Point in time (PIT) restore
> ---
>
> Key: HBASE-17364
> URL: https://issues.apache.org/jira/browse/HBASE-17364
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.1.0
>
>
> For applications where HBase version is a real timestamp we can implement 
> PIT, including *restore current state* option.
> *Restore current state* command will restore table from the most recent 
> backup and then continue restore operation by re-playing edits from WAL files 
> which are still around.
> Another option : restore time range of a table
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17363) Snapshot-less backup

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17363:
--
Fix Version/s: 2.1.0

> Snapshot-less backup
> 
>
> Key: HBASE-17363
> URL: https://issues.apache.org/jira/browse/HBASE-17363
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.1.0
>
>
> Currently, during full table backup we perform table snapshot. Too long and 
> not  very robust. This is brainstorming JIRA, how can we improve snapshots or 
> avoid them completely.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19127) Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234783#comment-16234783
 ] 

Hadoop QA commented on HBASE-19127:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
 4s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  1m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
7s{color} | {color:red} hbase-server: The patch generated 1 new + 180 unchanged 
- 9 fixed = 181 total (was 189) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
48s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
56m 52s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}  
1m 20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
39s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}120m 20s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
53s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}202m 22s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hbase.master.normalizer.TestSimpleRegionNormalizerOnCluster |
|   | hadoop.hbase.master.TestAssignmentListener |
|   | hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
|   | hadoop.hbase.namespace.TestNamespaceAuditor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | 

[jira] [Updated] (HBASE-17362) HBase Backup/Restore Phase 4

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17362:
--
Fix Version/s: 2.1.0

> HBase Backup/Restore Phase 4
> 
>
> Key: HBASE-17362
> URL: https://issues.apache.org/jira/browse/HBASE-17362
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 2.1.0
>
>
> The umbrella JIRA for next features of the backup/restore module



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18925) Need updated mockito for using java optional

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234771#comment-16234771
 ] 

Appy commented on HBASE-18925:
--

Fixed the 3 checkstyle errors. (This patch is anyways fixing a lot of them, net 
-53 errors)

Last precommit was +1 on tests, so i suspected this was just flakyness.
Ran findHangingTests.py script on 
[patch-unit-root.txt|https://builds.apache.org/job/PreCommit-HBASE-Build/9547/artifact/patchprocess/patch-unit-root.txt]
 and found the culprit tests.
{noformat}
Found 2 failed tests of which 0 timed out:
master.TestRollingRestart
client.TestAdmin2
{noformat}
Unlike previous cases when the tests reported by the script were failing 
locally and consistently (and the train of patches was to fix them), these two 
passed locally 3 times in row.

Committing to master and branch-2
Thanks again for the review [~stack].


> Need updated mockito for using java optional
> 
>
> Key: HBASE-18925
> URL: https://issues.apache.org/jira/browse/HBASE-18925
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18925.master.001.patch, 
> HBASE-18925.master.002.patch, HBASE-18925.master.002.patch, 
> HBASE-18925.master.003.patch, HBASE-18925.master.004.patch, 
> HBASE-18925.master.005.patch, HBASE-18925.master.006.patch, 
> HBASE-18925.master.007.patch, HBASE-18925.master.008.patch, 
> HBASE-18925.master.009.patch
>
>
> Came up when i was trying to test HBASE-18878.
> It kept failing because mock of RpcCall returned null where return type was 
> Optional.
> Instead, we want it to return Optional.empty(). 
> New mockito versions support this (and other java8 things) - 
> https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2
> We use mockito-all which was last released in Dec2014. However, mockito-core 
> has had more than 50 releases after that 
> (https://mvnrepository.com/artifact/org.mockito/mockito-core). 
> We need to change our deps from mockito-all to mockito-core.
> However that comes with fair breakages, so this is not a simple task of 
> changing pom files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19127) Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode

[Jerry He (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry He updated HBASE-19127:
-
Issue Type: Sub-task  (was: Improvement)
Parent: HBASE-19126

> Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in 
> RegionStatesNode
> -
>
> Key: HBASE-19127
> URL: https://issues.apache.org/jira/browse/HBASE-19127
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Major
> Attachments: state.patch
>
>
> In current code, we did not set above states to a region node at all, but we 
> still have statements like below to check if node have above states.
> {code}
> else if (!regionNode.isInState(State.CLOSING, State.SPLITTING)) {
> 
> }
> {code}
> We need to set above states in a correct place.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19120) IllegalArgumentException from ZNodeClearer when master shuts down

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234754#comment-16234754
 ] 

Hudson commented on HBASE-19120:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3982 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3982/])
HBASE-19120 IllegalArgumentException from ZNodeClearer when master shuts 
(tedyu: rev f10f4eb12133f534d47788121c4cfafd898872f0)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ZNodeClearer.java


> IllegalArgumentException from ZNodeClearer when master shuts down
> -
>
> Key: HBASE-19120
> URL: https://issues.apache.org/jira/browse/HBASE-19120
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-1
>
> Attachments: 19120.v1.txt
>
>
> Found the following in master log (build as of commit eee3b0) :
> {code}
> 2017-10-30 15:40:24,383 ERROR [main] util.ServerCommandLine: Failed to run
> java.lang.IllegalArgumentException: Path must start with / character
> at 
> org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:851)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:182)
> at 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1266)
> at 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1258)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:186)
> at 
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2873)
> {code}
> Looking at ZNodeClearer, it seems that intention was to remove znode under 
> /rs subtree.
> However, the znode name was passed without path.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17065) Perform more effective sorting for RPC Handler Tasks

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234757#comment-16234757
 ] 

Hudson commented on HBASE-17065:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3982 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3982/])
HBASE-17065 Perform more effective sorting for RPC Handler Tasks (tedyu: rev 
5e16e23fadc6cbd9474f6924be1a7ce538782bbe)
* (edit) 
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/common/TaskMonitorTmpl.jamon


> Perform more effective sorting for RPC Handler Tasks
> 
>
> Key: HBASE-17065
> URL: https://issues.apache.org/jira/browse/HBASE-17065
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
>Priority: Minor
>  Labels: rpc, ui
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-17065.master.001.patch, Screen Shot 2017-11-01 at 
> 5.38.53 PM.png, rpc-hdlr-tasks.png
>
>
> Under the 'Show All RPC Handler Tasks' tab, the tasks are not sorted 
> according to the duration they have been in the current state (see picture).
> In TaskMonitorTmpl.jamon :
> {code}
> long now = System.currentTimeMillis();
> Collections.reverse(tasks);
> {code}
> The underlying tasks are backed by CircularFifoBuffer, leading to ineffective 
> sorting.
> We should display tasks with the longest duration (in current state) first.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19141) [compat 1-2] getClusterStatus always return empty ClusterStatus

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234755#comment-16234755
 ] 

Hudson commented on HBASE-19141:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3982 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3982/])
HBASE-19141 [compat 1-2] getClusterStatus always return empty (chia7712: rev 
2d0da40fff2d3b4873c610015e948ef9167ac202)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientClusterStatus.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


>  [compat 1-2] getClusterStatus always return empty ClusterStatus
> 
>
> Key: HBASE-19141
> URL: https://issues.apache.org/jira/browse/HBASE-19141
> Project: HBase
>  Issue Type: Sub-task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19141.v0.patch, HBASE-19141.v1.patch
>
>
> We are able to limit the scope to get part of {{ClusterStatus}} in 2.0. 
> However the request sent by 1.x client has no specific scope info to retrieve 
> any information from {{ClusterStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19137) Nightly test should make junit reports optional rather than attempt archive after reporting.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234753#comment-16234753
 ] 

Hudson commented on HBASE-19137:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3982 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3982/])
HBASE-19137 Nightly test should make junit reports optional rather than 
(busbey: rev 91273e7b0e91ccb1acbc596a7f282acc76ce8a21)
* (edit) dev-support/Jenkinsfile


> Nightly test should make junit reports optional rather than attempt archive 
> after reporting.
> 
>
> Key: HBASE-19137
> URL: https://issues.apache.org/jira/browse/HBASE-19137
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1, 1.1.13
>
> Attachments: HBASE-19137.0.patch
>
>
> HBASE-19030 "nightly runs should attempt to log tests results after 
> archiving" was reopened because its fix was causing failures (reported by our 
> @busbey). This issue is about fixing the new failures (originally to be done 
> against reopened HBASE-19030 but time passed).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-12091) Optionally ignore edits for dropped tables for replication.

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234744#comment-16234744
 ] 

Lars Hofhansl edited comment on HBASE-12091 at 11/1/17 9:00 PM:


This is turning into a whole nightmare now. There's no way that I can find to 
transmit unadulterated data from the sink the source (or server to any client).

I changed the sink to just throw a TableNotFoundException, but 
IPCUtil.createRemoteException adds the trace and classname to the message, so 
the client would have parse the text to get the table, which is fragile.



was (Author: lhofhansl):
This is turning into a whole nightmare now. There's no way that I can find to 
transmit unadulterated data from the sink the source (or server to any client).

I changed the sink to just throw a TableNotFoundException, but 
IPCUtil.createRemoteException adds the trace to the message, so the client 
would have parse the text to get the table, which is fragile.


> Optionally ignore edits for dropped tables for replication.
> ---
>
> Key: HBASE-12091
> URL: https://issues.apache.org/jira/browse/HBASE-12091
> Project: HBase
>  Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: 12091-v2-branch-1.txt, 12091-v3-branch-1.txt, 
> 12091-v4-branch-1.txt, 12091-v5-branch-1.txt, 12091.txt
>
>
> We just ran into a scenario where we dropped a table from both the source and 
> the sink, but the source still has outstanding edits that now it could not 
> get rid of. Now all replication is backed up behind these unreplicatable 
> edits.
> We should have an option to ignore edits for tables dropped at the source.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-12091) Optionally ignore edits for dropped tables for replication.

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234744#comment-16234744
 ] 

Lars Hofhansl commented on HBASE-12091:
---

This is turning into a whole nightmare now. There's no way that I can find to 
transmit unadulterated data from the sink the source (or server to any client).

I changed the sink to just throw a TableNotFoundException, but 
IPCUtil.createRemoteException adds the trace to the message, so the client 
would have parse the text to get the table, which is fragile.


> Optionally ignore edits for dropped tables for replication.
> ---
>
> Key: HBASE-12091
> URL: https://issues.apache.org/jira/browse/HBASE-12091
> Project: HBase
>  Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: 12091-v2-branch-1.txt, 12091-v3-branch-1.txt, 
> 12091-v4-branch-1.txt, 12091-v5-branch-1.txt, 12091.txt
>
>
> We just ran into a scenario where we dropped a table from both the source and 
> the sink, but the source still has outstanding edits that now it could not 
> get rid of. Now all replication is backed up behind these unreplicatable 
> edits.
> We should have an option to ignore edits for tables dropped at the source.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234739#comment-16234739
 ] 

Sean Busbey commented on HBASE-19152:
-

{quote}
Next, deploy HBase to the Apache Maven repository, using
735 the `apache-release` profile instead of the `release` profile
736 when running the `mvn deploy` command.{quote}

Are we sure about this bit? I thought both profiles were needed to make sure 
the shaded artifacts get built correctly?

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19152:
--
Status: Patch Available  (was: Open)

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234736#comment-16234736
 ] 

stack commented on HBASE-19152:
---

.004 includes fix for above tag mess [~busbey] found and suggestions by 
[~apurtell] out on RB

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19152:
--
Attachment: HBASE-19152.master.004.patch

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-19109) Backup should provide and option to disable backup if full backups have started

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234734#comment-16234734
 ] 

Vladimir Rodionov edited comment on HBASE-19109 at 11/1/17 8:52 PM:


We can run only one backup session at a time. The synchronization/locking  is 
implemented by backup  itself. Not sure, I understand the rationals behind this 
request, [~vishk]


was (Author: vrodionov):
We can run only one backup session at a time. The synchronization/locking  is 
implemented by backup  itself. Not sure, I understand the rational behind this 
request, [~vishk]

> Backup should provide and option to disable backup if full backups have 
> started
> ---
>
> Key: HBASE-19109
> URL: https://issues.apache.org/jira/browse/HBASE-19109
> Project: HBase
>  Issue Type: New Feature
>  Components: backup
> Environment: Ability to disable backups for a particular set of 
> tables if backups are started and property to enable it backup
> Use disable / enable flag at table schema level , etc
> These can be probable changes
> * A new option/command line needs to added to enable/disable backups
> * A new property and field to track list of disabled backup tables
> * backupInfo.getTables() should have a logic to filter such disabled tables
> * Hbase:backup table needs to have a flag for enable and disable backups
> * During all table backups, backups for disabled table should not happen
> * During full/incremental backup backups should error out for disabled table 
> if it is part of the list
> * BackupInfo Table should have a list of disabled backups. 
> * If full backups is taken and backups are disabled then incremental backups 
> should not be performed 
> * If backups are enabled again, means full backups needs to be executed 
> before starting incremental backups
> * Restore of disabled backups should not be stopped 
>Reporter: Vishal Khandelwal
>Priority: Major
>  Labels: bac
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19152:
--
Attachment: HBASE-19152.master.003.patch

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19109) Backup should provide and option to disable backup if full backups have started

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234734#comment-16234734
 ] 

Vladimir Rodionov commented on HBASE-19109:
---

We can run only one backup session at a time. The synchronization/locking  is 
implemented by backup  itself. Not sure, I understand the rational behind this 
request, [~vishk]

> Backup should provide and option to disable backup if full backups have 
> started
> ---
>
> Key: HBASE-19109
> URL: https://issues.apache.org/jira/browse/HBASE-19109
> Project: HBase
>  Issue Type: New Feature
>  Components: backup
> Environment: Ability to disable backups for a particular set of 
> tables if backups are started and property to enable it backup
> Use disable / enable flag at table schema level , etc
> These can be probable changes
> * A new option/command line needs to added to enable/disable backups
> * A new property and field to track list of disabled backup tables
> * backupInfo.getTables() should have a logic to filter such disabled tables
> * Hbase:backup table needs to have a flag for enable and disable backups
> * During all table backups, backups for disabled table should not happen
> * During full/incremental backup backups should error out for disabled table 
> if it is part of the list
> * BackupInfo Table should have a list of disabled backups. 
> * If full backups is taken and backups are disabled then incremental backups 
> should not be performed 
> * If backups are enabled again, means full backups needs to be executed 
> before starting incremental backups
> * Restore of disabled backups should not be stopped 
>Reporter: Vishal Khandelwal
>Priority: Major
>  Labels: bac
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19108) Backups should understand the frequency of full and incremental backup

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234729#comment-16234729
 ] 

Vladimir Rodionov commented on HBASE-19108:
---

Why not cron or Oozie?

> Backups should understand the frequency of full and incremental backup
> --
>
> Key: HBASE-19108
> URL: https://issues.apache.org/jira/browse/HBASE-19108
> Project: HBase
>  Issue Type: New Feature
>  Components: backup
>Reporter: Vishal Khandelwal
>Priority: Major
>  Labels: backup
>
> Backup scheduler which figures out tables for which backups are pending and 
> executes all required backups.
> In case of failures, this automates required (full / incremental) backup 
> execution based on the set frequency. Thus the support to set frequency and 
> expire old backups should be added. This will reduce admin overhead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19107) Backup should resume from last failed state

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-19107:
--
Fix Version/s: 2.1.0

> Backup should resume from last failed state
> ---
>
> Key: HBASE-19107
> URL: https://issues.apache.org/jira/browse/HBASE-19107
> Project: HBase
>  Issue Type: New Feature
>  Components: backup
>Reporter: Vishal Khandelwal
>Priority: Major
>  Labels: backup
> Fix For: 2.1.0
>
>
> Consider that incremental backup is happenng for 10 tables for a set of 100 
> WALs. if backup @ 10th fails, then on net iteration it would be triggered 
> again from 100 WALs + new WALs all tables
> This is a issue because resource and time used to do he backup of 9 table & 
> 100 WALs are wasted
> Rather it should have functionality to resume from previous state complete 
> that backup and then do new set of WALs. This will make it more performant



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19107) Backup should resume from last failed state

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234725#comment-16234725
 ] 

Vladimir Rodionov commented on HBASE-19107:
---

Moving this to 2.1.0 release

> Backup should resume from last failed state
> ---
>
> Key: HBASE-19107
> URL: https://issues.apache.org/jira/browse/HBASE-19107
> Project: HBase
>  Issue Type: New Feature
>  Components: backup
>Reporter: Vishal Khandelwal
>Priority: Major
>  Labels: backup
> Fix For: 2.1.0
>
>
> Consider that incremental backup is happenng for 10 tables for a set of 100 
> WALs. if backup @ 10th fails, then on net iteration it would be triggered 
> again from 100 WALs + new WALs all tables
> This is a issue because resource and time used to do he backup of 9 table & 
> 100 WALs are wasted
> Rather it should have functionality to resume from previous state complete 
> that backup and then do new set of WALs. This will make it more performant



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234723#comment-16234723
 ] 

Sean Busbey commented on HBASE-19152:
-

{code}

651 671 
672 $ git push origin 2.0.0-alpha4
673 
674 +
{code}

this should either refer to "rel/2.0.0-alpha4" or to "2.0.0-alpha4-RC0" since 
those are the two tags that have been created prior to this point in the guide.

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)