[jira] [Commented] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690415#comment-16690415 ] Allan Yang commented on HBASE-21480: Just reviewed the V2 patch, +1 for it. It is a nice simple patch which can solve the issue here. > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480-v1.patch, HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21493) Release 1.2.9
[ https://issues.apache.org/jira/browse/HBASE-21493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690408#comment-16690408 ] Hudson commented on HBASE-21493: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1183 (See [https://builds.apache.org/job/HBase-1.2-IT/1183/]) HBASE-21493 update version for 1.2.9 RCs (busbey: rev 2bcedcafbd01e4e8baf829c6366800bd3044006f) * (edit) hbase-server/pom.xml * (edit) hbase-it/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-shell/pom.xml * (edit) hbase-testing-util/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-shaded/hbase-shaded-check-invariants/pom.xml * (edit) hbase-protocol/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-procedure/pom.xml * (edit) pom.xml * (edit) hbase-annotations/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-thrift/pom.xml HBASE-21493 update CHANGES.txt for 1.2.9 RC0 (busbey: rev fd0d55b1e5ef54eb9bf60cce1f0a8e4c1da073ef) * (edit) CHANGES.txt > Release 1.2.9 > - > > Key: HBASE-21493 > URL: https://issues.apache.org/jira/browse/HBASE-21493 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.9 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.9 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21480: -- Attachment: HBASE-21480-v1.patch > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480-v1.patch, HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21493) Release 1.2.9
[ https://issues.apache.org/jira/browse/HBASE-21493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690405#comment-16690405 ] Sean Busbey commented on HBASE-21493: - * updated ref guide on branch-1.2 * in jira but not in git: none * in git but not in jira: HBASE-21302 for marking 1.2.8 as done * updated CHANGES * update version # > Release 1.2.9 > - > > Key: HBASE-21493 > URL: https://issues.apache.org/jira/browse/HBASE-21493 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.9 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.9 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21493) Release 1.2.9
[ https://issues.apache.org/jira/browse/HBASE-21493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690404#comment-16690404 ] Hudson commented on HBASE-21493: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1182 (See [https://builds.apache.org/job/HBase-1.2-IT/1182/]) HBASE-21493 Update ref guide for branch-1.2 related changes (busbey: rev d1ac42b22cb5b6b9e7ffedf4dea6763394396304) * (edit) src/main/asciidoc/_chapters/security.adoc * (edit) src/main/asciidoc/_chapters/architecture.adoc * (edit) src/main/asciidoc/_chapters/ops_mgt.adoc * (edit) src/main/asciidoc/_chapters/troubleshooting.adoc * (edit) src/main/asciidoc/_chapters/preface.adoc > Release 1.2.9 > - > > Key: HBASE-21493 > URL: https://issues.apache.org/jira/browse/HBASE-21493 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.9 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.9 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690397#comment-16690397 ] Allan Yang commented on HBASE-21480: OK, I see, it is possible... > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690395#comment-16690395 ] Duo Zhang commented on HBASE-21480: --- If your timeout is big enough, i.e, more than 10 minutes, then there will be no problem. The snapshot operation will fail and finally the regions will become online. It will not make the cluster hang there for ever, just about 10 minutes... > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690394#comment-16690394
]
Duo Zhang commented on HBASE-21489:
---
+1.
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21489.master.001.patch
>
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
> test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:51:in `block in
> test_Revoke_should_rid_access_rights_appropriately'
> 48: create_test_table(@test_name)
> 49: table = table(@test_name)
> 50: user =
> org.apache.hadoop.hbase.security.User.getCurrent().getName();
> => 51: assert_equal(1,
> security_admin.user_permission(@test_name).length)
> 52: security_admin.revoke(user, @test_name)
> 53: assert_equal(0,
> security_admin.user_permission(@test_name).length)
> 54: end
> ===
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690392#comment-16690392
]
Hadoop QA commented on HBASE-21489:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
5s{color} | {color:red} The patch generated 14 new + 44 unchanged - 10 fixed =
58 total (was 54) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 2s{color} | {color:orange} The patch generated 4 new + 42 unchanged - 3
fixed = 46 total (was 45) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
50s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21489 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948582/HBASE-21489.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit rubocop
ruby_lint |
| uname | Linux bfcfe38f1083 4.4.0-131-generic #157~14.04.1-Ubuntu SMP Fri Jul
13 08:53:17 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 825e14b68e |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| rubocop | v0.60.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/15055/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/15055/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15055/testReport/ |
| Max. process+thread count | 2566 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15055/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21489.master.001.patch
>
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in
[jira] [Comment Edited] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690391#comment-16690391
]
Reid Chan edited comment on HBASE-21481 at 11/17/18 5:45 AM:
-
* Add a check on target user/group whether he is a superuser or a user in
supergroup or neither.
* Add a new test:
{{TestRpcAccessChecks#testGrantRevokeDeniedOnSuperUsersGroups}} which includes
group permission test cases.
* Change visibility of {{TestingGroups}} to _public_ for test, otherwise, it
can't be initialized in AccessChecker.
was (Author: reidchan):
* Add a check on target user/group where he is a superuser or a user in
supergroup.
* Add a new test:
{{TestRpcAccessChecks#testGrantRevokeDeniedOnSuperUsersGroups}} which includes
group permission test cases.
* Change visibility of {{TestingGroups}} to _public_ for test, otherwise, it
can't be initialized in AccessChecker.
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690391#comment-16690391
]
Reid Chan commented on HBASE-21481:
---
* Add a check on target user/group where he is a superuser or a user in
supergroup.
* Add a new test:
{{TestRpcAccessChecks#testGrantRevokeDeniedOnSuperUsersGroups}} which includes
group permission test cases.
* Change visibility of {{TestingGroups}} to _public_ for test, otherwise, it
can't be initialized in AccessChecker.
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21493) Release 1.2.9
[ https://issues.apache.org/jira/browse/HBASE-21493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690390#comment-16690390 ] Sean Busbey commented on HBASE-21493: - * booted out 1.2.9 things that show no progress * looked at recent fixes in branches 1 that weren't in branch-1.2. backported HBASE-21357 > Release 1.2.9 > - > > Key: HBASE-21493 > URL: https://issues.apache.org/jira/browse/HBASE-21493 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.9 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.9 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-21493) Release 1.2.9
[ https://issues.apache.org/jira/browse/HBASE-21493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-21493 started by Sean Busbey. --- > Release 1.2.9 > - > > Key: HBASE-21493 > URL: https://issues.apache.org/jira/browse/HBASE-21493 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.2.9 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.2.9 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21493) Release 1.2.9
Sean Busbey created HBASE-21493: --- Summary: Release 1.2.9 Key: HBASE-21493 URL: https://issues.apache.org/jira/browse/HBASE-21493 Project: HBase Issue Type: Task Components: community Affects Versions: 1.2.9 Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 1.2.9 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21357) RS should abort if OOM in Reader thread
[
https://issues.apache.org/jira/browse/HBASE-21357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690389#comment-16690389
]
Hudson commented on HBASE-21357:
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1181 (See
[https://builds.apache.org/job/HBase-1.2-IT/1181/])
HBASE-21357 RS should abort if OOM in Reader thread (busbey: rev
b8a7edab8be70ee860cd4242b397980d066c1ae7)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> RS should abort if OOM in Reader thread
> ---
>
> Key: HBASE-21357
> URL: https://issues.apache.org/jira/browse/HBASE-21357
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.8
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.9, 1.2.9
>
> Attachments: HBASE-21357.branch-1.001.patch,
> HBASE-21357.branch-1.001.patch
>
>
> It is a bit strange, we will abort the RS if OOM in Listener thread,
> Responder thread and in CallRunner thread, only not in Reader thread...
> We should abort RS if OOM happens in Reader thread, too. If not, the reader
> thread exists because of OOM, and the selector closes. Later connection
> select to this reader will be ignored
> {code}
> try {
> if (key.isValid()) {
> if (key.isAcceptable())
> doAccept(key);
> }
> } catch (IOException ignored) {
> if (LOG.isTraceEnabled()) LOG.trace("ignored", ignored);
> }
> {code}
> Leaving the client (or Master and other RS)'s call wait until SocketTimeout.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21481:
--
Attachment: HBASE-21481.master.001.patch
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690388#comment-16690388 ] Allan Yang commented on HBASE-21480: The UT also fails in branch-2.0 and branch-2.1. It is very strange, snapshot also randomly executed during ITBILL, but I never encountered this case. > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690386#comment-16690386
]
Reid Chan commented on HBASE-21489:
---
Test passed on local,
{code}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.hbase.client.TestShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 183.25 s
- in org.apache.hadoop.hbase.client.TestShell
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
{code}
Let's wait QA.
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-21489.master.001.patch
>
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
> test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:51:in `block in
> test_Revoke_should_rid_access_rights_appropriately'
> 48: create_test_table(@test_name)
> 49: table = table(@test_name)
> 50: user =
> org.apache.hadoop.hbase.security.User.getCurrent().getName();
> => 51: assert_equal(1,
> security_admin.user_permission(@test_name).length)
> 52: security_admin.revoke(user, @test_name)
> 53: assert_equal(0,
> security_admin.user_permission(@test_name).length)
> 54: end
> ===
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21275) Thrift Server (branch 1 fix) -> Disable TRACE HTTP method for thrift http server (branch 1 only)
[ https://issues.apache.org/jira/browse/HBASE-21275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21275: Fix Version/s: 1.5.0 > Thrift Server (branch 1 fix) -> Disable TRACE HTTP method for thrift http > server (branch 1 only) > > > Key: HBASE-21275 > URL: https://issues.apache.org/jira/browse/HBASE-21275 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Minor > Fix For: 1.5.0, 1.4.9 > > Attachments: HBASE-21275-branch-1.001.patch, > HBASE-21275-branch-1.2.001.patch, HBASE-21275-branch-1.2.002.patch, > HBASE-21275-branch-1.2.003.patch, HBASE-21275-branch-1.2.003.patch, > HBASE-21275-branch-1.4.001.patch > > > There's been a reasonable number of users running thrift http server on hbase > 1.x suffering with security audit tests pointing thrift server allows TRACE > requests. > After doing some search, I can see HBASE-20406 added restrictions for > TRACE/OPTIONS method when Thrift is running over http, but it relies on many > other commits applied to thrift http server. This patch was later reverted > from master. Then again later, HBASE-20004 had made TRACE/OPTIONS > configurable via "*hbase.thrift.http.allow.options.method*" property, with > both methods being disabled by default. This also seems to rely on many > changes applied to thrift http server, and a branch 1 compatible patch does > not seem feasible. > A solution for branch 1 is pretty simple though, am proposing a patch that > simply uses *WebAppContext*, instead of *Context*, as the context for the > *HttpServer* instance. *WebAppContext* will already restrict TRACE methods by > default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21481:
--
Attachment: (was: HBASE-21481.master.001.patch)
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21481:
--
Attachment: HBASE-21481.master.001.patch
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21481:
--
Status: Patch Available (was: Open)
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21489:
--
Fix Version/s: 2.2.0
3.0.0
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21489.master.001.patch
>
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
> test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:51:in `block in
> test_Revoke_should_rid_access_rights_appropriately'
> 48: create_test_table(@test_name)
> 49: table = table(@test_name)
> 50: user =
> org.apache.hadoop.hbase.security.User.getCurrent().getName();
> => 51: assert_equal(1,
> security_admin.user_permission(@test_name).length)
> 52: security_admin.revoke(user, @test_name)
> 53: assert_equal(0,
> security_admin.user_permission(@test_name).length)
> 54: end
> ===
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21357) RS should abort if OOM in Reader thread
[
https://issues.apache.org/jira/browse/HBASE-21357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21357:
Fix Version/s: 1.2.9
> RS should abort if OOM in Reader thread
> ---
>
> Key: HBASE-21357
> URL: https://issues.apache.org/jira/browse/HBASE-21357
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.8
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.9, 1.2.9
>
> Attachments: HBASE-21357.branch-1.001.patch,
> HBASE-21357.branch-1.001.patch
>
>
> It is a bit strange, we will abort the RS if OOM in Listener thread,
> Responder thread and in CallRunner thread, only not in Reader thread...
> We should abort RS if OOM happens in Reader thread, too. If not, the reader
> thread exists because of OOM, and the selector closes. Later connection
> select to this reader will be ignored
> {code}
> try {
> if (key.isValid()) {
> if (key.isAcceptable())
> doAccept(key);
> }
> } catch (IOException ignored) {
> if (LOG.isTraceEnabled()) LOG.trace("ignored", ignored);
> }
> {code}
> Leaving the client (or Master and other RS)'s call wait until SocketTimeout.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21489:
--
Affects Version/s: 2.2.0
3.0.0
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-21489.master.001.patch
>
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
> test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:51:in `block in
> test_Revoke_should_rid_access_rights_appropriately'
> 48: create_test_table(@test_name)
> 49: table = table(@test_name)
> 50: user =
> org.apache.hadoop.hbase.security.User.getCurrent().getName();
> => 51: assert_equal(1,
> security_admin.user_permission(@test_name).length)
> 52: security_admin.revoke(user, @test_name)
> 53: assert_equal(0,
> security_admin.user_permission(@test_name).length)
> 54: end
> ===
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21489:
--
Attachment: HBASE-21489.master.001.patch
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-21489.master.001.patch
>
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
> test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:51:in `block in
> test_Revoke_should_rid_access_rights_appropriately'
> 48: create_test_table(@test_name)
> 49: table = table(@test_name)
> 50: user =
> org.apache.hadoop.hbase.security.User.getCurrent().getName();
> => 51: assert_equal(1,
> security_admin.user_permission(@test_name).length)
> 52: security_admin.revoke(user, @test_name)
> 53: assert_equal(0,
> security_admin.user_permission(@test_name).length)
> 54: end
> ===
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-21489:
--
Status: Patch Available (was: Open)
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-21489.master.001.patch
>
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
> test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:51:in `block in
> test_Revoke_should_rid_access_rights_appropriately'
> 48: create_test_table(@test_name)
> 49: table = table(@test_name)
> 50: user =
> org.apache.hadoop.hbase.security.User.getCurrent().getName();
> => 51: assert_equal(1,
> security_admin.user_permission(@test_name).length)
> 52: security_admin.revoke(user, @test_name)
> 53: assert_equal(0,
> security_admin.user_permission(@test_name).length)
> 54: end
> ===
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21373) Backport to branch-1, "HBASE-21338 [balancer] If balancer is an ill-fit for cluster size, it gives little indication"
[ https://issues.apache.org/jira/browse/HBASE-21373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21373: Issue Type: Improvement (was: Bug) > Backport to branch-1, "HBASE-21338 [balancer] If balancer is an ill-fit for > cluster size, it gives little indication" > - > > Key: HBASE-21373 > URL: https://issues.apache.org/jira/browse/HBASE-21373 > Project: HBase > Issue Type: Improvement > Components: Operability >Reporter: stack >Assignee: Xu Cang >Priority: Major > Fix For: 1.5.0, 1.3.3, 1.4.9 > > Attachments: HBASE-21373.branch-1.001.patch, > HBASE-21373.branch-1.002.patch > > > Issue to backport to branch-1. Hope you don't mind my assigning it to you Xu > Cang. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21357) RS should abort if OOM in Reader thread
[
https://issues.apache.org/jira/browse/HBASE-21357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21357:
Priority: Critical (was: Major)
> RS should abort if OOM in Reader thread
> ---
>
> Key: HBASE-21357
> URL: https://issues.apache.org/jira/browse/HBASE-21357
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.8
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Critical
> Fix For: 1.5.0, 1.3.3, 1.4.9
>
> Attachments: HBASE-21357.branch-1.001.patch,
> HBASE-21357.branch-1.001.patch
>
>
> It is a bit strange, we will abort the RS if OOM in Listener thread,
> Responder thread and in CallRunner thread, only not in Reader thread...
> We should abort RS if OOM happens in Reader thread, too. If not, the reader
> thread exists because of OOM, and the selector closes. Later connection
> select to this reader will be ignored
> {code}
> try {
> if (key.isValid()) {
> if (key.isAcceptable())
> doAccept(key);
> }
> } catch (IOException ignored) {
> if (LOG.isTraceEnabled()) LOG.trace("ignored", ignored);
> }
> {code}
> Leaving the client (or Master and other RS)'s call wait until SocketTimeout.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21357) RS should abort if OOM in Reader thread
[
https://issues.apache.org/jira/browse/HBASE-21357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21357:
Component/s: regionserver
> RS should abort if OOM in Reader thread
> ---
>
> Key: HBASE-21357
> URL: https://issues.apache.org/jira/browse/HBASE-21357
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.8
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.9
>
> Attachments: HBASE-21357.branch-1.001.patch,
> HBASE-21357.branch-1.001.patch
>
>
> It is a bit strange, we will abort the RS if OOM in Listener thread,
> Responder thread and in CallRunner thread, only not in Reader thread...
> We should abort RS if OOM happens in Reader thread, too. If not, the reader
> thread exists because of OOM, and the selector closes. Later connection
> select to this reader will be ignored
> {code}
> try {
> if (key.isValid()) {
> if (key.isAcceptable())
> doAccept(key);
> }
> } catch (IOException ignored) {
> if (LOG.isTraceEnabled()) LOG.trace("ignored", ignored);
> }
> {code}
> Leaving the client (or Master and other RS)'s call wait until SocketTimeout.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20694) Consolidate warning on SecureBulkLoad directory permissions
[ https://issues.apache.org/jira/browse/HBASE-20694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20694: Fix Version/s: (was: 1.2.9) 1.2.10 Still want to do this yourself [~elserj]? Maybe move to unassigned and make it a beginner? > Consolidate warning on SecureBulkLoad directory permissions > --- > > Key: HBASE-20694 > URL: https://issues.apache.org/jira/browse/HBASE-20694 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 1.2.10 > > > Follow-on from HBASE-20605: > HBase 1.x has a check which ignores a directory permission check if you're > using a specific filesystem which we think doesnt' do security properly. > HBase 2.x dropped this check. > Since the security of bulk-loaded data is dependent upon this directory > permission (and thus the capabilities of the FileSystem), it would be better > to have a consistent warning across branches. > [~busbey] suggested that we make a WARN message which points admins to our > Book (and write such a section if we don't have something sufficient already) > and our supported filesystems, coupled with an option to disable that warning > message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21013) Backport "read part" of HBASE-18754 to all active 1.x branches
[ https://issues.apache.org/jira/browse/HBASE-21013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21013: Fix Version/s: (was: 1.2.9) 1.2.10 Any progress here? I'm inclined ot go with the forward port of the 0.98 version. > Backport "read part" of HBASE-18754 to all active 1.x branches > -- > > Key: HBASE-21013 > URL: https://issues.apache.org/jira/browse/HBASE-21013 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Mingdao Yang >Priority: Critical > Fix For: 1.5.0, 1.3.3, 1.4.9, 1.2.10 > > > The hfiles impacted by HBASE-18754 will have bytes of proto.TimeRangeTracker. > It makes all 1.x branches failed to read the hfile since all 1.x branches > can't deserialize the proto.TimeRangeTracker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17229) Backport of purge ThreadLocals
[ https://issues.apache.org/jira/browse/HBASE-17229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17229: Fix Version/s: (was: 1.2.9) 1.2.10 What do you folks think, still worth trying to get this in? Stable pointer already has moved to 1.4. Just tell folks hitting the issue that it's time to move to 1.4+? > Backport of purge ThreadLocals > -- > > Key: HBASE-17229 > URL: https://issues.apache.org/jira/browse/HBASE-17229 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 1.2.10 > > > Backport HBASE-17072 and HBASE-16146. The former needs to be backported to > 1.3 ([~mantonov]) and 1.2 ([~busbey]). The latter is already in 1.3. Needs > to be backported to 1.2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690364#comment-16690364
]
Hudson commented on HBASE-20952:
Results for branch HBASE-20952
[build #52 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/52/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/52//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/52//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/52//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21492) CellCodec Written To WAL Before It's Verified
[
https://issues.apache.org/jira/browse/HBASE-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21492:
Priority: Critical (was: Major)
> CellCodec Written To WAL Before It's Verified
> -
>
> Key: HBASE-21492
> URL: https://issues.apache.org/jira/browse/HBASE-21492
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.2.7, 2.0.2
>Reporter: BELUGA BEHR
>Priority: Critical
>
> The cell codec class name is written into the WAL file, but the cell codec
> class is not actually verified to exist. Therefore, users can inadvertently
> configure an invalid class name and it will be recorded into the WAL file.
> At that point, the WAL file becomes unreadable and blocks processing of all
> other WAL files.
> {code:java|title=AbstractProtobufLogWriter.java}
> private WALHeader buildWALHeader0(Configuration conf, WALHeader.Builder
> builder) {
> if (!builder.hasWriterClsName()) {
> builder.setWriterClsName(getWriterClassName());
> }
> if (!builder.hasCellCodecClsName()) {
> builder.setCellCodecClsName(WALCellCodec.getWALCellCodecClass(conf));
> }
> return builder.build();
> }
> {code}
> https://github.com/apache/hbase/blob/025ddce868eb06b4072b5152c5ffae5a01e7ae30/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractProtobufLogWriter.java#L78-L86
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21492) CellCodec Written To WAL Before It's Verified
[
https://issues.apache.org/jira/browse/HBASE-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21492:
Issue Type: Bug (was: Improvement)
> CellCodec Written To WAL Before It's Verified
> -
>
> Key: HBASE-21492
> URL: https://issues.apache.org/jira/browse/HBASE-21492
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.2.7, 2.0.2
>Reporter: BELUGA BEHR
>Priority: Major
>
> The cell codec class name is written into the WAL file, but the cell codec
> class is not actually verified to exist. Therefore, users can inadvertently
> configure an invalid class name and it will be recorded into the WAL file.
> At that point, the WAL file becomes unreadable and blocks processing of all
> other WAL files.
> {code:java|title=AbstractProtobufLogWriter.java}
> private WALHeader buildWALHeader0(Configuration conf, WALHeader.Builder
> builder) {
> if (!builder.hasWriterClsName()) {
> builder.setWriterClsName(getWriterClassName());
> }
> if (!builder.hasCellCodecClsName()) {
> builder.setCellCodecClsName(WALCellCodec.getWALCellCodecClass(conf));
> }
> return builder.build();
> }
> {code}
> https://github.com/apache/hbase/blob/025ddce868eb06b4072b5152c5ffae5a01e7ae30/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractProtobufLogWriter.java#L78-L86
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21492) CellCodec Written To WAL Before It's Verified
BELUGA BEHR created HBASE-21492:
---
Summary: CellCodec Written To WAL Before It's Verified
Key: HBASE-21492
URL: https://issues.apache.org/jira/browse/HBASE-21492
Project: HBase
Issue Type: Improvement
Components: wal
Affects Versions: 2.0.2, 1.2.7
Reporter: BELUGA BEHR
The cell codec class name is written into the WAL file, but the cell codec
class is not actually verified to exist. Therefore, users can inadvertently
configure an invalid class name and it will be recorded into the WAL file. At
that point, the WAL file becomes unreadable and blocks processing of all other
WAL files.
{code:java|title=AbstractProtobufLogWriter.java}
private WALHeader buildWALHeader0(Configuration conf, WALHeader.Builder
builder) {
if (!builder.hasWriterClsName()) {
builder.setWriterClsName(getWriterClassName());
}
if (!builder.hasCellCodecClsName()) {
builder.setCellCodecClsName(WALCellCodec.getWALCellCodecClass(conf));
}
return builder.build();
}
{code}
https://github.com/apache/hbase/blob/025ddce868eb06b4072b5152c5ffae5a01e7ae30/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractProtobufLogWriter.java#L78-L86
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21141: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Thanks for the patch, Artem. > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backuprestore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Fix For: 3.0.0 > > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch, HBASE-21141.v04.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[
https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689940#comment-16689940
]
Hadoop QA commented on HBASE-21141:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
46s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} hbase-backup generated 0 new + 59 unchanged - 3
fixed = 59 total (was 62) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 14s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 17m
15s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948540/HBASE-21141.v04.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 43dc4b1b6ee4 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24
13:14:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 43a10df70f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15054/testReport/ |
| Max. process+thread count | 4414 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15054/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was
[jira] [Updated] (HBASE-21491) [hbase-connectors] Edit on spark connector README
[ https://issues.apache.org/jira/browse/HBASE-21491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21491: -- Fix Version/s: connector-1.0.0 > [hbase-connectors] Edit on spark connector README > - > > Key: HBASE-21491 > URL: https://issues.apache.org/jira/browse/HBASE-21491 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Reporter: stack >Assignee: stack >Priority: Trivial > Fix For: connector-1.0.0 > > > Minor edit to see difference if any made by INFRA-17217 change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21491) [hbase-connectors] Edit on spark connector README
[ https://issues.apache.org/jira/browse/HBASE-21491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-21491. --- Resolution: Fixed Resolving. The merged patch no longer shows as a JIRA comment which is what we wanted. It shows as mail to issues@. Resovling INFRA-17217 too. > [hbase-connectors] Edit on spark connector README > - > > Key: HBASE-21491 > URL: https://issues.apache.org/jira/browse/HBASE-21491 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Reporter: stack >Assignee: stack >Priority: Trivial > > Minor edit to see difference if any made by INFRA-17217 change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[
https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689915#comment-16689915
]
Hadoop QA commented on HBASE-21141:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} hbase-backup generated 0 new + 59 unchanged - 3
fixed = 59 total (was 62) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} hbase-backup: The patch generated 1 new + 0 unchanged
- 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 33s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 17m
47s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948531/HBASE-21141.v03.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 946e665c2c67 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 43a10df70f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15053/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15053/testReport/ |
| Max. process+thread count | 4492 (vs. ulimit of 1) |
| modules | C: hbase-backup
[GitHub] saintstack closed pull request #11: HBASE-21491 [hbase-connectors] Edit on spark connector README
saintstack closed pull request #11: HBASE-21491 [hbase-connectors] Edit on spark connector README URL: https://github.com/apache/hbase-connectors/pull/11 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/spark/README.md b/spark/README.md index acb46bc..eccf663 100755 --- a/spark/README.md +++ b/spark/README.md @@ -8,4 +8,4 @@ To generate an artifact for a different [spark version](https://mvnrepository.co $ mvn -Dspark.version=2.2.2 -Dscala.version=2.11.7 -Dscala.binary.version=2.11 clean install ``` -Spark 2.1.x, 2.2.x, 2.3.x, and 2.4.0 work as does scala 2.11.x. Scala 2.12.x and 2.10.x need work. +See above linked spark version to match spark version and supported scala version. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] saintstack opened a new pull request #11: HBASE-21491 [hbase-connectors] Edit on spark connector README
saintstack opened a new pull request #11: HBASE-21491 [hbase-connectors] Edit on spark connector README URL: https://github.com/apache/hbase-connectors/pull/11 Trivial change just to see if any difference made by INFRA-17217 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-21491) [hbase-connectors] Edit on spark connector README
[ https://issues.apache.org/jira/browse/HBASE-21491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21491: -- Description: Minor edit to see difference if any made by INFRA-17217 change. > [hbase-connectors] Edit on spark connector README > - > > Key: HBASE-21491 > URL: https://issues.apache.org/jira/browse/HBASE-21491 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Reporter: stack >Assignee: stack >Priority: Trivial > > Minor edit to see difference if any made by INFRA-17217 change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21491) [hbase-connectors] Edit on spark connector README
stack created HBASE-21491: - Summary: [hbase-connectors] Edit on spark connector README Key: HBASE-21491 URL: https://issues.apache.org/jira/browse/HBASE-21491 Project: HBase Issue Type: Improvement Components: hbase-connectors Reporter: stack Assignee: stack -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21482) TestHRegion fails due to 'Too many open files'
[
https://issues.apache.org/jira/browse/HBASE-21482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689903#comment-16689903
]
Ted Yu commented on HBASE-21482:
Toward the end of
hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.TestHRegion-output.txt
branch-2 :
{code}
2018-11-15 19:05:42,036 INFO [Time-limited test] hbase.ResourceChecker(172):
after: regionserver.TestHRegion#testCheckAndDelete_ThatDeleteWasWritten
Thread=85 (was 85), OpenFileDescriptor=1276 (was 1273) - OpenFileDescriptor
LEAK? -, MaxFileDescriptor=32000 (was 32000), SystemLoadAverage=149 (was 149),
ProcessCount=361 (was 361), AvailableMemoryMB=36487 (was 36488)
{code}
master branch:
{code}
2018-11-16 19:06:59,290 INFO [Time-limited test] hbase.ResourceChecker(172):
after: regionserver.TestHRegion#testCheckAndDelete_ThatDeleteWasWritten
Thread=79 (was 78) - Thread LEAK? -, OpenFileDescriptor=31932 (was 31934),
MaxFileDescriptor=32000 (was 32000), SystemLoadAverage=82 (was 82),
ProcessCount=363 (was 363), AvailableMemoryMB=36785 (was 36784) -
AvailableMemoryMB LEAK? -
2018-11-16 19:06:59,290 WARN [Time-limited test] hbase.ResourceChecker(135):
OpenFileDescriptor=31932 is superior to 1024
{code}
> TestHRegion fails due to 'Too many open files'
> --
>
> Key: HBASE-21482
> URL: https://issues.apache.org/jira/browse/HBASE-21482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Major
> Attachments:
> org.apache.hadoop.hbase.regionserver.TestHRegion-output.txt,
> org.apache.hadoop.hbase.regionserver.TestHRegion.txt
>
>
> TestHRegion fails due to 'Too many open files' in master branch.
> Here is one failed subtest :
> {code}
> testCheckAndDelete_ThatDeleteWasWritten(org.apache.hadoop.hbase.regionserver.TestHRegion)
> Time elapsed: 2.373 sec <<< ERROR!
> java.lang.IllegalStateException: failed to create a child event loop
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4853)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4844)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4835)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testCheckAndDelete_ThatDeleteWasWritten(TestHRegion.java:2034)
> Caused by: org.apache.hbase.thirdparty.io.netty.channel.ChannelException:
> failed to open a new selector
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4853)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4844)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4835)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testCheckAndDelete_ThatDeleteWasWritten(TestHRegion.java:2034)
> Caused by: java.io.IOException: Too many open files
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4853)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4844)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.initHRegion(TestHRegion.java:4835)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testCheckAndDelete_ThatDeleteWasWritten(TestHRegion.java:2034)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Attachment: HBASE-21141.v04.patch > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backuprestore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch, HBASE-21141.v04.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689859#comment-16689859 ] Artem Ervits commented on HBASE-21141: -- that's a new one for me, I thought it was referring to method name length, not the actual method. The message is gone with v.04. [~yuzhih...@gmail.com] > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backuprestore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch, HBASE-21141.v04.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[
https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689834#comment-16689834
]
Ted Yu commented on HBASE-21141:
You can shrink the following into one line comment:
{code}
+//although split fail, this may not affect following check
+//In old split without AM2, if region's best split key is not found,
+//there are not exception thrown. But in current API, exception
+//will be thrown.
{code}
That would be 3 fewer lines.
> Enable MOB in backup / restore test involving incremental backup
>
>
> Key: HBASE-21141
> URL: https://issues.apache.org/jira/browse/HBASE-21141
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Labels: mob
> Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch,
> HBASE-21141.v03.patch
>
>
> Currently we only have one test (TestRemoteBackup) where MOB feature is
> enabled. The test only performs full backup.
> This issue is to enable MOB in backup / restore test(s) involving incremental
> backup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689825#comment-16689825 ] Ted Yu commented on HBASE-21141: w.r.t. the long method body, can you reduce method length to <= 150 lines by: * dropping some debug logs * removing some empty lines Thanks > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backuprestore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21246) Introduce WALIdentity interface
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689816#comment-16689816 ] Ted Yu commented on HBASE-21246: Patch v43 is mostly formatting change on top of v41. > Introduce WALIdentity interface > --- > > Key: HBASE-21246 > URL: https://issues.apache.org/jira/browse/HBASE-21246 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: HBASE-20952 > > Attachments: 21246.003.patch, 21246.20.txt, 21246.21.txt, > 21246.23.txt, 21246.24.txt, 21246.25.txt, 21246.26.txt, 21246.34.txt, > 21246.37.txt, 21246.39.txt, 21246.41.txt, 21246.43.txt, > 21246.HBASE-20952.001.patch, 21246.HBASE-20952.002.patch, > 21246.HBASE-20952.004.patch, 21246.HBASE-20952.005.patch, > 21246.HBASE-20952.007.patch, 21246.HBASE-20952.008.patch, > replication-src-creates-wal-reader.jpg, wal-factory-providers.png, > wal-providers.png, wal-splitter-reader.jpg, wal-splitter-writer.jpg > > > We are introducing WALIdentity interface so that the WAL representation can > be decoupled from distributed filesystem. > The interface provides getName method whose return value can represent > filename in distributed filesystem environment or, the name of the stream > when the WAL is backed by log stream. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21246) Introduce WALIdentity interface
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21246: --- Attachment: 21246.43.txt > Introduce WALIdentity interface > --- > > Key: HBASE-21246 > URL: https://issues.apache.org/jira/browse/HBASE-21246 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: HBASE-20952 > > Attachments: 21246.003.patch, 21246.20.txt, 21246.21.txt, > 21246.23.txt, 21246.24.txt, 21246.25.txt, 21246.26.txt, 21246.34.txt, > 21246.37.txt, 21246.39.txt, 21246.41.txt, 21246.43.txt, > 21246.HBASE-20952.001.patch, 21246.HBASE-20952.002.patch, > 21246.HBASE-20952.004.patch, 21246.HBASE-20952.005.patch, > 21246.HBASE-20952.007.patch, 21246.HBASE-20952.008.patch, > replication-src-creates-wal-reader.jpg, wal-factory-providers.png, > wal-providers.png, wal-splitter-reader.jpg, wal-splitter-writer.jpg > > > We are introducing WALIdentity interface so that the WAL representation can > be decoupled from distributed filesystem. > The interface provides getName method whose return value can represent > filename in distributed filesystem environment or, the name of the stream > when the WAL is backed by log stream. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[
https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689810#comment-16689810
]
Artem Ervits commented on HBASE-21141:
--
[~yuzhih...@gmail.com] added your comments, there's a checkstyle warning but I
am not sure how it's getting generated
{code:java}
Error sizes MethodLengthMethod length is 157 lines (max allowed is
150).76{code}
> Enable MOB in backup / restore test involving incremental backup
>
>
> Key: HBASE-21141
> URL: https://issues.apache.org/jira/browse/HBASE-21141
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Labels: mob
> Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch,
> HBASE-21141.v03.patch
>
>
> Currently we only have one test (TestRemoteBackup) where MOB feature is
> enabled. The test only performs full backup.
> This issue is to enable MOB in backup / restore test(s) involving incremental
> backup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Attachment: HBASE-21141.v03.patch > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backuprestore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[
https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689806#comment-16689806
]
Hadoop QA commented on HBASE-21141:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} hbase-backup generated 0 new + 59 unchanged - 3
fixed = 59 total (was 62) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 36s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 17m
40s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948526/HBASE-21141.v02.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3a7499c5c5be 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 43a10df70f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15052/testReport/ |
| Max. process+thread count | 4546 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15052/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was
[jira] [Commented] (HBASE-21297) ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change
[
https://issues.apache.org/jira/browse/HBASE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689786#comment-16689786
]
Nihal Jain commented on HBASE-21297:
Thanks for the review [~allan163]. Do you mind committing this or should we
wait for more reviews?
> ModifyTableProcedure can throw TNDE instead of IOE in case of
> REGION_REPLICATION change
> ---
>
> Key: HBASE-21297
> URL: https://issues.apache.org/jira/browse/HBASE-21297
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21297.master.001.patch
>
>
> Currently {{ModifyTableProcedure}} throws an {{IOException}} (See
> [ModifyTableProcedure.java#L252|https://github.com/apache/hbase/blob/924d183ba0e67b975e998f6006c993f457e03c20/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java#L252])
> when a user tries to modify REGION_REPLICATION for an enabled table.
> Instead, it can throw a more specific {{TableNotDisabledException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[
https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689701#comment-16689701
]
Ted Yu commented on HBASE-21141:
We're close.
{code}
+ // #3 - incremental backup for multiple tables
{code}
#3 is repeated. Do you mind re-numbering the steps so that it is easier to
follow ?
Please leave a blank line prior to each step for readability.
{code}
+ LOG.debug("mob has " + TEST_UTIL.countRows(hTable, mobName) + " rows");
+ Assert.assertEquals(TEST_UTIL.countRows(hTable, mobName), NB_ROWS_MOB);
{code}
countRows(hTable, mobName) is called twice - once for LOG and once for
assertion. Can you store the count in a variable so that counting is called
only once ?
Same applies to countRows(hTable, famName) and countRows(hTable, fam2Name).
> Enable MOB in backup / restore test involving incremental backup
>
>
> Key: HBASE-21141
> URL: https://issues.apache.org/jira/browse/HBASE-21141
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Labels: mob
> Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch
>
>
> Currently we only have one test (TestRemoteBackup) where MOB feature is
> enabled. The test only performs full backup.
> This issue is to enable MOB in backup / restore test(s) involving incremental
> backup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20456) Support removing a ReplicationSourceShipper for a special wal group
[
https://issues.apache.org/jira/browse/HBASE-20456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689696#comment-16689696
]
Wellington Chevreuil commented on HBASE-20456:
--
Hi [~Apache9],
Was doing some tests with replication on master branch, and I believe there's
an issue caused by the following change introduced by this patch:
{noformat}
- private void handleEmptyWALEntryBatch(Path currentPath) throws
InterruptedException {
+ private void handleEmptyWALEntryBatch() throws InterruptedException {
LOG.trace("Didn't read any new entries from WAL");
-if (source.isRecovered()) {
- // we're done with queue recovery, shut ourself down
+if (logQueue.isEmpty()) {
+ // we're done with current queue, either this is a recovered queue, or
it is the special group
+ // for a sync replication peer and the peer has been transited to DA or
S state.
{noformat}
The problem is that for a normal source (not replicated), if it starts while
the current WAL is still empty (for instance, just after an RS restart), the
reader will reach WAL's end, then shutdown the source. If then new edits are
added to this same WAL, it won't get replicated until this wal file gets into a
recovery queue.
This is easy to reproduce, just setup replication, restart source RS, do a put
on source, related edit does not get into target until the wal goes to a
recovery queue (restarting RS, for instance, causes the wal containing that put
to be recovered).
> Support removing a ReplicationSourceShipper for a special wal group
> ---
>
> Key: HBASE-20456
> URL: https://issues.apache.org/jira/browse/HBASE-20456
> Project: HBase
> Issue Type: Sub-task
> Components: Replication, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20456-HBASE-19064-v1.patch,
> HBASE-20456-HBASE-19064-v2.patch, HBASE-20456-HBASE-19064-v3.patch,
> HBASE-20456-HBASE-19064-v3.patch, HBASE-20456-HBASE-19064.patch
>
>
> For the multi wal implementation, if a new group is created, then we will
> always open a wal writer for it, even if no one uses it later. But for sync
> replication, if the peer is transited to DA or S from A, the special wal will
> be closed and there will be no more wal files for it so we need to close the
> shipper
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689693#comment-16689693 ] Artem Ervits commented on HBASE-21141: -- [~yuzhih...@gmail.com] v.02 patch addresses your comments. > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backuprestore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Attachment: HBASE-21141.v02.patch > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backuprestore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files
[
https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689656#comment-16689656
]
Ted Yu commented on HBASE-21387:
Haven't got around to adding new unit test (without introducing extra
synchronization primitive in snapshot classes).
Zheng:
If you have bandwidth, you can give it a try.
Thanks
> Race condition surrounding in progress snapshot handling in snapshot cache
> leads to loss of snapshot files
> --
>
> Key: HBASE-21387
> URL: https://issues.apache.org/jira/browse/HBASE-21387
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Labels: snapshot
> Attachments: 21387.dbg.txt, 21387.v10.txt, 21387.v11.txt,
> 21387.v12.txt, 21387.v2.txt, 21387.v3.txt, 21387.v6.txt, 21387.v7.txt,
> 21387.v8.txt, 21387.v9.txt, two-pass-cleaner.v4.txt, two-pass-cleaner.v6.txt,
> two-pass-cleaner.v9.txt
>
>
> During recent report from customer where ExportSnapshot failed:
> {code}
> 2018-10-09 18:54:32,559 ERROR [VerifySnapshot-pool1-t2]
> snapshot.SnapshotReferenceUtil: Can't find hfile:
> 44f6c3c646e84de6a63fe30da4fcb3aa in the real
> (hdfs://in.com:8020/apps/hbase/data/data/.../a/44f6c3c646e84de6a63fe30da4fcb3aa)
> or archive
> (hdfs://in.com:8020/apps/hbase/data/archive/data/.../a/44f6c3c646e84de6a63fe30da4fcb3aa)
> directory for the primary table.
> {code}
> We found the following in log:
> {code}
> 2018-10-09 18:54:23,675 DEBUG
> [00:16000.activeMasterManager-HFileCleaner.large-1539035367427]
> cleaner.HFileCleaner: Removing:
> hdfs:///apps/hbase/data/archive/data/.../a/44f6c3c646e84de6a63fe30da4fcb3aa
> from archive
> {code}
> The root cause is race condition surrounding in progress snapshot(s) handling
> between refreshCache() and getUnreferencedFiles().
> There are two callers of refreshCache: one from RefreshCacheTask#run and the
> other from SnapshotHFileCleaner.
> Let's look at the code of refreshCache:
> {code}
> if (!name.equals(SnapshotDescriptionUtils.SNAPSHOT_TMP_DIR_NAME)) {
> {code}
> whose intention is to exclude in progress snapshot(s).
> Suppose when the RefreshCacheTask runs refreshCache, there is some in
> progress snapshot (about to finish).
> When SnapshotHFileCleaner calls getUnreferencedFiles(), it sees that
> lastModifiedTime is up to date. So cleaner proceeds to check in progress
> snapshot(s). However, the snapshot has completed by that time, resulting in
> some file(s) deemed unreferenced.
> Here is timeline given by Josh illustrating the scenario:
> At time T0, we are checking if F1 is referenced. At time T1, there is a
> snapshot S1 in progress that is referencing a file F1. refreshCache() is
> called, but no completed snapshot references F1. At T2, the snapshot S1,
> which references F1, completes. At T3, we check in-progress snapshots and S1
> is not included. Thus, F1 is marked as unreferenced even though S1 references
> it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] saintstack commented on issue #1: Clean up for hbck2 now that 2.1.1 is out
saintstack commented on issue #1: Clean up for hbck2 now that 2.1.1 is out URL: https://github.com/apache/hbase-operator-tools/pull/1#issuecomment-439453798 Thanks for fixing my dumb check. Was trying to see if INFRA-17217 has been done. Will file new noop issue to see whats changed.. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[ https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689616#comment-16689616 ] stack commented on HBASE-21487: --- Agree that the two procedures should aggregate rather than be an either/or. You have a suggested fix [~arshiya9414]? Thanks. > Concurrent modify table ops can lead to unexpected results > -- > > Key: HBASE-21487 > URL: https://issues.apache.org/jira/browse/HBASE-21487 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.0 >Reporter: Syeda Arshiya Tabreen >Priority: Major > > Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect > result. After HBASE-18893, The behavior of add/delete/modify column family > during concurrent operation is changed compare to branch-1.When one client > is adding cf2 and another one cf3 .. In branch-1 final result will be > cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome depending > on which ModifyTableProcedure executed finally.Its because new table > descriptor is constructed before submitting the ModifyTableProcedure in > HMaster class and its not guarded by any lock. > *Steps to reproduce* > 1.Create table 't' with column family 'f1' > 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table > 't' concurrently. > *Expected Result* > Table should have three column families(f1,f2,f3) > *Actual Result* > Table 't' will have column family either (f1,f2) or (f1,f3) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21485) Add more debug logs for remote procedure execution
[ https://issues.apache.org/jira/browse/HBASE-21485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689612#comment-16689612 ] stack commented on HBASE-21485: --- +1 > Add more debug logs for remote procedure execution > -- > > Key: HBASE-21485 > URL: https://issues.apache.org/jira/browse/HBASE-21485 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.2 > > Attachments: HBASE-21485.patch > > > In our internal testing, sometimes the RefreshPeerProcedure could hang for a > very time. Add more debug logs for better debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] busbey commented on issue #1: Clean up for hbck2 now that 2.1.1 is out
busbey commented on issue #1: Clean up for hbck2 now that 2.1.1 is out URL: https://github.com/apache/hbase-operator-tools/pull/1#issuecomment-439442949 There were two associated JIRA. They're referenced in the commit messages. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] saintstack commented on issue #1: Clean up for hbck2 now that 2.1.1 is out
saintstack commented on issue #1: Clean up for hbck2 now that 2.1.1 is out URL: https://github.com/apache/hbase-operator-tools/pull/1#issuecomment-439438360 Hmm.. So this is pure PR. No associated JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] saintstack commented on issue #1: Clean up for hbck2 now that 2.1.1 is out
saintstack commented on issue #1: Clean up for hbck2 now that 2.1.1 is out URL: https://github.com/apache/hbase-operator-tools/pull/1#issuecomment-439437881 Writing a glowing review here to see effect of INFRA changes to our github/JIRA channel. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[
https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689504#comment-16689504
]
Hadoop QA commented on HBASE-21480:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
45s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 25s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}129m
46s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21480 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948489/HBASE-21480.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d00134cebf38 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 43a10df70f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15051/testReport/ |
| Max. process+thread count | 4773 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15051/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Taking snapshot when RS crashes
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689436#comment-16689436
]
Sean Busbey commented on HBASE-20952:
-
I pushed a commit to move this branch to weekly tests.
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan reassigned HBASE-21489:
-
Assignee: Reid Chan
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Reporter: Duo Zhang
>Assignee: Reid Chan
>Priority: Major
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
> test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:51:in `block in
> test_Revoke_should_rid_access_rights_appropriately'
> 48: create_test_table(@test_name)
> 49: table = table(@test_name)
> 50: user =
> org.apache.hadoop.hbase.security.User.getCurrent().getName();
> => 51: assert_equal(1,
> security_admin.user_permission(@test_name).length)
> 52: security_admin.revoke(user, @test_name)
> 53: assert_equal(0,
> security_admin.user_permission(@test_name).length)
> 54: end
> ===
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21255) [acl] Refactor TablePermission into three classes (Global, Namespace, Table)
[
https://issues.apache.org/jira/browse/HBASE-21255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689421#comment-16689421
]
Sean Busbey commented on HBASE-21255:
-
heads up on HBASE-21489. please take a look.
> [acl] Refactor TablePermission into three classes (Global, Namespace, Table)
>
>
> Key: HBASE-21255
> URL: https://issues.apache.org/jira/browse/HBASE-21255
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACLs, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21225.master.001.patch,
> HBASE-21225.master.002.patch, HBASE-21225.master.007.patch,
> HBASE-21225.master.008.patch, HBASE-21225.master.009.patch,
> HBASE-21225.master.009.patch, HBASE-21255.master.003.patch,
> HBASE-21255.master.004.patch, HBASE-21255.master.005.patch,
> HBASE-21255.master.006.patch, HBASE-21255.master.006.patch
>
>
> A TODO in {{TablePermission.java}}
> {code:java}
> //TODO refactor this class
> //we need to refacting this into three classes (Global, Table, Namespace)
> {code}
> Change Notes:
> * Divide origin TablePermission into three classes GlobalPermission,
> NamespacePermission, TablePermission
> * New UserPermission consists of a user name(string, not byte[], for
> convenience) and a permission in one of [Global, Namespace, Table]Permission.
> * Rename TableAuthManager to AuthManager(it is IA.P), and rename some
> methods for readability.
> * Make PermissionCache thread safe, and the ListMultiMap is changed to Set.
> * User cache and group cache in AuthManager is combined together.
> * Wire proto is kept, BC should be under guarantee.
> * Fix HBASE-21390.
> * Resolve a small {{TODO}} global entry should be handled differently in
> AccessControlLists
> * Add a new api in {{Permission#getAccessScope()}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21490) WALProcedure may remove proc wal files still with active procedures
Duo Zhang created HBASE-21490:
-
Summary: WALProcedure may remove proc wal files still with active
procedures
Key: HBASE-21490
URL: https://issues.apache.org/jira/browse/HBASE-21490
Project: HBase
Issue Type: Sub-task
Components: proc-v2
Reporter: Duo Zhang
It happens for me several times. After master restart, all the procedures are
gone.
And the proc wal files were deleted before restarting, I see this in the
master's log
{noformat}
2018-11-16,20:57:40,177 INFO [WALProcedureStoreSyncThread]
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore: Remove all
state logs with ID less than 184, since all the active procedures are in the
latest log
2018-11-16,20:57:40,177 INFO [WALProcedureStoreSyncThread]
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFile: Archiving
hdfs://c4tst-xiaomi/hbase/c4tst-sync1/MasterProcWALs/pv2-0184.log
to hdfs://c4tst-xiaomi/hbase/c4tst-sync1/oldWALs/pv2-0184.log
{noformat}
Let me dig...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21489) TestShell is broken
Duo Zhang created HBASE-21489: - Summary: TestShell is broken Key: HBASE-21489 URL: https://issues.apache.org/jira/browse/HBASE-21489 Project: HBase Issue Type: Umbrella Components: shell Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21489) TestShell is broken
[
https://issues.apache.org/jira/browse/HBASE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21489:
--
Description:
===
Error:
test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
TypeError: wrong argument type String (expected byte[])
org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
/home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
`block in user_permission'
/home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
`user_permission'
src/test/ruby/hbase/security_admin_test.rb:63:in `block in
test_Grant_should_set_access_rights_appropriately'
60: test_grant_revoke_user =
org.apache.hadoop.hbase.security.User.createUserForTesting(
61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
[]).getName()
62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
=> 63: security_admin.user_permission(@test_name) do |user, permission|
64: assert_match(eval("/WRITE/"), permission.to_s)
65: end
66:
===
===
Error:
test_Revoke_should_rid_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
TypeError: wrong argument type String (expected byte[])
org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
/home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
`block in user_permission'
/home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
`user_permission'
src/test/ruby/hbase/security_admin_test.rb:51:in `block in
test_Revoke_should_rid_access_rights_appropriately'
48: create_test_table(@test_name)
49: table = table(@test_name)
50: user =
org.apache.hadoop.hbase.security.User.getCurrent().getName();
=> 51: assert_equal(1,
security_admin.user_permission(@test_name).length)
52: security_admin.revoke(user, @test_name)
53: assert_equal(0,
security_admin.user_permission(@test_name).length)
54: end
===
> TestShell is broken
> ---
>
> Key: HBASE-21489
> URL: https://issues.apache.org/jira/browse/HBASE-21489
> Project: HBase
> Issue Type: Umbrella
> Components: shell
>Reporter: Duo Zhang
>Priority: Major
>
> ===
> Error:
> test_Grant_should_set_access_rights_appropriately(Hbase::SecureAdminMethodsTest):
> TypeError: wrong argument type String (expected byte[])
> org/jruby/javasupport/JavaArrayUtilities.java:76:in `bytes_to_ruby_string'
> org/jruby/javasupport/JavaArrayUtilities.java:57:in `bytes_to_ruby_string'
> org/jruby/java/addons/StringJavaAddons.java:16:in `from_java_bytes'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:145:in
> `block in user_permission'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/security.rb:144:in
> `user_permission'
> src/test/ruby/hbase/security_admin_test.rb:63:in `block in
> test_Grant_should_set_access_rights_appropriately'
> 60: test_grant_revoke_user =
> org.apache.hadoop.hbase.security.User.createUserForTesting(
> 61: $TEST_CLUSTER.getConfiguration, "test_grant_revoke",
> []).getName()
> 62: security_admin.grant(test_grant_revoke_user,"W", @test_name)
> => 63: security_admin.user_permission(@test_name) do |user,
> permission|
> 64: assert_match(eval("/WRITE/"), permission.to_s)
> 65: end
> 66:
> ===
> ===
> Error:
[jira] [Commented] (HBASE-21488) Reimplement proc-v1 based procedures with proc-v2
[ https://issues.apache.org/jira/browse/HBASE-21488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689344#comment-16689344 ] Duo Zhang commented on HBASE-21488: --- [~stack] FYI. > Reimplement proc-v1 based procedures with proc-v2 > - > > Key: HBASE-21488 > URL: https://issues.apache.org/jira/browse/HBASE-21488 > Project: HBase > Issue Type: Umbrella > Components: proc-v2 >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > 1. Flush table. > 2. Snapshot. > 3. Log rolling. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21488) Reimplement proc-v1 based procedures with proc-v2
Duo Zhang created HBASE-21488: - Summary: Reimplement proc-v1 based procedures with proc-v2 Key: HBASE-21488 URL: https://issues.apache.org/jira/browse/HBASE-21488 Project: HBase Issue Type: Umbrella Components: proc-v2 Reporter: Duo Zhang Fix For: 3.0.0, 2.2.0 1. Flush table. 2. Snapshot. 3. Log rolling. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[ https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syeda Arshiya Tabreen updated HBASE-21487: -- Description: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result. After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1.When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome depending on which ModifyTableProcedure executed finally.Its because new table descriptor is constructed before submitting the ModifyTableProcedure in HMaster class and its not guarded by any lock. *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. *Expected Result* Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) was: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result. After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1.When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. *Expected Result* Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? > Concurrent modify table ops can lead to unexpected results > -- > > Key: HBASE-21487 > URL: https://issues.apache.org/jira/browse/HBASE-21487 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.0 >Reporter: Syeda Arshiya Tabreen >Priority: Major > > Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect > result. After HBASE-18893, The behavior of add/delete/modify column family > during concurrent operation is changed compare to branch-1.When one client > is adding cf2 and another one cf3 .. In branch-1 final result will be > cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome depending > on which ModifyTableProcedure executed finally.Its because new table > descriptor is constructed before submitting the ModifyTableProcedure in > HMaster class and its not guarded by any lock. > *Steps to reproduce* > 1.Create table 't' with column family 'f1' > 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table > 't' concurrently. > *Expected Result* > Table should have three column families(f1,f2,f3) > *Actual Result* > Table 't' will have column family either (f1,f2) or (f1,f3) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[ https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syeda Arshiya Tabreen updated HBASE-21487: -- Description: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result. After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1.When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. *Expected Result* Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? was: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result. After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1.When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. Expected Result Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? > Concurrent modify table ops can lead to unexpected results > -- > > Key: HBASE-21487 > URL: https://issues.apache.org/jira/browse/HBASE-21487 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.0 >Reporter: Syeda Arshiya Tabreen >Priority: Major > > Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect > result. After HBASE-18893, The behavior of add/delete/modify column family > during concurrent operation is changed compare to branch-1.When one client > is adding cf2 and another one cf3 .. In branch-1 final result will be > cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. > *Steps to reproduce* > 1.Create table 't' with column family 'f1' > 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table > 't' concurrently. > *Expected Result* > Table should have three column families(f1,f2,f3) > *Actual Result* > Table 't' will have column family either (f1,f2) or (f1,f3) > *Analysis/Issue* > Its because new table descriptor is constructed before submitting the > modifyTableProcedure in HMaster class and its not guarded by any lock. > Is this the intended behavior? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[ https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syeda Arshiya Tabreen updated HBASE-21487: -- Description: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. Expected Result Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1 When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? > Concurrent modify table ops can lead to unexpected results > -- > > Key: HBASE-21487 > URL: https://issues.apache.org/jira/browse/HBASE-21487 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.0 >Reporter: Syeda Arshiya Tabreen >Priority: Major > > Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect > result > *Steps to reproduce* > 1.Create table 't' with column family 'f1' > 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table > 't' concurrently. > Expected Result > Table should have three column families(f1,f2,f3) > *Actual Result* > Table 't' will have column family either (f1,f2) or (f1,f3) > *Analysis/Issue* > After HBASE-18893, The behavior of add/delete/modify column family during > concurrent operation is changed compare to branch-1 > When one client is adding cf2 and another one cf3 .. In branch-1 final > result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the > outcome. > Its because new table descriptor is constructed before submitting the > modifyTableProcedure in HMaster class and its not guarded by any lock. > Is this the intended behavior? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[ https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syeda Arshiya Tabreen updated HBASE-21487: -- Description: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result. *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. Expected Result Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1 When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? was: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. Expected Result Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1 When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? > Concurrent modify table ops can lead to unexpected results > -- > > Key: HBASE-21487 > URL: https://issues.apache.org/jira/browse/HBASE-21487 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.0 >Reporter: Syeda Arshiya Tabreen >Priority: Major > > Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect > result. > *Steps to reproduce* > 1.Create table 't' with column family 'f1' > 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table > 't' concurrently. > Expected Result > Table should have three column families(f1,f2,f3) > *Actual Result* > Table 't' will have column family either (f1,f2) or (f1,f3) > *Analysis/Issue* > After HBASE-18893, The behavior of add/delete/modify column family during > concurrent operation is changed compare to branch-1 > When one client is adding cf2 and another one cf3 .. In branch-1 final > result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the > outcome. > Its because new table descriptor is constructed before submitting the > modifyTableProcedure in HMaster class and its not guarded by any lock. > Is this the intended behavior? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[ https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syeda Arshiya Tabreen updated HBASE-21487: -- Description: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result. After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1.When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. Expected Result Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? was: Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect result. *Steps to reproduce* 1.Create table 't' with column family 'f1' 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table 't' concurrently. Expected Result Table should have three column families(f1,f2,f3) *Actual Result* Table 't' will have column family either (f1,f2) or (f1,f3) *Analysis/Issue* After HBASE-18893, The behavior of add/delete/modify column family during concurrent operation is changed compare to branch-1 When one client is adding cf2 and another one cf3 .. In branch-1 final result will be cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. Its because new table descriptor is constructed before submitting the modifyTableProcedure in HMaster class and its not guarded by any lock. Is this the intended behavior? > Concurrent modify table ops can lead to unexpected results > -- > > Key: HBASE-21487 > URL: https://issues.apache.org/jira/browse/HBASE-21487 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.0 >Reporter: Syeda Arshiya Tabreen >Priority: Major > > Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect > result. After HBASE-18893, The behavior of add/delete/modify column family > during concurrent operation is changed compare to branch-1.When one client > is adding cf2 and another one cf3 .. In branch-1 final result will be > cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome. > *Steps to reproduce* > 1.Create table 't' with column family 'f1' > 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table > 't' concurrently. > Expected Result > Table should have three column families(f1,f2,f3) > *Actual Result* > Table 't' will have column family either (f1,f2) or (f1,f3) > *Analysis/Issue* > Its because new table descriptor is constructed before submitting the > modifyTableProcedure in HMaster class and its not guarded by any lock. > Is this the intended behavior? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21484) [HBCK2] hbck2 should default to a released hbase version
[ https://issues.apache.org/jira/browse/HBASE-21484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21484: Resolution: Fixed Status: Resolved (was: Patch Available) > [HBCK2] hbck2 should default to a released hbase version > > > Key: HBASE-21484 > URL: https://issues.apache.org/jira/browse/HBASE-21484 > Project: HBase > Issue Type: Bug > Components: hbck2 >Affects Versions: hbck2-1.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: hbck2-1.0.0 > > > can't build from clean checkout because 2.1.1-SNAPSHOT isn't a released > version -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21483) [HBCK2] version string checking should look for exactly the version we know doesn't work
[ https://issues.apache.org/jira/browse/HBASE-21483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21483: Resolution: Fixed Status: Resolved (was: Patch Available) > [HBCK2] version string checking should look for exactly the version we know > doesn't work > > > Key: HBASE-21483 > URL: https://issues.apache.org/jira/browse/HBASE-21483 > Project: HBase > Issue Type: Bug > Components: hbck2 >Affects Versions: hbck2-1.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: hbck2-1.0.0 > > > Right now the version check looks for anything that starts with "2.1.0" and > declares HBCK2 a lost cause. I presume this was to deal with > "2.1.0-SNAPSHOT", but that no longer needs to be a concern. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] busbey closed pull request #1: Clean up for hbck2 now that 2.1.1 is out
busbey closed pull request #1: Clean up for hbck2 now that 2.1.1 is out
URL: https://github.com/apache/hbase-operator-tools/pull/1
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
b/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
index 49e9c64..e9089e8 100644
--- a/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
+++ b/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
@@ -101,12 +101,13 @@ void checkHBCKSupport(Connection connection) throws
IOException {
}
static void checkVersion(final String versionStr) {
-if (versionStr.startsWith(TWO_POINT_ONE)) {
- throw new UnsupportedOperationException(TWO_POINT_ONE + " has no support
for hbck2");
-}
if (VersionInfo.compareVersion(MININUM_VERSION, versionStr) > 0) {
throw new UnsupportedOperationException("Requires " + MININUM_VERSION +
" at least.");
}
+// except 2.1.0 didn't ship with support
+if (VersionInfo.compareVersion(TWO_POINT_ONE, versionStr) == 0) {
+ throw new UnsupportedOperationException(TWO_POINT_ONE + " has no support
for hbck2");
+}
}
TableState setTableState(TableName tableName, TableState.State state) throws
IOException {
diff --git a/hbase-hbck2/src/test/java/org/apache/hbase/TestHBCK2.java
b/hbase-hbck2/src/test/java/org/apache/hbase/TestHBCK2.java
index 1b54687..e902275 100644
--- a/hbase-hbck2/src/test/java/org/apache/hbase/TestHBCK2.java
+++ b/hbase-hbck2/src/test/java/org/apache/hbase/TestHBCK2.java
@@ -69,6 +69,11 @@ public void testCheckVersion210() {
HBCK2.checkVersion("2.1.0");
}
+ @Test
+ public void testCheckVersionSpecial210() {
+HBCK2.checkVersion("2.1.0-patchedForHBCK2");
+ }
+
@Test
public void testCheckVersion203() {
HBCK2.checkVersion("2.0.3");
diff --git a/pom.xml b/pom.xml
index f241901..e975217 100644
--- a/pom.xml
+++ b/pom.xml
@@ -123,7 +123,7 @@
1.8
${compileSource}
3.3.3
-2.1.1-SNAPSHOT
+2.1.1
3.6.1
2.21.0
surefire-junit47
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21480: -- Status: Patch Available (was: Open) > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] busbey commented on issue #1: Clean up for hbck2 now that 2.1.1 is out
busbey commented on issue #1: Clean up for hbck2 now that 2.1.1 is out URL: https://github.com/apache/hbase-operator-tools/pull/1#issuecomment-439373422 thanks for the review! for now folks have to run the UTs locally. getting qa integration for these new repos is still an open task. hbase-operator-tools integration is [HBASE-21433](https://issues.apache.org/jira/browse/HBASE-21433) and hbase-connectors is [HBASE-21432](https://issues.apache.org/jira/browse/HBASE-21432) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689321#comment-16689321 ] Duo Zhang commented on HBASE-21480: --- Review board link: https://reviews.apache.org/r/69372/ > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21480: -- Attachment: HBASE-21480.patch > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch, > HBASE-21480.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-21480: - Assignee: Duo Zhang > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21480) Taking snapshot when RS crashes prevent we bring the regions online
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21480: -- Summary: Taking snapshot when RS crashes prevent we bring the regions online (was: Taking snapshot when RS crashes may hang) > Taking snapshot when RS crashes prevent we bring the regions online > --- > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21480) Taking snapshot when RS crashes may hang
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689314#comment-16689314 ] Duo Zhang commented on HBASE-21480: --- OK, the snapshot implementation itself can not deal with rs crashes, it will hang there forever if there are regions on crashed region servers... So the new UT just confirm that, the snapshot operation does not prevent we bring the regions online... > Taking snapshot when RS crashes may hang > > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21480) Taking snapshot when RS crashes may hang
[ https://issues.apache.org/jira/browse/HBASE-21480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21480: -- Attachment: HBASE-21480-UT-v1.patch > Taking snapshot when RS crashes may hang > > > Key: HBASE-21480 > URL: https://issues.apache.org/jira/browse/HBASE-21480 > Project: HBase > Issue Type: Improvement > Components: snapshots >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21480-UT-v1.patch, HBASE-21480-UT.patch > > > The current implementation is not good enough. It will take the exclusive > lock all the time which could hurt the availability, as we need to hold the > shared lock when assigning regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21485) Add more debug logs for remote procedure execution
[
https://issues.apache.org/jira/browse/HBASE-21485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689308#comment-16689308
]
Hudson commented on HBASE-21485:
Results for branch master
[build #610 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/610/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/610//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/610//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/610//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add more debug logs for remote procedure execution
> --
>
> Key: HBASE-21485
> URL: https://issues.apache.org/jira/browse/HBASE-21485
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21485.patch
>
>
> In our internal testing, sometimes the RefreshPeerProcedure could hang for a
> very time. Add more debug logs for better debugging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
Syeda Arshiya Tabreen created HBASE-21487: - Summary: Concurrent modify table ops can lead to unexpected results Key: HBASE-21487 URL: https://issues.apache.org/jira/browse/HBASE-21487 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 3.0.0 Reporter: Syeda Arshiya Tabreen -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21485) Add more debug logs for remote procedure execution
[
https://issues.apache.org/jira/browse/HBASE-21485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689303#comment-16689303
]
Hudson commented on HBASE-21485:
Results for branch branch-2
[build #1506 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1506/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1506//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1506//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1506//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add more debug logs for remote procedure execution
> --
>
> Key: HBASE-21485
> URL: https://issues.apache.org/jira/browse/HBASE-21485
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21485.patch
>
>
> In our internal testing, sometimes the RefreshPeerProcedure could hang for a
> very time. Add more debug logs for better debugging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21485) Add more debug logs for remote procedure execution
[
https://issues.apache.org/jira/browse/HBASE-21485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689288#comment-16689288
]
Hudson commented on HBASE-21485:
Results for branch branch-2.1
[build #611 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/611/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/611//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/611//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/611//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add more debug logs for remote procedure execution
> --
>
> Key: HBASE-21485
> URL: https://issues.apache.org/jira/browse/HBASE-21485
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21485.patch
>
>
> In our internal testing, sometimes the RefreshPeerProcedure could hang for a
> very time. Add more debug logs for better debugging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21460) correct Document Configurable Bucket Sizes in bucketCache
[
https://issues.apache.org/jira/browse/HBASE-21460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689275#comment-16689275
]
Hudson commented on HBASE-21460:
Results for branch master
[build #609 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/609/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/609//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/609//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/609//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> correct Document Configurable Bucket Sizes in bucketCache
> -
>
> Key: HBASE-21460
> URL: https://issues.apache.org/jira/browse/HBASE-21460
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21460.v1.patch, HBASE-21460.v2.patch
>
>
> we use the bucket cache(offheap),found the doc was error,
> the property bucket sizes shoul be "hbase.bucketcache.bucket.sizes" instead
> of "hfile.block.cache.sizes"
> CacheConfig.java
> /**
> * A comma-delimited array of values for use as bucket sizes.
> */
> public static final String BUCKET_CACHE_BUCKETS_KEY =
> "hbase.bucketcache.bucket.sizes";
> the doc was:
>
>
> HBASE-10641 introduced the ability to configure multiple sizes for the
> buckets of the BucketCache, in HBase 0.98 and newer. To configurable multiple
> bucket sizes, configure the new property
> {color:#ff}{{hfile.block.cache.sizes}}{color} (instead of{color:#ff}
> {{hfile.block.cache.size}}{color}) to a comma-separated list of block sizes,
> ordered from smallest to largest, with no spaces. The goal is to optimize the
> bucket sizes based on your data access patterns. The following example
> configures buckets of size 4096 and 8192.
> {color:#ff}hfile.block.cache.sizes{color}
> 4096,8192
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21428) Performance issue due to userRegionLock in the ConnectionManager.
[
https://issues.apache.org/jira/browse/HBASE-21428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689264#comment-16689264
]
Yu Li commented on HBASE-21428:
---
Thanks for drawing my attention [~reidchan].
As mentioned in HBASE-19260, allowing parallel accessing meta to locate region
is dangerous and a behavior change comparing 0.98, so we regarded it as a
regression issue.
While indeed we neglected the already existing
{{hbase.hconnection.meta.lookup.threads.max/core}} configurations and maybe it
could be a better idea to remove the lock and set the default settings of these
two properties to 1, so advanced users could increase the settings (although
not recommended) when they're sure their parallel access won't drive the
PriorityHandlers mad.
And I don't think we need a group of locks when tuning the configuration is
enough to control the parallelism.
> Performance issue due to userRegionLock in the ConnectionManager.
> -
>
> Key: HBASE-21428
> URL: https://issues.apache.org/jira/browse/HBASE-21428
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.7
>Reporter: koo
>Priority: Major
>
> My service is that execute a lot of puts using HTableMultiplexer.
> After the version change, most of the requests are rejected.
> It works fine in 1.2.6.1, but there is a problem in 1.2.7.
> This issue is related with the HBASE-19260.
> Most of my threads are using a lot of time as below.
>
> |"Worker-972" #2479 daemon prio=5 os_prio=0 tid=0x7f8cea86b000 nid=0x4c8c
> waiting on condition [0x7f8b78104000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005dd703b78> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1274)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1186)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1170)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1127)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:962)
> at
> org.apache.hadoop.hbase.client.HTableMultiplexer.put(HTableMultiplexer.java:206)
> at
> org.apache.hadoop.hbase.client.HTableMultiplexer.put(HTableMultiplexer.java:150)|
>
> When I looked at the issue(HBASE-19260), I recognized the dangerous of to
> allow accessessing multiple threads.
> However, Already create many threads with the limitations
> I think it is very inefficient to allow only one thread access.
>
> | this.metaLookupPool = getThreadPool(
> conf.getInt("hbase.hconnection.meta.lookup.threads.max", 128),
> conf.getInt("hbase.hconnection.meta.lookup.threads.core", 10),
> "-metaLookup-shared-", new LinkedBlockingQueue());|
>
> I want to suggest changing it that allow to have multiple locks.(but not the
> entire thread)
> The following is pseudocode.
>
> |int lockSize = conf.getInt("hbase.hconnection.meta.lookup.threads.max", 128)
> / 2;
> BlockingQueue userRegionLockQueue = new
> LinkedBlockingQueue();
> for (int i=0; i userRegionLockQueue.put(new ReentrantLock());
> }|
>
> thanks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
