[jira] [Commented] (HBASE-5976) Initial module naming
[ https://issues.apache.org/jira/browse/HBASE-5976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273861#comment-13273861 ] stack commented on HBASE-5976: -- bq. @Matt: why the disposition to pulling up rather than pushing down? Thinking on it, I think pulling up -- where up takes us toward more base types and down takes us toward client facing APIs -- is viable in that I can see us making headway extracting base types and utility out into coherent modules. Going the other way, starting out at a base and then pulling classes won't happen methinks. The master and the client and the regionserver packages all reach into other. You won't be able to make modules of them (IMO). In fact, I was thinking that the only way to go down would be by writing completely new stuff; e.g. a new client (perhaps in org.apache.hbase package rather than o.a.h.h to delineate it for sure a break w/ the past, or an hbase-server only this server is server -- o.a.h.Server -- does regionserver and master roles when called upon). If this the case, hbase-common might be hard to work with. If we went w/ hbase-common, how would we make superior modules? What would we have to name them: hbase-base? hbase-utils? hbase-base-types, hbase-foundation? hbase-fundamentals? hbase-cluster is a little awkward. I do appreciate that it is not 'loaded' like core and common are though so its hard to peg which 'tier' it belongs to. I would like to avoid our having core and common as over in hadoop. Its strikes me as a failed modeling exercise whenever I trip over it. Initial module naming - Key: HBASE-5976 URL: https://issues.apache.org/jira/browse/HBASE-5976 Project: HBase Issue Type: Brainstorming Components: build Affects Versions: 0.96.0 Reporter: Jesse Yates Original Estimate: 336h Remaining Estimate: 336h There is currently a lot of discussion around the 'right' name for the initial module that will host all the code in the primary modularization of hbase. The current contenders are: * hbase-core * hbase-common * hbase-server * hbase-bucket * hbase-hbase Let's duke it out and pick the best, while keeping in mind that this module will *not* remain the sole module going forward, but is merely the precursor Timeline to close this issue is the day before the code gets committed. Source: http://search-hadoop.com/m/pwi1t1e9K0R/modularizing+trunksubj=modularizing+trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5989) TestAdmin failed on Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5989: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to TRUNK. Thanks for the patch Jimmy. I looked in 0.94. The patch doesn't apply because there isn't code like this there. TestAdmin failed on Hadoop 2.0.0 Key: HBASE-5989 URL: https://issues.apache.org/jira/browse/HBASE-5989 Project: HBase Issue Type: Test Components: test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.96.0 Attachments: hbase_5989.patch Running org.apache.hadoop.hbase.client.TestAdmin Tests run: 39, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.155 sec FAILURE! Tests in error: testCheckHBaseAvailableWithoutCluster(org.apache.hadoop.hbase.client.TestAdmin): For input string: localhost -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha
[
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273870#comment-13273870
]
Hudson commented on HBASE-5966:
---
Integrated in HBase-TRUNK #2872 (See
[https://builds.apache.org/job/HBase-TRUNK/2872/])
HBASE-5966 MapReduce based tests broken on Hadoop 2.0.0-alpha (Revision
1337448)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
MapReduce based tests broken on Hadoop 2.0.0-alpha
--
Key: HBASE-5966
URL: https://issues.apache.org/jira/browse/HBASE-5966
Project: HBase
Issue Type: Bug
Components: mapred, mapreduce, test
Affects Versions: 0.94.0, 0.96.0
Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT,
Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
Reporter: Andrew Purtell
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: HBASE-5966-1.patch, HBASE-5966.patch, hbase-5966.patch
Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test
rigging. Below is a representative error, can be easily reproduced with:
{noformat}
mvn -PlocalTests -Psecurity \
-Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
clean test \
-Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
{noformat}
And the result:
{noformat}
---
T E S T S
---
Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec
FAILURE!
---
Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
---
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec
FAILURE!
testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)
Time elapsed: 21.935 sec ERROR!
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at
[jira] [Commented] (HBASE-5989) TestAdmin failed on Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273879#comment-13273879 ] Hudson commented on HBASE-5989: --- Integrated in HBase-TRUNK #2873 (See [https://builds.apache.org/job/HBase-TRUNK/2873/]) HBASE-5989 TestAdmin failed on Hadoop 2.0.0 (Revision 1337450) Result = FAILURE stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java TestAdmin failed on Hadoop 2.0.0 Key: HBASE-5989 URL: https://issues.apache.org/jira/browse/HBASE-5989 Project: HBase Issue Type: Test Components: test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.96.0 Attachments: hbase_5989.patch Running org.apache.hadoop.hbase.client.TestAdmin Tests run: 39, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.155 sec FAILURE! Tests in error: testCheckHBaseAvailableWithoutCluster(org.apache.hadoop.hbase.client.TestAdmin): For input string: localhost -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-5385: --- Attachment: HBASE-5385-v2.patch patch rebased, HBASE-5732 went in. (Also opened HBASE-5947, for the pre/post check for empty acl) Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-5342: --- Attachment: HBASE-5342-v5.patch patch rebased, HBASE-5732 went in. Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5984) TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273896#comment-13273896
]
Zhihong Yu commented on HBASE-5984:
---
+1 on the above suggestion.
TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
Key: HBASE-5984
URL: https://issues.apache.org/jira/browse/HBASE-5984
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: hbase_5984.patch
java.io.IOException: Cannot obtain block length for
LocatedBlock{BP-1455809779-127.0.0.1-1336670196362:blk_-6960847342982670493_1028;
getBlockSize()=1474; corrupt=false; offset=0; locs=[127.0.0.1:58343,
127.0.0.1:48427]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:232)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:177)
at
org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:119)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:112)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:928)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:212)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:75)
at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1768)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.openFile(SequenceFileLogReader.java:66)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1688)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1709)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.init(SequenceFileLogReader.java:58)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:166)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:659)
at
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5992: -- Attachment: 5992.v5.patch Re-attaching patch v5. Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273902#comment-13273902 ] Hadoop QA commented on HBASE-5385: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526613/HBASE-5385-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 27 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1856//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1856//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1856//console This message is automatically generated. Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5385: -- Attachment: 5385-v3.patch Patch v3 puts scanner.close() in finally block. Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273907#comment-13273907 ] Hadoop QA commented on HBASE-5342: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526614/HBASE-5342-v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 31 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestDrainingServer Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1857//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1857//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1857//console This message is automatically generated. Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5959) Add other load balancers
[
https://issues.apache.org/jira/browse/HBASE-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273912#comment-13273912
]
Zhihong Yu commented on HBASE-5959:
---
For a patch of this size, review board should be used.
Below is part 1 of my review.
It is necessary to see the benefit of stochastic balancer in a real cluster
compared with existing balancer.
TestRegionRebalancing is an important test which currently exercises default
balancer. It would be nice to apply new balancer there - or create a similar
test class.
{code}
+ * {@link AssignmentManager} to assign regions in the edge cases. It doesn't
{code}
Are edge cases the only ones covered by BaseLoadBalancer ?
{code}
+public class ClusterLoadState {
{code}
Please add javadoc for the above class.
{code}
+ private final NavigableMapServerAndLoad, ListHRegionInfo serversByLoad;
{code}
The above map doesn't contain region load information. Can region load be added
?
{code}
+ protected ListServerName getInternalTopBlockLocations( HRegionInfo region)
{
{code}
A better name for the above method would be internalGetTopBlockLocations().
Remove the space after '('.
{code}
+ * @param fs
+ * the filesystem
{code}
There is no need to put parameter explanation on separate line.
There're two almost identical getTableDescriptor() methods in master package:
{code}
private HTableDescriptor getTableDescriptor(byte[] tableName)
src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
private HTableDescriptor getTableDescriptor(byte[] tableName)
src/main/java/org/apache/hadoop/hbase/master/DefaultLoadBalancer.java
{code}
It would be nice to put it in a utility class.
For StochasticLoadBalancer:
{code}
+ * randomly try and mutate up to the cluster to Cprime. If F(Cprime) F(C)
then
{code}
Remove 'up to' above.
{code}
+ public void setMasterServices(MasterServices masterServices) {
+this.services = masterServices;
+ }
{code}
RegionLocationFinder has MasterServices member which should be set in the above
method.
{code}
+ * should always approach
{code}
Please finish the above sentence for balanceCluster().
{code}
+// No need to balance one cluster.
+if (clusterState.size() = 1) {
{code}
The above check should be placed at the beginning of the method. 'one cluster'
- 'cluster with one server'
{code}
+// Perform a stocastic walk to see if we can get a good fit.
{code}
'stocastic' - 'stochastic'
{code}
+HRegionInfo lRegion = pickRandmoRegion(leftRegionList, 0);
+HRegionInfo rRegion = pickRandmoRegion(rightRegionList, 0.5);
{code}
Please explain why selection of move is not symmetrical.
{code}
+boolean keep = RANDOM.nextFloat() (0.01 * (1 - scale(0, maxSteps,
stepNum)));
+return keep;
{code}
A single line should be enough - return directly.
{code}
+ * @return
+ */
+ private MapHRegionInfo, ServerName createRegionMapping(
{code}
Please add javadoc for return value.
{code}
+double regionCountSkewCost = 100 * computeSkewLoadCost(clusterState);
+double tableSkewCost = 50 * computeTableSkewLoadCost(clusterState);
+double localityCost = 40 * computeDataLocalityCost(clusterState);
{code}
May I ask how the above coefficients were determined ? Should user be able to
change them through config parameters ?
For computeSkewLoadCost():
{code}
+double max = (mean * (clusterState.size() - 1)) + (Math.abs(mean -
numRegions));
{code}
numRegions is total number of regions, meaning it is greater than mean. The
Math.abs() isn't necessary above.
May I ask how the above max is derived ?
Add other load balancers
Key: HBASE-5959
URL: https://issues.apache.org/jira/browse/HBASE-5959
Project: HBase
Issue Type: New Feature
Components: master
Affects Versions: 0.96.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-5959-0.patch, HBASE-5959-1.patch,
HBASE-5959-2.patch, HBASE-5959-3.patch
Now that balancers are pluggable we should give some options.b
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273914#comment-13273914 ] Hadoop QA commented on HBASE-5992: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526617/5992.v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 27 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1858//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1858//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1858//console This message is automatically generated. Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273913#comment-13273913 ] Zhihong Yu commented on HBASE-5342: --- Failed test is covered by HBASE-5992. Integrated to trunk. Thanks for the patch, Matteo. Thanks for the review, Andy. Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273915#comment-13273915 ] Zhihong Yu commented on HBASE-5385: --- @Matteo: Now that HBASE-5342 went in, can you rebase patch v3 :-) ? Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273916#comment-13273916 ] Hadoop QA commented on HBASE-5385: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526619/5385-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 27 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestDrainingServer Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1859//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1859//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1859//console This message is automatically generated. Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273918#comment-13273918 ] Zhihong Yu commented on HBASE-5992: --- Still cannot find the hanging test from Jenkins console. I am running test suite on Linux. If there is no hanging test there, will integrate the patch. Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273923#comment-13273923 ] Matteo Bertozzi commented on HBASE-5342: Can we backport this to 0.92 and 0.94? the security/access code is the same and this one doesn't break the compatibility. I'll attach a patch later for 0.92/0.94. Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273929#comment-13273929 ] Zhihong Yu commented on HBASE-5342: --- @Matteo: Please run through corresponding test suite before posting backport. @Andy, @Lars: What do you think of the backport ? Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-5385: --- Attachment: HBASE-5385-v3.patch rebase after HBASE-5342 merge Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch, HBASE-5385-v3.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273934#comment-13273934 ] Zhihong Yu commented on HBASE-5385: --- Patch v3 integrated to trunk. Thanks for the patch, Matteo. Thanks for the review, Andy. Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch, HBASE-5385-v3.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273945#comment-13273945 ] Hadoop QA commented on HBASE-5385: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526622/HBASE-5385-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 31 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestDrainingServer Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1860//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1860//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1860//console This message is automatically generated. Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch, HBASE-5385-v3.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273947#comment-13273947 ] Hudson commented on HBASE-5342: --- Integrated in HBase-TRUNK #2875 (See [https://builds.apache.org/job/HBase-TRUNK/2875/]) HBASE-5342 Grant/Revoke global permissions (Matteo Bertozzi) (Revision 1337499) Result = FAILURE tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java * /hbase/trunk/src/main/ruby/hbase/security.rb * /hbase/trunk/src/main/ruby/shell.rb * /hbase/trunk/src/main/ruby/shell/commands/grant.rb * /hbase/trunk/src/main/ruby/shell/commands/revoke.rb * /hbase/trunk/src/main/ruby/shell/commands/user_permission.rb * /hbase/trunk/src/main/ruby/shell/commands/whoami.rb * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273949#comment-13273949 ] Hudson commented on HBASE-5385: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #2 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/2/]) HBASE-5385 Delete table/column should delete stored permissions on -acl- table (Matteo Bertozi) (Revision 1337512) Result = FAILURE tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch, HBASE-5385-v3.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5989) TestAdmin failed on Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273952#comment-13273952 ] Hudson commented on HBASE-5989: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #2 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/2/]) HBASE-5989 TestAdmin failed on Hadoop 2.0.0 (Revision 1337450) Result = FAILURE stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java TestAdmin failed on Hadoop 2.0.0 Key: HBASE-5989 URL: https://issues.apache.org/jira/browse/HBASE-5989 Project: HBase Issue Type: Test Components: test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.96.0 Attachments: hbase_5989.patch Running org.apache.hadoop.hbase.client.TestAdmin Tests run: 39, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.155 sec FAILURE! Tests in error: testCheckHBaseAvailableWithoutCluster(org.apache.hadoop.hbase.client.TestAdmin): For input string: localhost -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273951#comment-13273951 ] Hudson commented on HBASE-5342: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #2 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/2/]) HBASE-5342 Grant/Revoke global permissions (Matteo Bertozzi) (Revision 1337499) Result = FAILURE tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java * /hbase/trunk/src/main/ruby/hbase/security.rb * /hbase/trunk/src/main/ruby/shell.rb * /hbase/trunk/src/main/ruby/shell/commands/grant.rb * /hbase/trunk/src/main/ruby/shell/commands/revoke.rb * /hbase/trunk/src/main/ruby/shell/commands/user_permission.rb * /hbase/trunk/src/main/ruby/shell/commands/whoami.rb * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha
[
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273950#comment-13273950
]
Hudson commented on HBASE-5966:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #2 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/2/])
HBASE-5966 MapReduce based tests broken on Hadoop 2.0.0-alpha (Revision
1337448)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
MapReduce based tests broken on Hadoop 2.0.0-alpha
--
Key: HBASE-5966
URL: https://issues.apache.org/jira/browse/HBASE-5966
Project: HBase
Issue Type: Bug
Components: mapred, mapreduce, test
Affects Versions: 0.94.0, 0.96.0
Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT,
Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
Reporter: Andrew Purtell
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: HBASE-5966-1.patch, HBASE-5966.patch, hbase-5966.patch
Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test
rigging. Below is a representative error, can be easily reproduced with:
{noformat}
mvn -PlocalTests -Psecurity \
-Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
clean test \
-Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
{noformat}
And the result:
{noformat}
---
T E S T S
---
Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec
FAILURE!
---
Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
---
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec
FAILURE!
testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)
Time elapsed: 21.935 sec ERROR!
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at
[jira] [Commented] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273973#comment-13273973 ] Hudson commented on HBASE-5385: --- Integrated in HBase-TRUNK #2876 (See [https://builds.apache.org/job/HBase-TRUNK/2876/]) HBASE-5385 Delete table/column should delete stored permissions on -acl- table (Matteo Bertozi) (Revision 1337512) Result = FAILURE tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java Delete table/column should delete stored permissions on -acl- table - Key: HBASE-5385 URL: https://issues.apache.org/jira/browse/HBASE-5385 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, HBASE-5385-v2.patch, HBASE-5385-v3.patch Deleting the table or a column does not cascade to the stored permissions at the -acl- table. We should also remove those permissions, otherwise, it can be a security leak, where freshly created tables contain permissions from previous same-named tables. We might also want to ensure, upon table creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273997#comment-13273997 ] nkeywal commented on HBASE-5992: It's org.apache.hadoop.hbase.TestDrainingServer. I fixed the issue, I'm retesting before uploading the patch. Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-5342: --- Attachment: HBASE-5342-0.94.patch HBASE-5342-0.92.patch the 0.92/0.94 version Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-0.92.patch, HBASE-5342-0.94.patch, HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[
https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274022#comment-13274022
]
Matteo Bertozzi commented on HBASE-5342:
{code}
Running org.apache.hadoop.hbase.security.token.TestTokenAuthentication
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.058 sec
Running org.apache.hadoop.hbase.security.token.TestZKSecretWatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.702 sec
Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
{code}
{code}
Running org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.706 sec
Running org.apache.hadoop.hbase.security.access.TestAccessControlFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.621 sec
Running org.apache.hadoop.hbase.security.access.TestAccessController
Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.041 sec
Running org.apache.hadoop.hbase.security.access.TestTablePermissions
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.577 sec
Results :
Tests run: 28, Failures: 0, Errors: 0, Skipped: 0
{code}
Grant/Revoke global permissions
---
Key: HBASE-5342
URL: https://issues.apache.org/jira/browse/HBASE-5342
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Matteo Bertozzi
Attachments: HBASE-5342-0.92.patch, HBASE-5342-0.94.patch,
HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch,
HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch,
HBASE-5342-v5.patch
HBASE-3025 introduced simple ACLs based on coprocessors. It defines
global/table/cf/cq level permissions. However, there is no way to
grant/revoke global level permissions, other than the hbase.superuser conf
setting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions
[ https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274023#comment-13274023 ] Hadoop QA commented on HBASE-5342: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526640/HBASE-5342-0.94.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1861//console This message is automatically generated. Grant/Revoke global permissions --- Key: HBASE-5342 URL: https://issues.apache.org/jira/browse/HBASE-5342 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Matteo Bertozzi Attachments: HBASE-5342-0.92.patch, HBASE-5342-0.94.patch, HBASE-5342-draft.patch, HBASE-5342-v0.patch, HBASE-5342-v1.patch, HBASE-5342-v2.patch, HBASE-5342-v3.patch, HBASE-5342-v4.patch, HBASE-5342-v5.patch HBASE-3025 introduced simple ACLs based on coprocessors. It defines global/table/cf/cq level permissions. However, there is no way to grant/revoke global level permissions, other than the hbase.superuser conf setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5984) TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274031#comment-13274031
]
Andrew Purtell commented on HBASE-5984:
---
As long as we can test and assure that HBase survives a rolling DN restart.
I'm not sure how common that scenario is but we've had to do it twice: once to
fix a DN bug, once to fix a configuration mistake.
TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
Key: HBASE-5984
URL: https://issues.apache.org/jira/browse/HBASE-5984
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: hbase_5984.patch
java.io.IOException: Cannot obtain block length for
LocatedBlock{BP-1455809779-127.0.0.1-1336670196362:blk_-6960847342982670493_1028;
getBlockSize()=1474; corrupt=false; offset=0; locs=[127.0.0.1:58343,
127.0.0.1:48427]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:232)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:177)
at
org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:119)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:112)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:928)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:212)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:75)
at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1768)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.openFile(SequenceFileLogReader.java:66)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1688)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1709)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.init(SequenceFileLogReader.java:58)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:166)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:659)
at
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly,
[jira] [Commented] (HBASE-5959) Add other load balancers
[
https://issues.apache.org/jira/browse/HBASE-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274041#comment-13274041
]
Zhihong Yu commented on HBASE-5959:
---
For computeTableSkewLoadCost(), normally we use region count (per table) for
region servers to judge skew.
I feel the skew in table count is not a good indicator.
{code}
+ * @return Cost of imbalance in table.
{code}
'table' - 'tables'
{code}
+for (EntryServerName, ListHRegionInfo entry : clusterState.entrySet())
{
+ SetString tablesOnRegionServer = new HashSetString();
{code}
We can clear the HashSet after the following line in the loop (instead of
creating HashSet every iteration):
{code}
+ tablesPerServer.add(new Double(tablesOnRegionServer.size()));
{code}
Math.abs() is not needed in the following:
{code}
+ skewCost += Math.abs(numTables - count.doubleValue());
{code}
For computeDataLocalityCost():
{code}
+// If we can't find where the data is getTopBlock returns null.
+// so count that as being the best possible.
+if (dataOnServers == null) {
{code}
I think worst should be assumed if block locations cannot be retrieved.
I asked the coefficients of various costs to be configurable. This is because
users tend to give high priority to locality cost component.
BalancerTestBase.java is missing license.
{code}
+ * Class used to be the base of unit tests on load balancers. It gives helper
{code}
'Class used to be the base' - 'Base class'.
assertClusterAsBalanced() should be package-private.
For reconcile(),
{code}
+ * This assumes the RegionPlan HSI instances are the same ones in the map, so
+ * actually no need to even pass in the map, but I think it's clearer.
{code}
Please expand HSI to full class name. The two parameters are both Lists. Where
is the map ? I think it is MapServerName, ServerAndLoad map.
Please describe what this method does.
Add other load balancers
Key: HBASE-5959
URL: https://issues.apache.org/jira/browse/HBASE-5959
Project: HBase
Issue Type: New Feature
Components: master
Affects Versions: 0.96.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-5959-0.patch, HBASE-5959-1.patch,
HBASE-5959-2.patch, HBASE-5959-3.patch
Now that balancers are pluggable we should give some options.b
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5990: --- Status: Open (was: Patch Available) TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5993) Add a no-read Append
Jacques created HBASE-5993: -- Summary: Add a no-read Append Key: HBASE-5993 URL: https://issues.apache.org/jira/browse/HBASE-5993 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.94.0 Reporter: Jacques HBASE-4102 added an atomic append. For high performance situations, it would be helpful to be able to do appends that don't actually require a read of the existing value. This would be useful in building a growing set of values. Our original use case was for implementing a form of search in HBase where a cell would contain a list of document ids associated with a particular keyword for search. However it seems like it would also be useful to provide substantial performance improvements for most Append scenarios. Within the client API, the simplest way to implement this would be to leverage the existing Append api. If the Append is marked as setReturnResults(false), use this code path. If result return is requested, use the existing Append implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5990: --- Attachment: hbase_5990_v2.patch TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5990: --- Hadoop Flags: Incompatible change Status: Patch Available (was: Open) TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274048#comment-13274048 ] Jimmy Xiang commented on HBASE-5990: How about patch 2? I added a . to the end of the message. So we are sure how to parse it no matter how hadoop/platform changes. TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274049#comment-13274049 ] Jimmy Xiang commented on HBASE-5990: It is not backward compatible. Fortunately, 0.96 is a singularity. TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274052#comment-13274052 ] Hadoop QA commented on HBASE-5990: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526645/hbase_5990_v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 31 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1862//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1862//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1862//console This message is automatically generated. TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5992: --- Status: Open (was: Patch Available) Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5992: --- Attachment: 5992.v11.patch Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v11.patch, 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5992: --- Status: Patch Available (was: Open) Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v11.patch, 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5994) Document and add to TableMapReduceUtil setup if security is enabled
Andrew Purtell created HBASE-5994:
-
Summary: Document and add to TableMapReduceUtil setup if security
is enabled
Key: HBASE-5994
URL: https://issues.apache.org/jira/browse/HBASE-5994
Project: HBase
Issue Type: Improvement
Reporter: Andrew Purtell
Assignee: Andrew Purtell
If secure RPC is enabled, then users must enable the TokenProvider coprocessor.
They also must acquire a job token and add it to the job configuration, by way
of something like:
{code}
if (User.isSecurityEnabled()) try {
User.getCurrent().obtainAuthTokenForJob(conf, job);
} catch (InterruptedException ie) {
Thread.interrupted();
}
{code}
Add this to the manual and update TableMapReduceUtil.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5959) Add other load balancers
[ https://issues.apache.org/jira/browse/HBASE-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274057#comment-13274057 ] Elliott Clark commented on HBASE-5959: -- I'll put the next version up on a review board. Add other load balancers Key: HBASE-5959 URL: https://issues.apache.org/jira/browse/HBASE-5959 Project: HBase Issue Type: New Feature Components: master Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-5959-0.patch, HBASE-5959-1.patch, HBASE-5959-2.patch, HBASE-5959-3.patch Now that balancers are pluggable we should give some options.b -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274061#comment-13274061 ] Hadoop QA commented on HBASE-5992: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526647/5992.v11.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 31 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1863//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1863//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1863//console This message is automatically generated. Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v11.patch, 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5991) Introduce sequential ZNode based read/write locks
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Feinberg reassigned HBASE-5991: Assignee: Alex Feinberg Introduce sequential ZNode based read/write locks -- Key: HBASE-5991 URL: https://issues.apache.org/jira/browse/HBASE-5991 Project: HBase Issue Type: Improvement Reporter: Alex Feinberg Assignee: Alex Feinberg This is a continuation of HBASE-5494: Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274070#comment-13274070 ] Zhihong Yu commented on HBASE-5992: --- I looped TestDrainingServer 4 times. TestSplitLogManager passes locally on MacBook. Integrated to trunk. Thanks for the patch, N. Thanks for the review, Stack. Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v11.patch, 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5990: - Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) Status: Resolved (was: Patch Available) Lets try it. Applied to TRUNK. Thanks for the patch Jimmy TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5993) Add a no-read Append
[ https://issues.apache.org/jira/browse/HBASE-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5993: - Priority: Critical (was: Major) Others have asked for this. Making critical so it gets consideration. Add a no-read Append Key: HBASE-5993 URL: https://issues.apache.org/jira/browse/HBASE-5993 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.94.0 Reporter: Jacques Priority: Critical HBASE-4102 added an atomic append. For high performance situations, it would be helpful to be able to do appends that don't actually require a read of the existing value. This would be useful in building a growing set of values. Our original use case was for implementing a form of search in HBase where a cell would contain a list of document ids associated with a particular keyword for search. However it seems like it would also be useful to provide substantial performance improvements for most Append scenarios. Within the client API, the simplest way to implement this would be to leverage the existing Append api. If the Append is marked as setReturnResults(false), use this code path. If result return is requested, use the existing Append implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5984) TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-5984.
--
Resolution: Fixed
Hadoop Flags: Reviewed
Applied to trunk. Thanks for the patch Jimmy.
I made a blocker on 0.96.0, HBASE-5995, to fix and reenable this test.
TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
Key: HBASE-5984
URL: https://issues.apache.org/jira/browse/HBASE-5984
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: hbase_5984.patch
java.io.IOException: Cannot obtain block length for
LocatedBlock{BP-1455809779-127.0.0.1-1336670196362:blk_-6960847342982670493_1028;
getBlockSize()=1474; corrupt=false; offset=0; locs=[127.0.0.1:58343,
127.0.0.1:48427]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:232)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:177)
at
org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:119)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:112)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:928)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:212)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:75)
at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1768)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.openFile(SequenceFileLogReader.java:66)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1688)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1709)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.init(SequenceFileLogReader.java:58)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:166)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:659)
at
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
[jira] [Created] (HBASE-5995) Fix and reenable TestLogRolling.testLogRollOnPipelineRestart
stack created HBASE-5995: Summary: Fix and reenable TestLogRolling.testLogRollOnPipelineRestart Key: HBASE-5995 URL: https://issues.apache.org/jira/browse/HBASE-5995 Project: HBase Issue Type: Task Components: test Affects Versions: 0.96.0 Reporter: stack Priority: Blocker Fix For: 0.96.0 HBASE-5984 disabled this flakey test (See the issue for more). This issue is about getting it enabled again. Made a blocker on 0.96.0 so it gets attention. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274081#comment-13274081 ] Hudson commented on HBASE-5992: --- Integrated in HBase-TRUNK #2881 (See [https://builds.apache.org/job/HBase-TRUNK/2881/]) HBASE-5992 Generalization of region move implementation + manage draining servers in bulk assign (N Keywal) (Revision 1337641) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v11.patch, 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5806) Handle split region related failures on master restart and RS restart
[
https://issues.apache.org/jira/browse/HBASE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274082#comment-13274082
]
stack commented on HBASE-5806:
--
+1 on on v3.
For the next time, not meant for this patch, Chinna, you could write the
below...
{code}
+ case RS_ZK_REGION_SPLITTING:
+LOG.debug(Processed region in state : + et);
+break;
+ case RS_ZK_REGION_SPLIT:
+LOG.debug(Processed region in state : + et);
+break;
{code}
as
{code}
+ case RS_ZK_REGION_SPLITTING:
+ case RS_ZK_REGION_SPLIT:
+LOG.debug(Processed region in state : + et);
+break;
{code}
..but lets get v3 in. Good stuff.
Handle split region related failures on master restart and RS restart
-
Key: HBASE-5806
URL: https://issues.apache.org/jira/browse/HBASE-5806
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: Chinna Rao Lalam
Fix For: 0.92.2, 0.96.0, 0.94.1
Attachments: HBASE-5806.patch, HBASE-5806_0.92.patch,
HBASE-5806_0.94.patch, HBASE-5806_0.94_1.patch, HBASE-5806_0.94_2.patch,
HBASE-5806_trunk.patch, HBASE-5806_trunk_1.patch, HBASE-5806_trunk_2.patch,
HBASE-5806_trunk_3.patch
This issue is raised to solve issues that comes out of partial region split
happened and the region node in the ZK which is in RS_ZK_REGION_SPLITTING and
RS_ZK_REGION_SPLIT is not yet processed.
This also tries to address HBASE-5615.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5983) UI should not use InfoPort to list servers
[ https://issues.apache.org/jira/browse/HBASE-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274083#comment-13274083 ] stack commented on HBASE-5983: -- bq. When disabling the info port (setting it to -1) while running more than one RegionServer daemon on the same machine, the UI pages are getting confused/confusing. Doesn't setting the info port to -1 turn off the UI? If so, which UI are you referring to boss where stuff gets listed wrong? UI should not use InfoPort to list servers -- Key: HBASE-5983 URL: https://issues.apache.org/jira/browse/HBASE-5983 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0 Reporter: Lars George Priority: Minor When disabling the info port (setting it to -1) while running more than one RegionServer daemon on the same machine, the UI pages are getting confused/confusing. They list servers by hostname/info-port but since both instances use -1 it shows incoherent info. The UI should always use the hostname/rpc-port to list the servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274084#comment-13274084 ] stack commented on HBASE-5904: -- @Ram As I read it above, we are about backing out hbase-5155 Thats what your patch does David? If so, I'm +1 on applying it to 0.90. is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Fix For: 0.90.7 Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5904: - Priority: Blocker (was: Major) Fix Version/s: 0.90.7 Making this a blocker on 0.90.7 so we don't forget it. is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Priority: Blocker Fix For: 0.90.7 Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5981) Change the log level from FATAL to WARN when failed syncing or appending
[ https://issues.apache.org/jira/browse/HBASE-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274086#comment-13274086 ] stack commented on HBASE-5981: -- Shouldn't we be aborting around here if we are dropping edits? Can't write to hlog? Change the log level from FATAL to WARN when failed syncing or appending Key: HBASE-5981 URL: https://issues.apache.org/jira/browse/HBASE-5981 Project: HBase Issue Type: Bug Components: wal Reporter: chunhui shen Assignee: chunhui shen Attachments: HBASE-5981.patch LOG.fatal(Could not append. Requesting close of hlog, e); LOG.fatal(Could not sync. Requesting close of hlog, e); Since we don't abort regionserver here, should we change the log level from FATAL to WARN? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274090#comment-13274090 ] Hudson commented on HBASE-5990: --- Integrated in HBase-TRUNK #2882 (See [https://builds.apache.org/job/HBase-TRUNK/2882/]) HBASE-5990 TestHCM failed with Hadoop 2.0.0 (Revision 1337695) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/RegionMovedException.java TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5984) TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274089#comment-13274089
]
Hudson commented on HBASE-5984:
---
Integrated in HBase-TRUNK #2882 (See
[https://builds.apache.org/job/HBase-TRUNK/2882/])
HBASE-5984 TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP
2.0.0 (Revision 1337697)
Result = FAILURE
stack :
Files :
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
Key: HBASE-5984
URL: https://issues.apache.org/jira/browse/HBASE-5984
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: hbase_5984.patch
java.io.IOException: Cannot obtain block length for
LocatedBlock{BP-1455809779-127.0.0.1-1336670196362:blk_-6960847342982670493_1028;
getBlockSize()=1474; corrupt=false; offset=0; locs=[127.0.0.1:58343,
127.0.0.1:48427]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:232)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:177)
at
org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:119)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:112)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:928)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:212)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:75)
at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1768)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.openFile(SequenceFileLogReader.java:66)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1688)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1709)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.init(SequenceFileLogReader.java:58)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:166)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:659)
at
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
[jira] [Created] (HBASE-5996) Improve multiPut/multiDelete by moving HLog.append and updateTimestamp out of the updateLock.readLock.lock()/unlock() functionality
Amitanand Aiyer created HBASE-5996: -- Summary: Improve multiPut/multiDelete by moving HLog.append and updateTimestamp out of the updateLock.readLock.lock()/unlock() functionality Key: HBASE-5996 URL: https://issues.apache.org/jira/browse/HBASE-5996 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.89.20100924, 0.94.0, 0.89-fb Reporter: Amitanand Aiyer Assignee: Amitanand Aiyer Priority: Minor whenever we do a batchMutateWithLocks in HRegion, we get the HRegion.updateLock.readLock() ... My understanding is that we need the updateLock.readLock only to protect the updates to the memStore. (i) HLog.append() has its own serialization/locking using HLog.updateLock We can move this out of the HRegion.updateLock lock grabbing. (ii) updating the timestamp for deltes and puts, can also be done before grabbing the lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274099#comment-13274099 ] Hudson commented on HBASE-5992: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #3 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/3/]) HBASE-5992 Generalization of region move implementation + manage draining servers in bulk assign (N Keywal) (Revision 1337641) Result = FAILURE tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v11.patch, 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5984) TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274098#comment-13274098
]
Hudson commented on HBASE-5984:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #3 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/3/])
HBASE-5984 TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP
2.0.0 (Revision 1337697)
Result = FAILURE
stack :
Files :
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
TestLogRolling.testLogRollOnPipelineRestart failed with HADOOP 2.0.0
Key: HBASE-5984
URL: https://issues.apache.org/jira/browse/HBASE-5984
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: hbase_5984.patch
java.io.IOException: Cannot obtain block length for
LocatedBlock{BP-1455809779-127.0.0.1-1336670196362:blk_-6960847342982670493_1028;
getBlockSize()=1474; corrupt=false; offset=0; locs=[127.0.0.1:58343,
127.0.0.1:48427]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:232)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:177)
at
org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:119)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:112)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:928)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:212)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:75)
at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1768)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.openFile(SequenceFileLogReader.java:66)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1688)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1709)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.init(SequenceFileLogReader.java:58)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:166)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:659)
at
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
[jira] [Commented] (HBASE-5990) TestHCM failed with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274100#comment-13274100 ] Hudson commented on HBASE-5990: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #3 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/3/]) HBASE-5990 TestHCM failed with Hadoop 2.0.0 (Revision 1337695) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/RegionMovedException.java TestHCM failed with Hadoop 2.0.0 Key: HBASE-5990 URL: https://issues.apache.org/jira/browse/HBASE-5990 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5990.patch, hbase_5990_v2.patch Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.742 sec FAILURE! Failed tests: testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274101#comment-13274101 ] David S. Wang commented on HBASE-5904: -- Stack: That (backing out HBASE-5155 only) is what my patch does. is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Priority: Blocker Fix For: 0.90.7 Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5992) Generalization of region move implementation + manage draining servers in bulk assign
[ https://issues.apache.org/jira/browse/HBASE-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5992: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Generalization of region move implementation + manage draining servers in bulk assign - Key: HBASE-5992 URL: https://issues.apache.org/jira/browse/HBASE-5992 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5992.v11.patch, 5992.v2.patch, 5992.v5.patch, 5992.v5.patch The region move implementation now has now a similar behavior whatever the destination server is specified or not. This allows: - to benefit from the improvement in HBASE-5877 - as a side effect to have the coprocessors calls when the destination server is not specified This includes various fixes around draining servers. Draining servers were not excluded during a bulk assign. This is now fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5453) Switch on-disk formats (reference files, HFile meta fields, etc) to PB
[ https://issues.apache.org/jira/browse/HBASE-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274159#comment-13274159 ] Jimmy Xiang commented on HBASE-5453: v2 looks good to me. Switch on-disk formats (reference files, HFile meta fields, etc) to PB -- Key: HBASE-5453 URL: https://issues.apache.org/jira/browse/HBASE-5453 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Todd Lipcon Assignee: stack Attachments: 5453.txt, 5453v2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
