[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694515#comment-13694515
]
stack commented on HBASE-8799:
--
[~andrew.purt...@gmail.com] Mighty Andrew, I added debug and it dumped this:
java.lang.AssertionError: Expected action to pass for user 'rwuser' but was
denied: org.apache.hadoop.hbase.exceptions.AccessDeniedException:
org.apache.hadoop.hbase.exceptions.AccessDeniedException: Insufficient
permissions (user=rwuser, scope=testBulkLoad, family=, action=CREATE)
here for the failing bulk load (Here is example:
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/508/testReport/org.apache.hadoop.hbase.security.access/TestAccessController/testBulkLoad/)
Is the fix giving rwuser CREATE perms? Should the rwuser have this perm?
Thanks.
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[ https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8737: - Attachment: 8737v3.txt Patch that allows CellScanner to optionally carry HeapSize. If present, we'll use the size for sizing buffer creating cellblock. Also changed private method name whose task is putting Cells into a cellBlock from createCellBlock to instead be jailer (From Gary Helmling). [replication] Change replication RPC to use cell blocks --- Key: HBASE-8737 URL: https://issues.apache.org/jira/browse/HBASE-8737 Project: HBase Issue Type: Improvement Components: Replication Reporter: Chris Trezzo Assignee: stack Priority: Critical Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt, 8737v3.txt Currently, the replication rpc that ships edits simply dumps the byte value of WAL edit key/value pairs into a protobuf message. Modify the replication rpc mechanism to use cell blocks so it can leverage encoding and compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8811) REST service ignores misspelled check= parameter, causing unexpected mutations
[
https://issues.apache.org/jira/browse/HBASE-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694528#comment-13694528
]
Hudson commented on HBASE-8811:
---
Integrated in hbase-0.95 #272 (See
[https://builds.apache.org/job/hbase-0.95/272/])
HBASE-8811. REST service ignores misspelled 'check' parameter, causing
unexpected mutations (Chip Salzenberg and Andrew Purtell) (Revision 1497210)
Result = FAILURE
apurtell :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/TestRowResource.java
REST service ignores misspelled check= parameter, causing unexpected
mutations
Key: HBASE-8811
URL: https://issues.apache.org/jira/browse/HBASE-8811
Project: HBase
Issue Type: Bug
Components: REST
Affects Versions: 0.98.0, 0.95.2, 0.94.9
Reporter: Chip Salzenberg
Assignee: Chip Salzenberg
Priority: Critical
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: 8811-0.94.patch, 8811.patch
In rest.RowResource.update(), this code keeps executing a request if a
misspelled check= parameter is provided.
{noformat}
if (CHECK_PUT.equalsIgnoreCase(check)) {
return checkAndPut(model);
} else if (CHECK_DELETE.equalsIgnoreCase(check)) {
return checkAndDelete(model);
} else if (check != null check.length() 0) {
LOG.warn(Unknown check value: + check + , ignored);
}
{noformat}
By my reading of the code, this results in the provided cell value that was
intended as a check instead being treated as a mutation, which is sure to
destroy user data. Thus the priority of this bug, as it can cause corruption.
I suggest that a better reaction than a warning would be, approximately:
{noformat}
return Response.status(Response.Status.BAD_REQUEST)
.type(MIMETYPE_TEXT).entity(Invalid check value ' + check + ')
.build();
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8780) a column Family can have VERSIONS less than zero
[
https://issues.apache.org/jira/browse/HBASE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694536#comment-13694536
]
Demai Ni commented on HBASE-8780:
-
Anoop, thanks. for some reason, when I followed the 'test results' links above,
a 404 error popped up. anyway, I am currently traveling around and have very
limited access to my code and testing environment. I will look into this once
back to office, in 2~3 weeks. sorry for the delay
a column Family can have VERSIONS less than zero
-
Key: HBASE-8780
URL: https://issues.apache.org/jira/browse/HBASE-8780
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.94.8
Reporter: Demai Ni
Assignee: Demai Ni
Priority: Trivial
Attachments: HBASE-8780-0.94.8-v0.patch, HBASE-8780-test.txt,
HBASE-8780-trunk.patch
User can create/alter a columnfam and set its VERSION(aka maxVERSIONS) to a
negative or zero value. Although there is a checking in
HColumnDesciptor#construtor, hbase shell command will invoke the
setter(setMaxVersions and setMinVersions) directly, hence by pass the
checking. For example:
{code:title=set VERSIONS = -1}
hbase(main):016:0 create 't5_dn',{NAME='cf1',VERSIONS=-1}
0 row(s) in 1.0420 seconds
hbase(main):017:0 put 't5_dn','row1','cf1:q1','row1cf1_v1'
0 row(s) in 0.0700 seconds
hbase(main):018:0 scan 't5_dn'
ROW COLUMN+CELL
0 row(s) in 0.0090 seconds
hbase(main):019:0 describe 't5_dn'
DESCRIPTION ENABLED
't5_dn', {NAME = 'cf1', REPLICATION_SCOPE = '0', true
KEEP_DELETED_CELLS = 'false', COMPRESSION = 'NONE
', ENCODE_ON_DISK = 'true', BLOCKCACHE = 'true',
MIN_VERSIONS = '0', DATA_BLOCK_ENCODING = 'NONE',
IN_MEMORY = 'false', BLOOMFILTER = 'NONE', TTL =
'2147483647', VERSIONS = '-1', BLOCKSIZE = '655
36'}
1 row(s) in 0.0410 seconds
{code}
above example shows VERSIONS = '-1', and put/scan doesn't keep the data
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[
https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694542#comment-13694542
]
Hadoop QA commented on HBASE-8737:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589843/8737v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.replication.TestReplicationQueueFailover
org.apache.hadoop.hbase.replication.TestMasterReplication
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.security.access.TestAccessController
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6154//console
This message is automatically generated.
[replication] Change replication RPC to use cell blocks
---
Key: HBASE-8737
URL: https://issues.apache.org/jira/browse/HBASE-8737
Project: HBase
Issue Type: Improvement
Components: Replication
Reporter: Chris Trezzo
Assignee: stack
Priority: Critical
Fix For: 0.98.0, 0.95.2
Attachments:
0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt,
8737v3.txt
Currently, the replication rpc that ships edits simply dumps the byte value
of WAL edit key/value pairs into a protobuf message.
Modify the replication rpc mechanism to use cell blocks so it can leverage
encoding and compression.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6295) Possible performance improvement in client batch operations: presplit and send in background
[
https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-6295:
---
Attachment: 6295.addendum.patch
Possible performance improvement in client batch operations: presplit and
send in background
Key: HBASE-6295
URL: https://issues.apache.org/jira/browse/HBASE-6295
Project: HBase
Issue Type: Improvement
Components: Client, Performance
Affects Versions: 0.95.2
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Labels: noob
Fix For: 0.98.0, 0.95.2
Attachments: 6295.addendum.patch, 6295.v11.patch, 6295.v12.patch,
6295.v14.patch, 6295.v15.patch, 6295.v1.patch, 6295.v2.patch, 6295.v3.patch,
6295.v4.patch, 6295.v5.patch, 6295.v6.patch, 6295.v8.patch, 6295.v9.patch
today batch algo is:
{noformat}
for Operation o: ListOp{
add o to todolist
if todolist maxsize or o last in list
split todolist per location
send split lists to region servers
clear todolist
wait
}
{noformat}
We could:
- create immediately the final object instead of an intermediate array
- split per location immediately
- instead of sending when the list as a whole is full, send it when there is
enough data for a single location
It would be:
{noformat}
for Operation o: ListOp{
get location
add o to todo location.todolist
if (location.todolist maxLocationSize)
send location.todolist to region server
clear location.todolist
// don't wait, continue the loop
}
send remaining
wait
{noformat}
It's not trivial to write if you add error management: retried list must be
shared with the operations added in the todolist. But it's doable.
It's interesting mainly for 'big' writes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6295) Possible performance improvement in client batch operations: presplit and send in background
[
https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694560#comment-13694560
]
Nicolas Liochon commented on HBASE-6295:
The tests worked all the time with a setting of 30. I had issues with 14,
that's the value in the common xml. The default in HConstant is 20. As the
right values are worked out in another jira, I've just went for 20 in this
patch, it seems to be ok on a small sample. I think we should get rid of the
one in hbase-server as well...
I will commit today if there is no objections.
Possible performance improvement in client batch operations: presplit and
send in background
Key: HBASE-6295
URL: https://issues.apache.org/jira/browse/HBASE-6295
Project: HBase
Issue Type: Improvement
Components: Client, Performance
Affects Versions: 0.95.2
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Labels: noob
Fix For: 0.98.0, 0.95.2
Attachments: 6295.addendum.patch, 6295.v11.patch, 6295.v12.patch,
6295.v14.patch, 6295.v15.patch, 6295.v1.patch, 6295.v2.patch, 6295.v3.patch,
6295.v4.patch, 6295.v5.patch, 6295.v6.patch, 6295.v8.patch, 6295.v9.patch
today batch algo is:
{noformat}
for Operation o: ListOp{
add o to todolist
if todolist maxsize or o last in list
split todolist per location
send split lists to region servers
clear todolist
wait
}
{noformat}
We could:
- create immediately the final object instead of an intermediate array
- split per location immediately
- instead of sending when the list as a whole is full, send it when there is
enough data for a single location
It would be:
{noformat}
for Operation o: ListOp{
get location
add o to todo location.todolist
if (location.todolist maxLocationSize)
send location.todolist to region server
clear location.todolist
// don't wait, continue the loop
}
send remaining
wait
{noformat}
It's not trivial to write if you add error management: retried list must be
shared with the operations added in the todolist. But it's doable.
It's interesting mainly for 'big' writes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8814) Possible NPE in split if a region has empty store files.
rajeshbabu created HBASE-8814:
-
Summary: Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
{code}
2013-06-27 14:12:54,472 INFO [RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile(HRegionFileSystem.java:539)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:610)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.access$1(SplitTransaction.java:607)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:633)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{code}
If a storefile is empty(can be because of puts and deletes) then first and
lastkey of the file will be empty. Then we will get first or last key as null.
Then we will end up in NPE when we will check splitkey in the range or not.
{code}
if (top) {
//check if larger than last key.
KeyValue splitKey = KeyValue.createFirstOnRow(splitRow);
byte[] lastKey = f.createReader().getLastKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0,
lastKey.length) 0) {
return null;
}
} else {
//check if smaller than first key
KeyValue splitKey = KeyValue.createLastOnRow(splitRow);
byte[] firstKey = f.createReader().getFirstKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0,
firstKey.length) 0) {
return null;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8814) Possible NPE in split if a region has empty store files.
[
https://issues.apache.org/jira/browse/HBASE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-8814:
--
Attachment: HBASE-8667_trunk.patch
Patch for trunk. Please review.
Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: HBASE-8814_trunk.patch
{code}
2013-06-27 14:12:54,472 INFO
[RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile(HRegionFileSystem.java:539)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:610)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.access$1(SplitTransaction.java:607)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:633)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{code}
If a storefile is empty(can be because of puts and deletes) then first and
lastkey of the file will be empty. Then we will get first or last key as
null. Then we will end up in NPE when we will check splitkey in the range or
not.
{code}
if (top) {
//check if larger than last key.
KeyValue splitKey = KeyValue.createFirstOnRow(splitRow);
byte[] lastKey = f.createReader().getLastKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0,
lastKey.length) 0) {
return null;
}
} else {
//check if smaller than first key
KeyValue splitKey = KeyValue.createLastOnRow(splitRow);
byte[] firstKey = f.createReader().getFirstKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0,
firstKey.length) 0) {
return null;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8814) Possible NPE in split if a region has empty store files.
[
https://issues.apache.org/jira/browse/HBASE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-8814:
--
Attachment: HBASE-8814_trunk.patch
Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: HBASE-8814_trunk.patch
{code}
2013-06-27 14:12:54,472 INFO
[RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile(HRegionFileSystem.java:539)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:610)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.access$1(SplitTransaction.java:607)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:633)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{code}
If a storefile is empty(can be because of puts and deletes) then first and
lastkey of the file will be empty. Then we will get first or last key as
null. Then we will end up in NPE when we will check splitkey in the range or
not.
{code}
if (top) {
//check if larger than last key.
KeyValue splitKey = KeyValue.createFirstOnRow(splitRow);
byte[] lastKey = f.createReader().getLastKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0,
lastKey.length) 0) {
return null;
}
} else {
//check if smaller than first key
KeyValue splitKey = KeyValue.createLastOnRow(splitRow);
byte[] firstKey = f.createReader().getFirstKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0,
firstKey.length) 0) {
return null;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8814) Possible NPE in split if a region has empty store files.
[
https://issues.apache.org/jira/browse/HBASE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-8814:
--
Attachment: (was: HBASE-8667_trunk.patch)
Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: HBASE-8814_trunk.patch
{code}
2013-06-27 14:12:54,472 INFO
[RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile(HRegionFileSystem.java:539)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:610)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.access$1(SplitTransaction.java:607)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:633)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{code}
If a storefile is empty(can be because of puts and deletes) then first and
lastkey of the file will be empty. Then we will get first or last key as
null. Then we will end up in NPE when we will check splitkey in the range or
not.
{code}
if (top) {
//check if larger than last key.
KeyValue splitKey = KeyValue.createFirstOnRow(splitRow);
byte[] lastKey = f.createReader().getLastKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0,
lastKey.length) 0) {
return null;
}
} else {
//check if smaller than first key
KeyValue splitKey = KeyValue.createLastOnRow(splitRow);
byte[] firstKey = f.createReader().getFirstKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0,
firstKey.length) 0) {
return null;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8814) Possible NPE in split if a region has empty store files.
[
https://issues.apache.org/jira/browse/HBASE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-8814:
--
Status: Patch Available (was: Open)
Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: HBASE-8814_trunk.patch
{code}
2013-06-27 14:12:54,472 INFO
[RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile(HRegionFileSystem.java:539)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:610)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.access$1(SplitTransaction.java:607)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:633)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{code}
If a storefile is empty(can be because of puts and deletes) then first and
lastkey of the file will be empty. Then we will get first or last key as
null. Then we will end up in NPE when we will check splitkey in the range or
not.
{code}
if (top) {
//check if larger than last key.
KeyValue splitKey = KeyValue.createFirstOnRow(splitRow);
byte[] lastKey = f.createReader().getLastKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0,
lastKey.length) 0) {
return null;
}
} else {
//check if smaller than first key
KeyValue splitKey = KeyValue.createLastOnRow(splitRow);
byte[] firstKey = f.createReader().getFirstKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0,
firstKey.length) 0) {
return null;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8814) Possible NPE in split if a region has empty store files.
[
https://issues.apache.org/jira/browse/HBASE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694599#comment-13694599
]
Hadoop QA commented on HBASE-8814:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589864/HBASE-8814_trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6155//console
This message is automatically generated.
Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: HBASE-8814_trunk.patch
{code}
2013-06-27 14:12:54,472 INFO
[RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
[jira] [Commented] (HBASE-8814) Possible NPE in split if a region has empty store files.
[
https://issues.apache.org/jira/browse/HBASE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694608#comment-13694608
]
rajeshbabu commented on HBASE-8814:
---
Test failure(TestAccessController.testBulkLoad) is not related to this issue.
Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: HBASE-8814_trunk.patch
{code}
2013-06-27 14:12:54,472 INFO
[RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile(HRegionFileSystem.java:539)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:610)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.access$1(SplitTransaction.java:607)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:633)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{code}
If a storefile is empty(can be because of puts and deletes) then first and
lastkey of the file will be empty. Then we will get first or last key as
null. Then we will end up in NPE when we will check splitkey in the range or
not.
{code}
if (top) {
//check if larger than last key.
KeyValue splitKey = KeyValue.createFirstOnRow(splitRow);
byte[] lastKey = f.createReader().getLastKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0,
lastKey.length) 0) {
return null;
}
} else {
//check if smaller than first key
KeyValue splitKey = KeyValue.createLastOnRow(splitRow);
byte[] firstKey = f.createReader().getFirstKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0,
firstKey.length) 0) {
return null;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8814) Possible NPE in split if a region has empty store files.
[
https://issues.apache.org/jira/browse/HBASE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694620#comment-13694620
]
Anoop Sam John commented on HBASE-8814:
---
The store file(which u refer) is a result of a major compaction where all the
puts are removed by later deletes right? Does the test case ensures this?
Possible NPE in split if a region has empty store files.
Key: HBASE-8814
URL: https://issues.apache.org/jira/browse/HBASE-8814
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: HBASE-8814_trunk.patch
{code}
2013-06-27 14:12:54,472 INFO
[RS:1;BLRY2R009039160:49833-splits-1372322572806]
regionserver.SplitRequest(92): Running rollback/cleanup of failed split of
testSplitShouldNotThrowNPEEvenARegionHasEmptySplitFiles,,1372322556662.276e00da1420119e2f91f3a4c4c41d78.;
java.util.concurrent.ExecutionException: java.lang.NullPointerException
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:602)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:297)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:466)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:596)
... 6 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile(HRegionFileSystem.java:539)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:610)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.access$1(SplitTransaction.java:607)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:633)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction$StoreFileSplitter.call(SplitTransaction.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
{code}
If a storefile is empty(can be because of puts and deletes) then first and
lastkey of the file will be empty. Then we will get first or last key as
null. Then we will end up in NPE when we will check splitkey in the range or
not.
{code}
if (top) {
//check if larger than last key.
KeyValue splitKey = KeyValue.createFirstOnRow(splitRow);
byte[] lastKey = f.createReader().getLastKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0,
lastKey.length) 0) {
return null;
}
} else {
//check if smaller than first key
KeyValue splitKey = KeyValue.createLastOnRow(splitRow);
byte[] firstKey = f.createReader().getFirstKey();
if (f.getReader().getComparator().compare(splitKey.getBuffer(),
splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0,
firstKey.length) 0) {
return null;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8811) REST service ignores misspelled check= parameter, causing unexpected mutations
[
https://issues.apache.org/jira/browse/HBASE-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694662#comment-13694662
]
Hudson commented on HBASE-8811:
---
Integrated in hbase-0.95-on-hadoop2 #151 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/151/])
HBASE-8811. REST service ignores misspelled 'check' parameter, causing
unexpected mutations (Chip Salzenberg and Andrew Purtell) (Revision 1497210)
Result = FAILURE
apurtell :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/TestRowResource.java
REST service ignores misspelled check= parameter, causing unexpected
mutations
Key: HBASE-8811
URL: https://issues.apache.org/jira/browse/HBASE-8811
Project: HBase
Issue Type: Bug
Components: REST
Affects Versions: 0.98.0, 0.95.2, 0.94.9
Reporter: Chip Salzenberg
Assignee: Chip Salzenberg
Priority: Critical
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: 8811-0.94.patch, 8811.patch
In rest.RowResource.update(), this code keeps executing a request if a
misspelled check= parameter is provided.
{noformat}
if (CHECK_PUT.equalsIgnoreCase(check)) {
return checkAndPut(model);
} else if (CHECK_DELETE.equalsIgnoreCase(check)) {
return checkAndDelete(model);
} else if (check != null check.length() 0) {
LOG.warn(Unknown check value: + check + , ignored);
}
{noformat}
By my reading of the code, this results in the provided cell value that was
intended as a check instead being treated as a mutation, which is sure to
destroy user data. Thus the priority of this bug, as it can cause corruption.
I suggest that a better reaction than a warning would be, approximately:
{noformat}
return Response.status(Response.Status.BAD_REQUEST)
.type(MIMETYPE_TEXT).entity(Invalid check value ' + check + ')
.build();
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8790) NullPointerException thrown when stopping regionserver
[ https://issues.apache.org/jira/browse/HBASE-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694663#comment-13694663 ] Hudson commented on HBASE-8790: --- Integrated in hbase-0.95-on-hadoop2 #151 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/151/]) HBASE-8790 NullPointerException thrown when stopping regionserver (Liang Xie) (Revision 1497172) Result = FAILURE tedyu : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java NullPointerException thrown when stopping regionserver -- Key: HBASE-8790 URL: https://issues.apache.org/jira/browse/HBASE-8790 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.95.1 Environment: CentOS 5.9 x86_64, java version 1.6.0_45, CDH4.3 Reporter: Xiong LIU Assignee: Liang Xie Fix For: 0.98.0, 0.95.2 Attachments: HBase-8790.txt The Hbase cluster is a fresh start with one regionserver. When we stop hbase, an unhandled NullPointerException is throwed in the regionserver. The regionserver's log is as follows: 2013-06-21 10:21:11,284 INFO [regionserver61020] regionserver.HRegionServer: Closing user regions 2013-06-21 10:21:14,288 DEBUG [regionserver61020] regionserver.HRegionServer: Waiting on 1028785192 2013-06-21 10:21:14,290 FATAL [regionserver61020] regionserver.HRegionServer: ABORTING region server HOSTNAME_TEST,61020,1371781086817 : Unhandled: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:988) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:832) at java.lang.Thread.run(Thread.java:662) 2013-06-21 10:21:14,292 FATAL [regionserver61020] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache .hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2013-06-21 10:21:14,293 INFO [regionserver61020] regionserver.HRegionServer: STOPPED: Unhandled: null 2013-06-21 10:21:14,293 INFO [regionserver61020] ipc.RpcServer: Stopping server on 61020 It seems that after closing user regions, the rssStub is null. update: we found that if setting hbase.client.ipc.pool.type to RoundRobinPool(or other pool type) and hbase.client.ipc.pool.size to 10(possibly other values) in hbase-site.xml, the regionserver is continuously attempting connect to master. and if we stop hbase, the above NullPointerException occurred. With hbase.client.ipc.pool.size set to 1, the cluster can be completely stopped. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8811) REST service ignores misspelled check= parameter, causing unexpected mutations
[
https://issues.apache.org/jira/browse/HBASE-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694675#comment-13694675
]
Hudson commented on HBASE-8811:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #586 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/586/])
HBASE-8811. REST service ignores misspelled 'check' parameter, causing
unexpected mutations (Chip Salzenberg and Andrew Purtell) (Revision 1497209)
Result = FAILURE
apurtell :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/rest/TestRowResource.java
REST service ignores misspelled check= parameter, causing unexpected
mutations
Key: HBASE-8811
URL: https://issues.apache.org/jira/browse/HBASE-8811
Project: HBase
Issue Type: Bug
Components: REST
Affects Versions: 0.98.0, 0.95.2, 0.94.9
Reporter: Chip Salzenberg
Assignee: Chip Salzenberg
Priority: Critical
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: 8811-0.94.patch, 8811.patch
In rest.RowResource.update(), this code keeps executing a request if a
misspelled check= parameter is provided.
{noformat}
if (CHECK_PUT.equalsIgnoreCase(check)) {
return checkAndPut(model);
} else if (CHECK_DELETE.equalsIgnoreCase(check)) {
return checkAndDelete(model);
} else if (check != null check.length() 0) {
LOG.warn(Unknown check value: + check + , ignored);
}
{noformat}
By my reading of the code, this results in the provided cell value that was
intended as a check instead being treated as a mutation, which is sure to
destroy user data. Thus the priority of this bug, as it can cause corruption.
I suggest that a better reaction than a warning would be, approximately:
{noformat}
return Response.status(Response.Status.BAD_REQUEST)
.type(MIMETYPE_TEXT).entity(Invalid check value ' + check + ')
.build();
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8790) NullPointerException thrown when stopping regionserver
[ https://issues.apache.org/jira/browse/HBASE-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694676#comment-13694676 ] Hudson commented on HBASE-8790: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #586 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/586/]) HBASE-8790 NullPointerException thrown when stopping regionserver (Liang Xie) (Revision 1497171) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java NullPointerException thrown when stopping regionserver -- Key: HBASE-8790 URL: https://issues.apache.org/jira/browse/HBASE-8790 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.95.1 Environment: CentOS 5.9 x86_64, java version 1.6.0_45, CDH4.3 Reporter: Xiong LIU Assignee: Liang Xie Fix For: 0.98.0, 0.95.2 Attachments: HBase-8790.txt The Hbase cluster is a fresh start with one regionserver. When we stop hbase, an unhandled NullPointerException is throwed in the regionserver. The regionserver's log is as follows: 2013-06-21 10:21:11,284 INFO [regionserver61020] regionserver.HRegionServer: Closing user regions 2013-06-21 10:21:14,288 DEBUG [regionserver61020] regionserver.HRegionServer: Waiting on 1028785192 2013-06-21 10:21:14,290 FATAL [regionserver61020] regionserver.HRegionServer: ABORTING region server HOSTNAME_TEST,61020,1371781086817 : Unhandled: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:988) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:832) at java.lang.Thread.run(Thread.java:662) 2013-06-21 10:21:14,292 FATAL [regionserver61020] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache .hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2013-06-21 10:21:14,293 INFO [regionserver61020] regionserver.HRegionServer: STOPPED: Unhandled: null 2013-06-21 10:21:14,293 INFO [regionserver61020] ipc.RpcServer: Stopping server on 61020 It seems that after closing user regions, the rssStub is null. update: we found that if setting hbase.client.ipc.pool.type to RoundRobinPool(or other pool type) and hbase.client.ipc.pool.size to 10(possibly other values) in hbase-site.xml, the regionserver is continuously attempting connect to master. and if we stop hbase, the above NullPointerException occurred. With hbase.client.ipc.pool.size set to 1, the cluster can be completely stopped. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6930) [89-fb] Avoid acquiring the same row lock repeatedly
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694681#comment-13694681 ] Dave Latham commented on HBASE-6930: Definitely confused here, along with Lars. Did the change describe (avoid acquiring the same row lock) happen somewhere else? Discussing something similar in HBASE-8806. [89-fb] Avoid acquiring the same row lock repeatedly Key: HBASE-6930 URL: https://issues.apache.org/jira/browse/HBASE-6930 Project: HBase Issue Type: Bug Reporter: Liyin Tang Attachments: D5841.1.patch, D5841.2.patch When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again. So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-8812: -- Summary: Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers (was: Avoid a wide line on the HMaster webUI if we have more zookeeper quorums) Fixing the title, HBase has only one ensemble (quorum was a misnomer) and an ensemble is made of many ZK servers. Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[ https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8737: - Attachment: 8737v4.txt Fix counting error [replication] Change replication RPC to use cell blocks --- Key: HBASE-8737 URL: https://issues.apache.org/jira/browse/HBASE-8737 Project: HBase Issue Type: Improvement Components: Replication Reporter: Chris Trezzo Assignee: stack Priority: Critical Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt, 8737v3.txt, 8737v4.txt Currently, the replication rpc that ships edits simply dumps the byte value of WAL edit key/value pairs into a protobuf message. Modify the replication rpc mechanism to use cell blocks so it can leverage encoding and compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like
[ https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694858#comment-13694858 ] Matt Corgan commented on HBASE-8496: A quick comment on encoding the tag lengths - I would recommend using a VInt rather than fixed 2 bytes. Most tags will be 128 or 256 bytes (depending on which VInt you choose), so will fit nicely into 1 byte. A 1 byte VInt is pretty trivial to decode, maybe as easy or easier than 2 bytes. Implement tags and the internals of how a tag should look like -- Key: HBASE-8496 URL: https://issues.apache.org/jira/browse/HBASE-8496 Project: HBase Issue Type: New Feature Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.0 Attachments: Tag design.pdf The intent of this JIRA comes from HBASE-7897. This would help us to decide on the structure and format of how the tags should look like. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8692) [AccessController] Restrict HTableDescriptor enumeration
[ https://issues.apache.org/jira/browse/HBASE-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694869#comment-13694869 ] stack commented on HBASE-8692: -- This looks to have broke TestAccessController. See http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/508/testReport/org.apache.hadoop.hbase.security.access/TestAccessController/testBulkLoad/ I added debug to the exception: Expected action to pass for user 'rwuser' but was denied: org.apache.hadoop.hbase.exceptions.AccessDeniedException: org.apache.hadoop.hbase.exceptions.AccessDeniedException: Insufficient permissions (user=rwuser, scope=testBulkLoad, family=, action=CREATE) at org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:351) at org.apache.hadoop.hbase.security.access.AccessController.preGetTableDescriptors(AccessController.java:1391) at org.apache.hadoop.hbase.master.MasterCoprocessorHost.preGetTableDescriptors(MasterCoprocessorHost.java:1125) at org.apache.hadoop.hbase.master.HMaster.getTableDescriptors(HMaster.java:2418) at org.apache.hadoop.hbase.protobuf.generated.MasterMonitorProtos$MasterMonitorService$2.callBlockingMethod(MasterMonitorProtos.java:2702) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122) at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829) The rwuser does not have the now required CREATE permission. The testBulkLoad has been failing solidly for a while now. I'll disable it for the moment till this addressed over in HBASE-8799 [AccessController] Restrict HTableDescriptor enumeration Key: HBASE-8692 URL: https://issues.apache.org/jira/browse/HBASE-8692 Project: HBase Issue Type: Improvement Components: Coprocessors, security Affects Versions: 0.98.0, 0.95.1, 0.94.9 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.98.0, 0.95.2, 0.94.9 Attachments: 8692-0.94.patch, 8692-0.94.patch, 8692-0.94.patch, 8692-0.94.patch, 8692.patch, 8692.patch, 8692.patch, 8692.patch Some users are concerned about having table schema exposed to every user and would like it protected, similar to the rest of the admin operations for schema. This used to be hopeless because META would leak HTableDescriptors in HRegionInfo, but that is no longer the case in 0.94+. Consider adding CP hooks in the master for intercepting HMasterInterface#getHTableDescriptors and HMasterInterface#getHTableDescriptors(ListString). Add support in the AccessController for only allowing GLOBAL ADMIN to the first method. Add support in the AccessController for allowing access to the descriptors for the table names in the list of the second method only if the user has TABLE ADMIN privilege for all of the listed table names. Then, fix the code in HBaseAdmin (and elsewhere) that expects to be able to enumerate all table descriptors e.g. in deleteTable. A TABLE ADMIN can delete a table but won’t have GLOBAL ADMIN privilege to enumerate the total list. So a minor fixup is needed here, and in other places like this which make the same assumption. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694872#comment-13694872
]
stack commented on HBASE-8799:
--
The commit of HBASE-8692 looks related requiring CREATE permissions. Going to
disable the testBulkLoad for now so tests can start passing again while we
figure the clash between HBASE-8692 and this.
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8799:
-
Attachment: 8799disableFailingTest.txt
Disabling the failing test for now.
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799disableFailingTest.txt, 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694874#comment-13694874
]
stack commented on HBASE-8799:
--
Committed the disabling patch to 0.95 and trunk.
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799disableFailingTest.txt, 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694878#comment-13694878
]
Hadoop QA commented on HBASE-8799:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589936/8799disableFailingTest.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6157//console
This message is automatically generated.
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799disableFailingTest.txt, 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8692) [AccessController] Restrict HTableDescriptor enumeration
[ https://issues.apache.org/jira/browse/HBASE-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694877#comment-13694877 ] stack commented on HBASE-8692: -- Here is another sample failure: http://54.241.6.143/job/HBase-0.95/518/org.apache.hbase$hbase-server/testReport/org.apache.hadoop.hbase.security.access/TestAccessController/testBulkLoad/ [AccessController] Restrict HTableDescriptor enumeration Key: HBASE-8692 URL: https://issues.apache.org/jira/browse/HBASE-8692 Project: HBase Issue Type: Improvement Components: Coprocessors, security Affects Versions: 0.98.0, 0.95.1, 0.94.9 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.98.0, 0.95.2, 0.94.9 Attachments: 8692-0.94.patch, 8692-0.94.patch, 8692-0.94.patch, 8692-0.94.patch, 8692.patch, 8692.patch, 8692.patch, 8692.patch Some users are concerned about having table schema exposed to every user and would like it protected, similar to the rest of the admin operations for schema. This used to be hopeless because META would leak HTableDescriptors in HRegionInfo, but that is no longer the case in 0.94+. Consider adding CP hooks in the master for intercepting HMasterInterface#getHTableDescriptors and HMasterInterface#getHTableDescriptors(ListString). Add support in the AccessController for only allowing GLOBAL ADMIN to the first method. Add support in the AccessController for allowing access to the descriptors for the table names in the list of the second method only if the user has TABLE ADMIN privilege for all of the listed table names. Then, fix the code in HBaseAdmin (and elsewhere) that expects to be able to enumerate all table descriptors e.g. in deleteTable. A TABLE ADMIN can delete a table but won’t have GLOBAL ADMIN privilege to enumerate the total list. So a minor fixup is needed here, and in other places like this which make the same assumption. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[ https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694884#comment-13694884 ] Varun Sharma commented on HBASE-8813: - Attached patch increasing to 64 seconds. Fix time b/w recoverLease invocations from HBASE 8449 - Key: HBASE-8813 URL: https://issues.apache.org/jira/browse/HBASE-8813 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.1 Reporter: Varun Sharma Attachments: 8813.patch The time b/w recover lease attempts is conservative but is still not correct. It does not factor in Datanode heartbeat time intervals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[ https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Sharma updated HBASE-8813: Attachment: 8813.patch Fix time b/w recoverLease invocations from HBASE 8449 - Key: HBASE-8813 URL: https://issues.apache.org/jira/browse/HBASE-8813 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.1 Reporter: Varun Sharma Attachments: 8813.patch The time b/w recover lease attempts is conservative but is still not correct. It does not factor in Datanode heartbeat time intervals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[ https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Sharma updated HBASE-8813: Assignee: Varun Sharma Status: Patch Available (was: Open) Fix time b/w recoverLease invocations from HBASE 8449 - Key: HBASE-8813 URL: https://issues.apache.org/jira/browse/HBASE-8813 Project: HBase Issue Type: Bug Affects Versions: 0.95.1, 0.98.0 Reporter: Varun Sharma Assignee: Varun Sharma Attachments: 8813.patch The time b/w recover lease attempts is conservative but is still not correct. It does not factor in Datanode heartbeat time intervals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8806) Row locks are acquired repeatedly in HRegion.doMiniBatchMutation for duplicate rows.
[
https://issues.apache.org/jira/browse/HBASE-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694888#comment-13694888
]
rahul gidwani commented on HBASE-8806:
--
while the logic is not wrong, i agree its not as clear as can be.
a few lines down you see
{code}
if (failedToAcquire) {
// We failed to grab another lock
assert !shouldBlock : Should never fail to get lock when blocking;
break; // stop acquiring more rows for this batch
}
{code}
but we shouldn't be adding to a set for no reason, I agree. I will fix this
and submit a new patch. I will also look at how this affects the throughput
and provide you guys with some results.
Row locks are acquired repeatedly in HRegion.doMiniBatchMutation for
duplicate rows.
Key: HBASE-8806
URL: https://issues.apache.org/jira/browse/HBASE-8806
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.5
Reporter: rahul gidwani
Fix For: 0.95.2, 0.94.10
Attachments: HBASE-8806-0.94.10.patch, HBASE-8806-0.94.10-v2.patch
If we already have the lock in the doMiniBatchMutation we don't need to
re-acquire it. The solution would be to keep a cache of the rowKeys already
locked for a miniBatchMutation and If we already have the
rowKey in the cache, we don't repeatedly try and acquire the lock. A fix to
this problem would be to keep a set of rows we already locked and not try to
acquire the lock for these rows.
We have tested this fix in our production environment and has improved
replication performance quite a bit. We saw a replication batch go from 3+
minutes to less than 10 seconds for batches with duplicate row keys.
{code}
static final int ACQUIRE_LOCK_COUNT = 0;
@Test
public void testRedundantRowKeys() throws Exception {
final int batchSize = 10;
String tableName = getClass().getSimpleName();
Configuration conf = HBaseConfiguration.create();
conf.setClass(HConstants.REGION_IMPL, MockHRegion.class, HeapSize.class);
MockHRegion region = (MockHRegion)
TestHRegion.initHRegion(Bytes.toBytes(tableName), tableName, conf,
Bytes.toBytes(a));
ListPairMutation, Integer someBatch = Lists.newArrayList();
int i = 0;
while (i batchSize) {
if (i % 2 == 0) {
someBatch.add(new PairMutation, Integer(new Put(Bytes.toBytes(0)),
null));
} else {
someBatch.add(new PairMutation, Integer(new Put(Bytes.toBytes(1)),
null));
}
i++;
}
long startTime = System.currentTimeMillis();
region.batchMutate(someBatch.toArray(new Pair[0]));
long endTime = System.currentTimeMillis();
long duration = endTime - startTime;
System.out.println(duration: + duration + ms);
assertEquals(2, ACQUIRE_LOCK_COUNT);
}
@Override
public Integer getLock(Integer lockid, byte[] row, boolean waitForLock)
throws IOException {
ACQUIRE_LOCK_COUNT++;
return super.getLock(lockid, row, waitForLock);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[ https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8813: - Resolution: Fixed Fix Version/s: 0.95.2 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.95 and to trunk. Thanks for the follow-up Varun. Fix time b/w recoverLease invocations from HBASE 8449 - Key: HBASE-8813 URL: https://issues.apache.org/jira/browse/HBASE-8813 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.1 Reporter: Varun Sharma Assignee: Varun Sharma Fix For: 0.98.0, 0.95.2 Attachments: 8813.patch The time b/w recover lease attempts is conservative but is still not correct. It does not factor in Datanode heartbeat time intervals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6930) [89-fb] Avoid acquiring the same row lock repeatedly
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HBASE-6930: -- Attachment: (was: D5841.2.patch) [89-fb] Avoid acquiring the same row lock repeatedly Key: HBASE-6930 URL: https://issues.apache.org/jira/browse/HBASE-6930 Project: HBase Issue Type: Bug Reporter: Liyin Tang Attachments: HBASE-6930.diff When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again. So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6930) [89-fb] Avoid acquiring the same row lock repeatedly
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HBASE-6930: -- Attachment: HBASE-6930.diff [89-fb] Avoid acquiring the same row lock repeatedly Key: HBASE-6930 URL: https://issues.apache.org/jira/browse/HBASE-6930 Project: HBase Issue Type: Bug Reporter: Liyin Tang Attachments: HBASE-6930.diff When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again. So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6930) [89-fb] Avoid acquiring the same row lock repeatedly
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694916#comment-13694916 ] Liyin Tang commented on HBASE-6930: --- The patch here seems to be attached to a wrong jira. Sorry about the confusion. I have re-attached the path here. [89-fb] Avoid acquiring the same row lock repeatedly Key: HBASE-6930 URL: https://issues.apache.org/jira/browse/HBASE-6930 Project: HBase Issue Type: Bug Reporter: Liyin Tang Attachments: HBASE-6930.diff When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again. So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[
https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694917#comment-13694917
]
Hadoop QA commented on HBASE-8737:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589929/8737v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6156//console
This message is automatically generated.
[replication] Change replication RPC to use cell blocks
---
Key: HBASE-8737
URL: https://issues.apache.org/jira/browse/HBASE-8737
Project: HBase
Issue Type: Improvement
Components: Replication
Reporter: Chris Trezzo
Assignee: stack
Priority: Critical
Fix For: 0.98.0, 0.95.2
Attachments:
0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt,
8737v3.txt, 8737v4.txt
Currently, the replication rpc that ships edits simply dumps the byte value
of WAL edit key/value pairs into a protobuf message.
Modify the replication rpc mechanism to use cell blocks so it can leverage
encoding and compression.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6930) [89-fb] Avoid acquiring the same row lock repeatedly
[ https://issues.apache.org/jira/browse/HBASE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HBASE-6930: -- Attachment: (was: D5841.1.patch) [89-fb] Avoid acquiring the same row lock repeatedly Key: HBASE-6930 URL: https://issues.apache.org/jira/browse/HBASE-6930 Project: HBase Issue Type: Bug Reporter: Liyin Tang Attachments: HBASE-6930.diff When processing the multiPut, multiMutations or multiDelete operations, each IPC handler thread tries to acquire a lock for each row key in these batches. If there are duplicated row keys in these batches, previously the IPC handler thread will repeatedly acquire the same row key again and again. So the optimization is to sort each batch operation based on the row key in the client side, and skip acquiring the same row lock repeatedly in the server side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like
[ https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694923#comment-13694923 ] stack commented on HBASE-8496: -- Add author, date, and JIRA you are referring to. Tags are going to be handled by cps and not be native? Or is thought that we do them as CP first and then pull them native? Why does hfile care about tags? It is not doing interpretation of the Cells it has? You don't want to add addTags to Mutation so you can do Put#addTags? We can do this later I suppose. If tags as attribute, maybe have a special key, something like, tags.visibility... so know which attributes are for inclusion in tags. Should you make a TaggedKeyValueCodec or you think it fine just making KeyValueCodec handle tags if present? So, pass a cellblock to hfile and then add codec metadata to hfile metadata? Hmm... I suppose using a codec breaks when you get to hfile because you have to do a kv at a time, unless you can do append(Codec). Besides, hfile is broke at the moment in that it only knows one kind of kv serialization and it is baked in everwhere. How you propose we fix this? Can we move hfile to be Cell based? Is it good adding tags at end of kv? I suppose it is fine for now getting them in. We can do an alternate serialization later, one that keeps tags close so we can spin through them fast w/o having to read value data. With KeyValuecodecs in place now we would issue two reads to read the total KV size and then read the entire byte array and then form the KVs. -- it has to be two reads? COuld we not write the block as a KeyValueCodec? And then in hfile metadata say what Codec class writing blocks was? We need to change hfile so it does Cells. HFileBlockEncoders also are broke in that they presume the KV serialization format. HFileBlockEncoders should be like the new Cell Codecs. You say All the encoders will need to understand the tags. Would be best for all if hfiles understood Cells. When you say additional 'read', do you mean a new seek? ...but we would not be able to use the KeyValueCodec.decoder() on the HFileBlocks. can we not make this happen? HFileBlockEncoders are broke. This would involve changes to the HFileBlockHeader. Could hfile blocks be cellblocks? And in meta data for hfile say what decoder to use? Thanks Ram. Implement tags and the internals of how a tag should look like -- Key: HBASE-8496 URL: https://issues.apache.org/jira/browse/HBASE-8496 Project: HBase Issue Type: New Feature Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.0 Attachments: Tag design.pdf The intent of this JIRA comes from HBASE-7897. This would help us to decide on the structure and format of how the tags should look like. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[ https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694927#comment-13694927 ] stack commented on HBASE-8737: -- TestAccessController should be fixed now (a an issue apart from this patch). Any chance of a review of this critical issue? Thanks. [replication] Change replication RPC to use cell blocks --- Key: HBASE-8737 URL: https://issues.apache.org/jira/browse/HBASE-8737 Project: HBase Issue Type: Improvement Components: Replication Reporter: Chris Trezzo Assignee: stack Priority: Critical Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt, 8737v3.txt, 8737v4.txt Currently, the replication rpc that ships edits simply dumps the byte value of WAL edit key/value pairs into a protobuf message. Modify the replication rpc mechanism to use cell blocks so it can leverage encoding and compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like
[ https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694928#comment-13694928 ] stack commented on HBASE-8496: -- I suppose bottom-line, can we not just do Cells rather than KV-with-tags? The same problems have to be solved (basically). If Cells, you'll get your tags carried along for you. Your CPs could then exploit them when present. The serialization could be your proposed KV-with-tags-on-the-end but when we touch hfile, can we pass cellblocks and read cellblocks (encoding and/or compression could be metadata in hfile -- I suppose this means one encoding/compression per file which is probably fine. Compressed/encoded blocks in memory would have to be accessed w/ CellScanner... would need resettable CellScanner though I suppose. Implement tags and the internals of how a tag should look like -- Key: HBASE-8496 URL: https://issues.apache.org/jira/browse/HBASE-8496 Project: HBase Issue Type: New Feature Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.0 Attachments: Tag design.pdf The intent of this JIRA comes from HBASE-7897. This would help us to decide on the structure and format of how the tags should look like. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[
https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694938#comment-13694938
]
Hadoop QA commented on HBASE-8813:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589937/8813.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6158//console
This message is automatically generated.
Fix time b/w recoverLease invocations from HBASE 8449
-
Key: HBASE-8813
URL: https://issues.apache.org/jira/browse/HBASE-8813
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.1
Reporter: Varun Sharma
Assignee: Varun Sharma
Fix For: 0.98.0, 0.95.2
Attachments: 8813.patch
The time b/w recover lease attempts is conservative but is still not correct.
It does not factor in Datanode heartbeat time intervals.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark reassigned HBASE-8810: Assignee: Elliott Clark Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694948#comment-13694948 ] Elliott Clark commented on HBASE-8810: -- Thanks I'll add those to my patch. Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Reporter: Elliott Clark After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like
[ https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695002#comment-13695002 ] Andrew Purtell commented on HBASE-8496: --- My read is tags are in core but the example users are coprocessors. Except for the operation attribute part, but that's just the other side of the story: if a cp is consuming tags presumably it is setting/managing them too. To what extent are we comfortable changing the client API to be tags aware? In 0.94 too? bq. Could hfile blocks be cellblocks? And in meta data for hfile say what decoder to use? I think it makes sense to have one codec for cells, or transitionally cells as two distinct serializers - keyvalue+tags, leaving the keyvalue class alone for the most part. We have keyvaluecodec in both trunk/0.95 and 0.94. Right now its only used for WALs. Should extend this to use for HFile blocks too - one common codec for keyvalue/cell serialization dealing with stuff like tags. So agreed hfile block encoding should change at least to incorporate this new common interface. If we have leeway to change how HFile blocks are constructed, maybe we can bump HFile minor and try more than just cell by cell. Could pack tags together at the block level (and even dedup/share) if we can go this far with the kind of changes people would be comfortable with in trunk *and* 0.94. Implement tags and the internals of how a tag should look like -- Key: HBASE-8496 URL: https://issues.apache.org/jira/browse/HBASE-8496 Project: HBase Issue Type: New Feature Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.0 Attachments: Tag design.pdf The intent of this JIRA comes from HBASE-7897. This would help us to decide on the structure and format of how the tags should look like. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8772) Separate Replication from HBase RegionServer process
[ https://issues.apache.org/jira/browse/HBASE-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695004#comment-13695004 ] Sameer Vaishampayan commented on HBASE-8772: In my testing I have seen replication slow down significantly on several unrelated things (to replication). Observed itfor compactions, major compactions, gc in regionservers. More nuanced is that the replication balancing of files is based on the regionserver's regions. Maybe I am not putting it correctly, but heres the scenario I see, in a cluster of 16 data nodes, couple of them have active region servers and are seeing pileup of to-replicate-files in zkdump. 14 other nodes are sitting idle saying nothing to replicate. surely the cluster as a whole has much greater capability to replicate. Maybe this replication rebalancing is a separate feature but it surely could be separated out from balancing regions. As after the pileup has occured even balancing regions does little good, right away. If only I could reassign/move the to-replicate files to idle machines. Separate Replication from HBase RegionServer process Key: HBASE-8772 URL: https://issues.apache.org/jira/browse/HBASE-8772 Project: HBase Issue Type: New Feature Components: regionserver, Replication Reporter: Sameer Vaishampayan Labels: performance Replication is a separate functionality than managing regions and should be able to be managed separately as a service rather than rolled into RegionServer. Load on RegionServer, gc etc shouldn't affect the replication service. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8501) The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string
[ https://issues.apache.org/jira/browse/HBASE-8501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8501: - Attachment: 8501.txt Patch that reformats our ensemble String so instead of server1,server2,server3:port... it is instead server1:port,server2:port, etc., as zk expects in its connectString. The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string Key: HBASE-8501 URL: https://issues.apache.org/jira/browse/HBASE-8501 Project: HBase Issue Type: Bug Components: Admin, Usability, Zookeeper Affects Versions: 0.94.2, 0.94.3, 0.95.0 Reporter: Bene Guo Fix For: 0.95.2 Attachments: 8501.txt The ZooKeeperMainServerArg return String like this:IP1,IP2,IP3:port, but zookeeper client connection String shoud like this :IP1:port,IP1:port,IP1:port, if changed the zookeeper client port, the zkcli can't connection the first and the second ip. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8501) The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string
[ https://issues.apache.org/jira/browse/HBASE-8501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-8501: Assignee: stack The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string Key: HBASE-8501 URL: https://issues.apache.org/jira/browse/HBASE-8501 Project: HBase Issue Type: Bug Components: Admin, Usability, Zookeeper Affects Versions: 0.94.2, 0.94.3, 0.95.0 Reporter: Bene Guo Assignee: stack Fix For: 0.95.2 Attachments: 8501.txt The ZooKeeperMainServerArg return String like this:IP1,IP2,IP3:port, but zookeeper client connection String shoud like this :IP1:port,IP1:port,IP1:port, if changed the zookeeper client port, the zkcli can't connection the first and the second ip. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8501) The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string
[ https://issues.apache.org/jira/browse/HBASE-8501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8501: - Release Note: Formats the ensemble string we pass to the zookeeper constructor so instead of server1,server2,server3:port, it is instead server1:port,server2:port, etc., as zk expects in its connectString parameter. Status: Patch Available (was: Open) The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string Key: HBASE-8501 URL: https://issues.apache.org/jira/browse/HBASE-8501 Project: HBase Issue Type: Bug Components: Admin, Usability, Zookeeper Affects Versions: 0.95.0, 0.94.3, 0.94.2 Reporter: Bene Guo Fix For: 0.95.2 Attachments: 8501.txt The ZooKeeperMainServerArg return String like this:IP1,IP2,IP3:port, but zookeeper client connection String shoud like this :IP1:port,IP1:port,IP1:port, if changed the zookeeper client port, the zkcli can't connection the first and the second ip. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8692) [AccessController] Restrict HTableDescriptor enumeration
[ https://issues.apache.org/jira/browse/HBASE-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695029#comment-13695029 ] Andrew Purtell commented on HBASE-8692: --- I saw that failure on trunk without the htabledescriptor change. Doesn't matter will fix the failing case regardless. [AccessController] Restrict HTableDescriptor enumeration Key: HBASE-8692 URL: https://issues.apache.org/jira/browse/HBASE-8692 Project: HBase Issue Type: Improvement Components: Coprocessors, security Affects Versions: 0.98.0, 0.95.1, 0.94.9 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.98.0, 0.95.2, 0.94.9 Attachments: 8692-0.94.patch, 8692-0.94.patch, 8692-0.94.patch, 8692-0.94.patch, 8692.patch, 8692.patch, 8692.patch, 8692.patch Some users are concerned about having table schema exposed to every user and would like it protected, similar to the rest of the admin operations for schema. This used to be hopeless because META would leak HTableDescriptors in HRegionInfo, but that is no longer the case in 0.94+. Consider adding CP hooks in the master for intercepting HMasterInterface#getHTableDescriptors and HMasterInterface#getHTableDescriptors(ListString). Add support in the AccessController for only allowing GLOBAL ADMIN to the first method. Add support in the AccessController for allowing access to the descriptors for the table names in the list of the second method only if the user has TABLE ADMIN privilege for all of the listed table names. Then, fix the code in HBaseAdmin (and elsewhere) that expects to be able to enumerate all table descriptors e.g. in deleteTable. A TABLE ADMIN can delete a table but won’t have GLOBAL ADMIN privilege to enumerate the total list. So a minor fixup is needed here, and in other places like this which make the same assumption. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695035#comment-13695035
]
Andrew Purtell commented on HBASE-8799:
---
I saw this fail on vanilla trunk before I applied the htabledescriptor change.
The CREATE detail is interesting. I suppose it could have been a dirty
workspace. In any event I have it on my to do list to fix this issue.
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799disableFailingTest.txt, 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695033#comment-13695033
]
Hudson commented on HBASE-8799:
---
Integrated in hbase-0.95 #273 (See
[https://builds.apache.org/job/hbase-0.95/273/])
HBASE-8799 TestAccessController#testBulkLoad has been failing for some time
on trunk/0.95; DISABLING FALING TEST FOR NOW (Revision 1497453)
Result = SUCCESS
stack :
Files :
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799disableFailingTest.txt, 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[ https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695032#comment-13695032 ] Hudson commented on HBASE-8813: --- Integrated in hbase-0.95 #273 (See [https://builds.apache.org/job/hbase-0.95/273/]) HBASE-8813 Fix time b/w recoverLease invocations from HBASE 8449 (Revision 1497464) Result = SUCCESS stack : Files : * /hbase/branches/0.95/hbase-common/src/main/resources/hbase-default.xml Fix time b/w recoverLease invocations from HBASE 8449 - Key: HBASE-8813 URL: https://issues.apache.org/jira/browse/HBASE-8813 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.1 Reporter: Varun Sharma Assignee: Varun Sharma Fix For: 0.98.0, 0.95.2 Attachments: 8813.patch The time b/w recover lease attempts is conservative but is still not correct. It does not factor in Datanode heartbeat time intervals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8815) A replicated cross cluster client
Varun Sharma created HBASE-8815: --- Summary: A replicated cross cluster client Key: HBASE-8815 URL: https://issues.apache.org/jira/browse/HBASE-8815 Project: HBase Issue Type: New Feature Reporter: Varun Sharma I would like to float this idea for brain storming. HBase is a strongly consistent system modelled after bigtable which means a machine going down results in loss of availability of around 2 minutes as it stands today. So there is a trade off. However, for high availability and redundancy, it is common practice for online/mission critical applications to run replicated clusters. For example, we run replicated clusters at pinterest in different EC2 az(s) and at google, critical data is always replicated across bigtable cells. At high volumes, 2 minutes of downtime can also be critical, however, today our client does not make use of the fact, that there is an available slave replica cluster from which slightly inconsistent data can be read. It only reads from one cluster. When you have replication, it is a very common practice for reading from slave if the error rate from master is high. That is how, web sites serve data out of MySQL and survive machine failures by directing their reads to slave machines when the master goes down. I am sure folks love the strong consistency gaurantee from HBase, but I think that this way, we can make better use of the replica cluster, much in the same way people use MySQL slaves for reads. In case of regions going offline, it would be nice if, for the offline regions only (a small fraction), reads could be directed to the slave cluster. I know one company which follows this model. At Google, a replicated client api is used for reads which is able to farm reads to multiple clusters and also writes to multiple clusters depending on availability in case of Multi master replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like
[ https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695056#comment-13695056 ] stack commented on HBASE-8496: -- You fellas want this to work on 0.94? Otherwise, hfilev3! Go for it! Implement tags and the internals of how a tag should look like -- Key: HBASE-8496 URL: https://issues.apache.org/jira/browse/HBASE-8496 Project: HBase Issue Type: New Feature Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.0 Attachments: Tag design.pdf The intent of this JIRA comes from HBASE-7897. This would help us to decide on the structure and format of how the tags should look like. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like
[ https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695067#comment-13695067 ] Andrew Purtell commented on HBASE-8496: --- bq. You fellas want this to work on 0.94? Yes, in the sense for cell ACLs, visibility labels, and such we would like to share as much as possible between 0.94 (prod) and 0.95+ (future) Implement tags and the internals of how a tag should look like -- Key: HBASE-8496 URL: https://issues.apache.org/jira/browse/HBASE-8496 Project: HBase Issue Type: New Feature Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.0 Attachments: Tag design.pdf The intent of this JIRA comes from HBASE-7897. This would help us to decide on the structure and format of how the tags should look like. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8501) The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string
[
https://issues.apache.org/jira/browse/HBASE-8501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695079#comment-13695079
]
Hadoop QA commented on HBASE-8501:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589960/8501.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6159//console
This message is automatically generated.
The hbase zkcli will connection failure the first and the second ip from
ZooKeeperMainServerArg return connection string
Key: HBASE-8501
URL: https://issues.apache.org/jira/browse/HBASE-8501
Project: HBase
Issue Type: Bug
Components: Admin, Usability, Zookeeper
Affects Versions: 0.94.2, 0.94.3, 0.95.0
Reporter: Bene Guo
Assignee: stack
Fix For: 0.95.2
Attachments: 8501.txt
The ZooKeeperMainServerArg return String like this:IP1,IP2,IP3:port,
but zookeeper client connection String shoud like this
:IP1:port,IP1:port,IP1:port, if changed the zookeeper client
port, the zkcli can't connection the first and the second ip.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[ https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695085#comment-13695085 ] Jesse Yates commented on HBASE-8737: Mind throwing it up on RB? This fella is a bit big... [replication] Change replication RPC to use cell blocks --- Key: HBASE-8737 URL: https://issues.apache.org/jira/browse/HBASE-8737 Project: HBase Issue Type: Improvement Components: Replication Reporter: Chris Trezzo Assignee: stack Priority: Critical Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt, 8737v3.txt, 8737v4.txt Currently, the replication rpc that ships edits simply dumps the byte value of WAL edit key/value pairs into a protobuf message. Modify the replication rpc mechanism to use cell blocks so it can leverage encoding and compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[ https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695085#comment-13695085 ] Jesse Yates edited comment on HBASE-8737 at 6/27/13 9:52 PM: - Mind throwing it up on RB? This fella is a bit big... But off the bat, it looks pretty good (minus some minor cruft :) was (Author: jesse_yates): Mind throwing it up on RB? This fella is a bit big... [replication] Change replication RPC to use cell blocks --- Key: HBASE-8737 URL: https://issues.apache.org/jira/browse/HBASE-8737 Project: HBase Issue Type: Improvement Components: Replication Reporter: Chris Trezzo Assignee: stack Priority: Critical Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt, 8737v3.txt, 8737v4.txt Currently, the replication rpc that ships edits simply dumps the byte value of WAL edit key/value pairs into a protobuf message. Modify the replication rpc mechanism to use cell blocks so it can leverage encoding and compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8501) The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string
[ https://issues.apache.org/jira/browse/HBASE-8501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695102#comment-13695102 ] stack commented on HBASE-8501: -- Any chance of a review? Thanks. The hbase zkcli will connection failure the first and the second ip from ZooKeeperMainServerArg return connection string Key: HBASE-8501 URL: https://issues.apache.org/jira/browse/HBASE-8501 Project: HBase Issue Type: Bug Components: Admin, Usability, Zookeeper Affects Versions: 0.94.2, 0.94.3, 0.95.0 Reporter: Bene Guo Assignee: stack Fix For: 0.95.2 Attachments: 8501.txt The ZooKeeperMainServerArg return String like this:IP1,IP2,IP3:port, but zookeeper client connection String shoud like this :IP1:port,IP1:port,IP1:port, if changed the zookeeper client port, the zkcli can't connection the first and the second ip. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8737) [replication] Change replication RPC to use cell blocks
[ https://issues.apache.org/jira/browse/HBASE-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695101#comment-13695101 ] stack commented on HBASE-8737: -- [~jesse_yates] Pardon me. Here you go: https://reviews.apache.org/r/12142/ [replication] Change replication RPC to use cell blocks --- Key: HBASE-8737 URL: https://issues.apache.org/jira/browse/HBASE-8737 Project: HBase Issue Type: Improvement Components: Replication Reporter: Chris Trezzo Assignee: stack Priority: Critical Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8737-replication-Change-replication-RPC-to-use.patch, 8737.txt, 8737v3.txt, 8737v4.txt Currently, the replication rpc that ships edits simply dumps the byte value of WAL edit key/value pairs into a protobuf message. Modify the replication rpc mechanism to use cell blocks so it can leverage encoding and compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8774) Add BatchSize and Filter to Thrift2
[ https://issues.apache.org/jira/browse/HBASE-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695100#comment-13695100 ] Hamed Madani commented on HBASE-8774: - If you like me to use HBASE-6073 approach for this patch, I can modify the patch. Please let me know what you think. Add BatchSize and Filter to Thrift2 --- Key: HBASE-8774 URL: https://issues.apache.org/jira/browse/HBASE-8774 Project: HBase Issue Type: New Feature Components: Thrift Affects Versions: 0.95.1 Reporter: Hamed Madani Assignee: Hamed Madani Attachments: HBASE_8774.patch Attached Patch will add BatchSize and Filter support to Thrift2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7386) Investigate providing some supervisor support for znode deletion
[ https://issues.apache.org/jira/browse/HBASE-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7386: - Fix Version/s: 0.95.2 Assignee: stack (was: Gregory Chanan) Pulling into 0.95 because of discussion here: http://search-hadoop.com/m/PIMWWYRIuk1 Assigning myself. Investigate providing some supervisor support for znode deletion Key: HBASE-7386 URL: https://issues.apache.org/jira/browse/HBASE-7386 Project: HBase Issue Type: Task Components: master, regionserver, scripts Reporter: Gregory Chanan Assignee: stack Fix For: 0.95.2 Attachments: HBASE-7386-v0.patch, supervisordconfigs-v0.patch There a couple of JIRAs for deleting the znode on a process failure: HBASE-5844 (RS) HBASE-5926 (Master) which are pretty neat; on process failure, they delete the znode of the underlying process so HBase can recover faster. These JIRAs were implemented via the startup scripts; i.e. the script hangs around and waits for the process to exit, then deletes the znode. There are a few problems associated with this approach, as listed in the below JIRAs: 1) Hides startup output in script https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463401page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463401 2) two hbase processes listed per launched daemon https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463409page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463409 3) Not run by a real supervisor https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463409page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463409 4) Weird output after kill -9 actual process in standalone mode https://issues.apache.org/jira/browse/HBASE-5926?focusedCommentId=13506801page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13506801 5) Can kill existing RS if called again https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463401page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463401 6) Hides stdout/stderr[6] https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13506832page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13506832 I suspect running in via something like supervisor.d can solve these issues if we provide the right support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8434) Allow enabling hbase 8354 to support real lease recovery
[ https://issues.apache.org/jira/browse/HBASE-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695163#comment-13695163 ] stack commented on HBASE-8434: -- +1 on adding this patche's config. We should up the timeout too? It is 4 seconds in 0.94. Should we make it the 64 second change that was just made in trunk? int waitingPeriod = conf.getInt(hbase.lease.recovery.waiting.period, 4000); Allow enabling hbase 8354 to support real lease recovery Key: HBASE-8434 URL: https://issues.apache.org/jira/browse/HBASE-8434 Project: HBase Issue Type: Improvement Components: MTTR Reporter: Varun Sharma Assignee: Varun Sharma Fix For: 0.94.10 Attachments: 8434.patch Please see discussion in HBase 8389. For environments where lease recovery time can be bounded on the HDFS side through tight timeouts, provide a toggle for users who want the WAL splitting to continue only after the lease is truly recovered and the file is closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[ https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695173#comment-13695173 ] Hudson commented on HBASE-8813: --- Integrated in hbase-0.95-on-hadoop2 #152 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/152/]) HBASE-8813 Fix time b/w recoverLease invocations from HBASE 8449 (Revision 1497464) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-common/src/main/resources/hbase-default.xml Fix time b/w recoverLease invocations from HBASE 8449 - Key: HBASE-8813 URL: https://issues.apache.org/jira/browse/HBASE-8813 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.1 Reporter: Varun Sharma Assignee: Varun Sharma Fix For: 0.98.0, 0.95.2 Attachments: 8813.patch The time b/w recover lease attempts is conservative but is still not correct. It does not factor in Datanode heartbeat time intervals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695174#comment-13695174
]
Hudson commented on HBASE-8799:
---
Integrated in hbase-0.95-on-hadoop2 #152 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/152/])
HBASE-8799 TestAccessController#testBulkLoad has been failing for some time
on trunk/0.95; DISABLING FALING TEST FOR NOW (Revision 1497453)
Result = FAILURE
stack :
Files :
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799disableFailingTest.txt, 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8813) Fix time b/w recoverLease invocations from HBASE 8449
[ https://issues.apache.org/jira/browse/HBASE-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695205#comment-13695205 ] Hudson commented on HBASE-8813: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #587 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/587/]) HBASE-8813 Fix time b/w recoverLease invocations from HBASE 8449 (Revision 1497465) Result = FAILURE stack : Files : * /hbase/trunk/hbase-common/src/main/resources/hbase-default.xml Fix time b/w recoverLease invocations from HBASE 8449 - Key: HBASE-8813 URL: https://issues.apache.org/jira/browse/HBASE-8813 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.1 Reporter: Varun Sharma Assignee: Varun Sharma Fix For: 0.98.0, 0.95.2 Attachments: 8813.patch The time b/w recover lease attempts is conservative but is still not correct. It does not factor in Datanode heartbeat time intervals. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8799) TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
[
https://issues.apache.org/jira/browse/HBASE-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695206#comment-13695206
]
Hudson commented on HBASE-8799:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #587 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/587/])
HBASE-8799 TestAccessController#testBulkLoad has been failing for some time
on trunk/0.95; DISABLING FALING TEST FOR NOW (Revision 1497452)
Result = FAILURE
stack :
Files :
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
TestAccessController#testBulkLoad has been failing for some time on trunk/0.95
--
Key: HBASE-8799
URL: https://issues.apache.org/jira/browse/HBASE-8799
Project: HBase
Issue Type: Bug
Components: Coprocessors, security, test
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: stack
Fix For: 0.95.2
Attachments: 8799disableFailingTest.txt, 8799.txt
I've observed this in Jenkins reports and also while I was working on
HBASE-8692, only on trunk/0.95, not on 0.94:
{quote}
Failed tests:
testBulkLoad(org.apache.hadoop.hbase.security.access.TestAccessController):
Expected action to pass for user 'rwuser' but was denied
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8790) NullPointerException thrown when stopping regionserver
[ https://issues.apache.org/jira/browse/HBASE-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8790: -- Resolution: Fixed Status: Resolved (was: Patch Available) NullPointerException thrown when stopping regionserver -- Key: HBASE-8790 URL: https://issues.apache.org/jira/browse/HBASE-8790 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.95.1 Environment: CentOS 5.9 x86_64, java version 1.6.0_45, CDH4.3 Reporter: Xiong LIU Assignee: Liang Xie Fix For: 0.98.0, 0.95.2 Attachments: HBase-8790.txt The Hbase cluster is a fresh start with one regionserver. When we stop hbase, an unhandled NullPointerException is throwed in the regionserver. The regionserver's log is as follows: 2013-06-21 10:21:11,284 INFO [regionserver61020] regionserver.HRegionServer: Closing user regions 2013-06-21 10:21:14,288 DEBUG [regionserver61020] regionserver.HRegionServer: Waiting on 1028785192 2013-06-21 10:21:14,290 FATAL [regionserver61020] regionserver.HRegionServer: ABORTING region server HOSTNAME_TEST,61020,1371781086817 : Unhandled: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:988) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:832) at java.lang.Thread.run(Thread.java:662) 2013-06-21 10:21:14,292 FATAL [regionserver61020] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache .hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2013-06-21 10:21:14,293 INFO [regionserver61020] regionserver.HRegionServer: STOPPED: Unhandled: null 2013-06-21 10:21:14,293 INFO [regionserver61020] ipc.RpcServer: Stopping server on 61020 It seems that after closing user regions, the rssStub is null. update: we found that if setting hbase.client.ipc.pool.type to RoundRobinPool(or other pool type) and hbase.client.ipc.pool.size to 10(possibly other values) in hbase-site.xml, the regionserver is continuously attempting connect to master. and if we stop hbase, the above NullPointerException occurred. With hbase.client.ipc.pool.size set to 1, the cluster can be completely stopped. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8815) A replicated cross cluster client
[ https://issues.apache.org/jira/browse/HBASE-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695243#comment-13695243 ] Anoop Sam John commented on HBASE-8815: --- I am also working with similar item and was thinking on the possibilities.. A client layer on top of HTable can be made which can do the autoswitching of the peer.. I will look into this more next week and get back. (Have some more open Qs in mind :) ) A replicated cross cluster client - Key: HBASE-8815 URL: https://issues.apache.org/jira/browse/HBASE-8815 Project: HBase Issue Type: New Feature Reporter: Varun Sharma I would like to float this idea for brain storming. HBase is a strongly consistent system modelled after bigtable which means a machine going down results in loss of availability of around 2 minutes as it stands today. So there is a trade off. However, for high availability and redundancy, it is common practice for online/mission critical applications to run replicated clusters. For example, we run replicated clusters at pinterest in different EC2 az(s) and at google, critical data is always replicated across bigtable cells. At high volumes, 2 minutes of downtime can also be critical, however, today our client does not make use of the fact, that there is an available slave replica cluster from which slightly inconsistent data can be read. It only reads from one cluster. When you have replication, it is a very common practice for reading from slave if the error rate from master is high. That is how, web sites serve data out of MySQL and survive machine failures by directing their reads to slave machines when the master goes down. I am sure folks love the strong consistency gaurantee from HBase, but I think that this way, we can make better use of the replica cluster, much in the same way people use MySQL slaves for reads. In case of regions going offline, it would be nice if, for the offline regions only (a small fraction), reads could be directed to the slave cluster. I know one company which follows this model. At Google, a replicated client api is used for reads which is able to farm reads to multiple clusters and also writes to multiple clusters depending on availability in case of Multi master replication. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8816) Add support of loading multiple tables into LoadTestTool
Jeffrey Zhong created HBASE-8816: Summary: Add support of loading multiple tables into LoadTestTool Key: HBASE-8816 URL: https://issues.apache.org/jira/browse/HBASE-8816 Project: HBase Issue Type: Test Components: test Affects Versions: 0.94.9 Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Priority: Minor Introducing an optional parameter 'concurrent_factor' into LoadTestTool. When it's specified with positive integer n, LoadTestTool will load n tables parallely. -tn parameter value becomes table name prefix. Tables are created with name in format tn_1...tn_n. A sample command line -tn test -concurrent_factor 2 will create load tables:test_1 and test_2 The motivation is to add a handy way to load multiple tables concurrently. In addition, we could use this option to test resource leakage of long running clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8816) Add support of loading multiple tables into LoadTestTool
[ https://issues.apache.org/jira/browse/HBASE-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-8816: - Status: Patch Available (was: Open) Add support of loading multiple tables into LoadTestTool Key: HBASE-8816 URL: https://issues.apache.org/jira/browse/HBASE-8816 Project: HBase Issue Type: Test Components: test Affects Versions: 0.94.9 Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Priority: Minor Attachments: hbase-8816.patch Introducing an optional parameter 'concurrent_factor' into LoadTestTool. When it's specified with positive integer n, LoadTestTool will load n tables parallely. -tn parameter value becomes table name prefix. Tables are created with name in format tn_1...tn_n. A sample command line -tn test -concurrent_factor 2 will create load tables:test_1 and test_2 The motivation is to add a handy way to load multiple tables concurrently. In addition, we could use this option to test resource leakage of long running clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8816) Add support of loading multiple tables into LoadTestTool
[ https://issues.apache.org/jira/browse/HBASE-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-8816: - Attachment: hbase-8816.patch Add support of loading multiple tables into LoadTestTool Key: HBASE-8816 URL: https://issues.apache.org/jira/browse/HBASE-8816 Project: HBase Issue Type: Test Components: test Affects Versions: 0.94.9 Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Priority: Minor Attachments: hbase-8816.patch Introducing an optional parameter 'concurrent_factor' into LoadTestTool. When it's specified with positive integer n, LoadTestTool will load n tables parallely. -tn parameter value becomes table name prefix. Tables are created with name in format tn_1...tn_n. A sample command line -tn test -concurrent_factor 2 will create load tables:test_1 and test_2 The motivation is to add a handy way to load multiple tables concurrently. In addition, we could use this option to test resource leakage of long running clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8816) Add support of loading multiple tables into LoadTestTool
[
https://issues.apache.org/jira/browse/HBASE-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695258#comment-13695258
]
Hadoop QA commented on HBASE-8816:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589996/hbase-8816.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6160//console
This message is automatically generated.
Add support of loading multiple tables into LoadTestTool
Key: HBASE-8816
URL: https://issues.apache.org/jira/browse/HBASE-8816
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 0.94.9
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Priority: Minor
Attachments: hbase-8816.patch
Introducing an optional parameter 'concurrent_factor' into LoadTestTool. When
it's specified with positive integer n, LoadTestTool will load n tables
parallely. -tn parameter value becomes table name prefix. Tables are created
with name in format tn_1...tn_n. A sample command line -tn test
-concurrent_factor 2 will create load tables:test_1 and test_2
The motivation is to add a handy way to load multiple tables concurrently. In
addition, we could use this option to test resource leakage of long running
clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
