[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096609#comment-14096609
]
stack commented on HBASE-11657:
---
Yeah, we've been trying to shutdown notions of 'region' showing through in API.
We don't want to break API for 1.0. Mark new Interface Private audience too
instead of evolving as it is currently (IIRC)?
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
HBASE_11657_v3.patch, HBASE_11657_v3.patch, HBASE_11657_v4.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11736) Document SKIP_FLUSH snapshot option
[
https://issues.apache.org/jira/browse/HBASE-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096643#comment-14096643
]
Hadoop QA commented on HBASE-11736:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661613/HBASE-11736.patch
against trunk revision .
ATTACHMENT ID: 12661613
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.client.TestReplicaWithCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//console
This message is automatically generated.
Document SKIP_FLUSH snapshot option
---
Key: HBASE-11736
URL: https://issues.apache.org/jira/browse/HBASE-11736
Project: HBase
Issue Type: Sub-task
Components: documentation, snapshots
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Fix For: 0.99.0, 0.94.21, 0.98.4
Attachments: HBASE-11736.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096648#comment-14096648 ] Hudson commented on HBASE-11550: SUCCESS: Integrated in HBase-0.98 #447 (See [https://builds.apache.org/job/HBase-0.98/447/]) HBASE-11550 Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted (Gustavo Anatoly) (ndimiduk: rev 7e1e818bbc0bed59c12fbd8c003eaed702cf16ad) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted -- Key: HBASE-11550 URL: https://issues.apache.org/jira/browse/HBASE-11550 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 0.98.4, 0.98.5 Reporter: Ted Yu Assignee: Gustavo Anatoly Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, HBASE-11550.patch User can pass bucket sizes through hbase.bucketcache.bucket.sizes config entry. The sizes are supposed to be in increasing order. Validation should be added in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11739) Document blockCache contents report in the UI
[
https://issues.apache.org/jira/browse/HBASE-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096656#comment-14096656
]
Hadoop QA commented on HBASE-11739:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661628/HBASE-11739.patch
against trunk revision .
ATTACHMENT ID: 12661628
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testCloneAndRestoreSnapshot(TestRestoreSnapshotFromClient.java:255)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//console
This message is automatically generated.
Document blockCache contents report in the UI
-
Key: HBASE-11739
URL: https://issues.apache.org/jira/browse/HBASE-11739
Project: HBase
Issue Type: Sub-task
Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Fix For: 0.99.0
Attachments: HBASE-11739.patch, bc_basic.png, bc_config.png,
bc_l1.png, bc_l2_buckets.png, bc_stats.png
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11738) Document improvements to LoadTestTool and PerformanceEvaluation
[
https://issues.apache.org/jira/browse/HBASE-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096668#comment-14096668
]
Hadoop QA commented on HBASE-11738:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661643/HBABASE-11738.patch
against trunk revision .
ATTACHMENT ID: 12661643
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//console
This message is automatically generated.
Document improvements to LoadTestTool and PerformanceEvaluation
---
Key: HBASE-11738
URL: https://issues.apache.org/jira/browse/HBASE-11738
Project: HBase
Issue Type: Sub-task
Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Priority: Minor
Fix For: 0.99.0, 0.98.4
Attachments: HBABASE-11738.patch, HBASE-11738.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096676#comment-14096676 ] Lars Hofhansl commented on HBASE-11733: --- Nice! Could this have performance implications, especially for hooks into hot paths like preScannerNext(...), etc. We're creating one more object upon each invocation now. Avoid copy-paste in Master/Region CoprocessorHost - Key: HBASE-11733 URL: https://issues.apache.org/jira/browse/HBASE-11733 Project: HBase Issue Type: Bug Components: Coprocessors Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Trivial Fix For: 2.0.0 Attachments: HBASE-11733-v0.patch all the methods of MasterCoprocessorHost and RegionCoprocessorHost are copy-paste, and a couple of them have some missing fixes. The body of each function is the same, the only difference is the call to the specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096684#comment-14096684
]
Matt Corgan commented on HBASE-11728:
-
Just seeing this. Thank you for all the investigation, especially reducing it
to a unit test. I don't currently have an environment setup, but could
possibly debug this weekend if it's still a problem. My first guess is that
there is a bug in the seek method in the decoder, possibly seeking to the wrong
start position in the block.
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096686#comment-14096686
]
ramkrishna.s.vasudevan commented on HBASE-11728:
bq.seek method in the decoder, possibly seeking to the wrong start position in
the block
Should be. I will try it out till then and [~mcorgan] if you find the fix pls
feel free to update here.
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11438: --- Status: Open (was: Patch Available) [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-11438. Resolution: Fixed Fix Version/s: 2.0.0 1.0.0 Hadoop Flags: Reviewed [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11438: --- Attachment: HBASE-11438_v10.patch Patch that was committed to master, branch-1 and 0.98. Thanks for the reviews and discussion. Will raise a JIRA for documenting the usage of the new client API for quoting. [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11741) Document the how users can use the Cell Visibility and authorizations with special characters
ramkrishna.s.vasudevan created HBASE-11741: -- Summary: Document the how users can use the Cell Visibility and authorizations with special characters Key: HBASE-11741 URL: https://issues.apache.org/jira/browse/HBASE-11741 Project: HBase Issue Type: Task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11729) Document HFile v3
[
https://issues.apache.org/jira/browse/HBASE-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096703#comment-14096703
]
Hadoop QA commented on HBASE-11729:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661625/HBASE-11729.patch
against trunk revision .
ATTACHMENT ID: 12661625
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+paraAs we will be discussing changes to the HFile format, it is
useful to give a short overview of the original (HFile version 1) format./para
+ footnoteparaImage courtesy of Lars George, link
xlink:href=http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html;hbase-architecture-101-storage.html/link./para/footnote
+ paraThe number of entries in the block index is stored in the fixed file
trailer, and has to be passed in to the method that reads the block index. One
of the limitations of the block index in version 1 is that it does not provide
the compressed size of a block, which turns out to be necessary for
decompression. Therefore, the HFile reader has to infer this compressed size
from the offset difference between blocks. We fix this limitation in version 2,
where we store on-disk block size instead of uncompressed size, and get
uncompressed size from the block header./para
+ paraWe found it necessary to revise the HFile format after encountering
high memory usage and slow startup times caused by large Bloom filters and
block indexes in the region server. Bloom filters can get as large as 100 MB
per HFile, which adds up to 2 GB when aggregated over 20 regions. Block indexes
can grow as large as 6 GB in aggregate size over the same set of regions. A
region is not considered opened until all of its block index data is loaded.
Large Bloom filters produce a different performance problem: the first get
request that requires a Bloom filter lookup will incur the latency of loading
the entire Bloom filter bit array./para
+ paraTo speed up region server startup we break Bloom filters and block
indexes into multiple blocks and write those blocks out as they fill up, which
also reduces the HFile writer���s memory footprint. In the Bloom filter case,
���filling up a block��� means accumulating enough keys to efficiently utilize
a fixed-size bit array, and in the block index case we accumulate an ���index
block��� of the desired size. Bloom filter blocks and index blocks (we call
these ���inline blocks���) become interspersed with data blocks, and as a side
effect we can no longer rely on the difference between block offsets to
determine data block length, as it was done in version 1./para
+ paraHFile is a low-level file format by design, and it should not deal
with application-specific details such as Bloom filters, which are handled at
StoreFile level. Therefore, we call Bloom filter blocks in an HFile inline
blocks. We also supply HFile with an interface to write those inline blocks.
/para
+ paraAnother format modification aimed at reducing the region server
startup time is to use a contiguous ���load-on-open��� section that has to be
loaded in memory at the time an HFile is being opened. Currently, as an HFile
opens, there are separate seek operations to read the trailer, data/meta
indexes, and file info. To read the Bloom filter, there are two more seek
operations for its ���data��� and ���meta��� portions. In version 2, we seek
once to read the trailer and seek again to read everything else we need to open
the file from a contiguous block./para/section
+ paraThe version of HBase introducing the above features reads both
version 1 and 2 HFiles, but only writes version 2 HFiles. A version 2 HFile is
structured as follows:
+ para8 bytes: Block type, a sequence of bytes equivalent to version
1's magic records. Supported block types are: /para
+ INTERMEDIATE_INDEX ��� intermediate-level index blocks in
a multi-level blockindex
{color:green}+1 site{color}. The mvn site goal succeeds with
[jira] [Updated] (HBASE-11741) Document the how users can use the Cell Visibility and authorizations with special characters
[ https://issues.apache.org/jira/browse/HBASE-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11741: --- Component/s: documentation Document the how users can use the Cell Visibility and authorizations with special characters - Key: HBASE-11741 URL: https://issues.apache.org/jira/browse/HBASE-11741 Project: HBase Issue Type: Task Components: documentation Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0, 0.98.6 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11741) Document the how users can use the Cell Visibility and authorizations with special characters
[ https://issues.apache.org/jira/browse/HBASE-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11741: --- Affects Version/s: 0.98.0 Document the how users can use the Cell Visibility and authorizations with special characters - Key: HBASE-11741 URL: https://issues.apache.org/jira/browse/HBASE-11741 Project: HBase Issue Type: Task Components: documentation Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0, 0.98.6 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11741) Document the how users can use the Cell Visibility and authorizations with special characters
[ https://issues.apache.org/jira/browse/HBASE-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11741: --- Fix Version/s: 0.98.6 2.0.0 0.99.0 Document the how users can use the Cell Visibility and authorizations with special characters - Key: HBASE-11741 URL: https://issues.apache.org/jira/browse/HBASE-11741 Project: HBase Issue Type: Task Components: documentation Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0, 0.98.6 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096724#comment-14096724 ] Andrey Stepachev commented on HBASE-11689: -- Researched how it implemented now. We have a couple of options to do this task: # We can, as proposed in comments above, enhance MetaRegionServer with additional _optional_ field typed RegionState.State. But that solution has implications: #* we need to combat with possibly concurrent updates (RS and Master, both write to that znode) #* RS has to know what state to write into updated PB (read and versioned update, I suppose). # We can create new znode or hierarchy of znodes and store state for each meta region (only for one currently) and can restrict PB to have all fields _required_. I'd prefer to go with second solution, it is more cosine and provides good visibility and write paths (only Master writes to state and only RS writes to location znode). What do you think about that. Or there is another, better solution exists? Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Jimmy Xiang Assignee: Andrey Stepachev With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096739#comment-14096739 ] Hudson commented on HBASE-11550: FAILURE: Integrated in HBase-1.0 #100 (See [https://builds.apache.org/job/HBase-1.0/100/]) HBASE-11550 Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted (Gustavo Anatoly) (ndimiduk: rev cd59a023c4f84ba458b9665e1a12add2f702ad96) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted -- Key: HBASE-11550 URL: https://issues.apache.org/jira/browse/HBASE-11550 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 0.98.4, 0.98.5 Reporter: Ted Yu Assignee: Gustavo Anatoly Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, HBASE-11550.patch User can pass bucket sizes through hbase.bucketcache.bucket.sizes config entry. The sizes are supposed to be in increasing order. Validation should be added in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096741#comment-14096741
]
wuchengzhi commented on HBASE-11728:
I am doing more tests,i download the HFile to my localdisk, and using
HFileReaderV2 HFileScan for debugging
So I get some information,maybe it's unuseful ,and i want to say..
byte[] startRow = KeyValue.createFirstDeleteFamilyOnRow(a-b-A-1-.getBytes(),
Bytes.toBytes(cf_1)).getKey();
HFileScan.seekTo(startRow) ;
//a-b-A-1/cf_1:/LATEST_TIMESTAMP/DeleteFamily/vlen=0/mvcc=0
after done, ptSearcher.current() of this seeker is
a-b-0-0/cf_1:qf_1/1407908210357/Put/vlen=8/mvcc=0
and i call HFileScan.next() to get the prefix of a-b-A-1
then i got output as below :
a-b-A-1/cf_1:qf_1/1407908210411/Put/vlen=8/mvcc=0
a-b-A-1-1402329600-1402396277/cf_1:qf_2/1407908210445/Put/vlen=8/mvcc=0
a-b-A-1-1402397227-1402415999/cf_1:qf_2/1407908210494/Put/vlen=10/mvcc=0
Also,i got some debug info for PrefixTreeArrayScanner. before:advance();
seeker.current() is a-b-A-1/cf_1:qf_1/1407908210411/Put/vlen=8/mvcc=0
currentRowNode: fan:-,token:-1,numCells:1,fanIndex:-1
rowNodes:
[0]fan:0AB,token:a-b-,numCells:0,fanIndex:1(A)
[1]fan:-,token:-1,numCells:1,fanIndex:-1
and i try to modify the startRow to
a-b-A-1-/cf_1:/LATEST_TIMESTAMP/DeleteFamily/vlen=0/mvcc=0 and test again.
after done, ptSearcher.current() of this seeker is
a-b-A-1/cf_1:qf_1/1407908210411/Put/vlen=8/mvcc=0
but i call HfileScan.next(), i got nothing..
and some debug info for PrefixTreeArrayScanner. before:advance();
seeker.current() is a-b-A-1/cf_1:qf_1/1407908210411/Put/vlen=8/mvcc=0
fan:-,token:-1,numCells:1,fanIndex:0(-)
rowNodes:
[0]fan:0AB,token:a-b-,numCells:0,fanIndex:1(A)
[1]fan:-,token:-1,numCells:1,fanIndex:0(-)
[2]fan:29,token:14023,numCells:0,fanIndex:0(2)
[3]fan:,token:9600-1402396277,numCells:1,fanIndex:-1
and then after next(),the seeker.current() is
a-b-B-2-1402397300-1402416535/cf_1:qf_2/1407908210526/Put/vlen=10/mvcc=0
As the same location,but i got difference tree nodes? i think it's abnormal...
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the
[jira] [Updated] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuchengzhi updated HBASE-11728:
---
Attachment: 29cb562fad564b468ea9d61a2d60e8b0
HFileAnalys.java
analysis the storefile manually
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 29cb562fad564b468ea9d61a2d60e8b0, HFileAnalys.java,
TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11736) Document SKIP_FLUSH snapshot option
[
https://issues.apache.org/jira/browse/HBASE-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096745#comment-14096745
]
Matteo Bertozzi commented on HBASE-11736:
-
{quote}The default behavior is to require all regions of a table to coordinate
in order to
prevent writing to the table, and to perform a flush at the same time. This
causes the
table to be unavailable for writes while the snapshot is being taken.{quote}
There is no locking anywhere in snapshots.
Coordination is just the process of asking to each Region to perform the
snapshot operation and then report back once they are done.
The default behavior of snapshot is flushing (not locking), which means that
the data in the memstore at the moment of the snapshot operation on the region
will be included in the snapshot. but you still can write or read while the
snapshot operation is going on.
{quote}However, if your set-up can tolerate the possibility of some data not
being captured by the snapshot, you can use the optionSKIP_FLUSH/option
option of the snapshot command to disable locking and flushing while taking the
snapshot.{quote}
sort of ok, since there is no locking you don't really know which data is
present in the snapshot. A snapshot is an row-consistent image of what was in
the table between the time you asked for the snapshot and the completion. Skip
flush will avoid to flush the memstore which means the data not flushed yet
will not be included in the snapshot (you don't have any way to know what).
just emphasize that the only difference is the memstore not flushed
{quote}hbase snapshot 'mytable', 'snapshot123', 'SKIP_FLUSH'{quote}
That does not work, the correct one is:
{code}snapshot 'namespace:sourceTable', 'snapshotName', {SKIP_FLUSH =
true}{code}
{quote}Using this option can result in data lost if you try to restore a
snapshot which is missing data that was written while the snapshot was
taken.{quote}
can you reword this avoiding the can result in data lost. You don't lose any
data on restore.
Since there is no locking between the start and the end of the snapshot
operation, you can still insert data while the snapshot is in progress. The
snapshot is just a snapshot at some point in time, so the data inserted on a
particular region after the snapshot was taken is not in the snapshot. With the
skip flush not only the data inserted after the snapshot operation, but all the
data in the memstore is not available.
Document SKIP_FLUSH snapshot option
---
Key: HBASE-11736
URL: https://issues.apache.org/jira/browse/HBASE-11736
Project: HBase
Issue Type: Sub-task
Components: documentation, snapshots
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Fix For: 0.99.0, 0.94.21, 0.98.4
Attachments: HBASE-11736.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[ https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-10576: --- Status: In Progress (was: Patch Available) Didn't find time to work because of some internal assignments and local indexing integration to phoenix. Now I have time to do this. I have started working on this.. Custom load balancer to co-locate the regions of two tables which are having same split keys Key: HBASE-10576 URL: https://issues.apache.org/jira/browse/HBASE-10576 Project: HBase Issue Type: Sub-task Components: Balancer Reporter: rajeshbabu Assignee: rajeshbabu Attachments: HBASE-10536_v2.patch, HBASE-10576.patch To support local indexing both user table and index table should have same split keys. This issue to provide custom balancer to colocate the regions of two tables which are having same split keys. This helps in Phoenix as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[ https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096747#comment-14096747 ] rajeshbabu edited comment on HBASE-10576 at 8/14/14 8:44 AM: - Didn't find time to work on this issue because of some internal assignments and local indexing integration to phoenix. Now I have time to do this. I have started working on this.. was (Author: rajesh23): Didn't find time to work because of some internal assignments and local indexing integration to phoenix. Now I have time to do this. I have started working on this.. Custom load balancer to co-locate the regions of two tables which are having same split keys Key: HBASE-10576 URL: https://issues.apache.org/jira/browse/HBASE-10576 Project: HBase Issue Type: Sub-task Components: Balancer Reporter: rajeshbabu Assignee: rajeshbabu Attachments: HBASE-10536_v2.patch, HBASE-10576.patch To support local indexing both user table and index table should have same split keys. This issue to provide custom balancer to colocate the regions of two tables which are having same split keys. This helps in Phoenix as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11737) Document callQueue improvements from HBASE-11355
[
https://issues.apache.org/jira/browse/HBASE-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096753#comment-14096753
]
Matteo Bertozzi commented on HBASE-11737:
-
{quote}introduces several callQueue improvements, which can increase
performance. See the JIRA for some benchmarking information{quote}
Improvements seems something like on by default. We don't have anything on
by default.
Is more like new options to experiments with tunings.
ipc.server.callqueue. There was a jira that you documented that the options
were renamed in hbase.ipc...
for read.share see HBASE-11724 (in-progress), apparently the doc with 0, 0.5
and 1 is not clear enough.
overall this doc doesn't seems to add any value to what is already in
hbase-default.xml
I think that the doc should provide more detailed information on why increasing
that number is good or bad, what will be the result and so on. I'll try to come
up with something for you.
Document callQueue improvements from HBASE-11355
Key: HBASE-11737
URL: https://issues.apache.org/jira/browse/HBASE-11737
Project: HBase
Issue Type: Sub-task
Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Fix For: 0.99.0, 0.98.4
Attachments: HBASE-11737.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096757#comment-14096757
]
ramkrishna.s.vasudevan commented on HBASE-11728:
This is what is happening now
The trie structure formed for the above data is
{code}
B0 1 a-b-
L 1 2 0-0
N 1 2 A-1
B0 3 -14023
L 1 4 29600-1402396277
L 1 4 97227-1402415999
L 1 2 B-2-1402397300-1402416535
{code}
For the case where we add the column to the scan, the scan tries to find a kv
that is greater than
a-b-A-1,cf_1,qf_2.
So the seeker goes thro these row nodes
{code}
fan:0AB,token:a-b-,numCells:0,fanIndex:-1
fan:-,token:-1,numCells:1,fanIndex:-1
fan:29,token:14023,numCells:0,fanIndex:-1
fan:,token:9600-1402396277,numCells:1,fanIndex:-1
{code}
Finally this points to the required kv a-b-A-1-1402397227-1402415999, cf_1,
qf_2.
Since the scanner always tries to fetch the previous kv the seeker moves back
to the kv just before this following the below path
{code}
fan:29,token:14023,numCells:0,fanIndex:0(2)
fan:-,token:-1,numCells:1,fanIndex:0(-)
{code}
So the actual next() call should advance() and get the kv from where the
previous() was done. But in this case the path followed is this
{code}
fan:-,token:-1,numCells:1,fanIndex:0(-)
fan:0AB,token:a-b-,numCells:0,fanIndex:1(A)
{code}
And hence the next ends up in fetching the node that is directly under a-b- and
that is the kv a-b-B-2-1402397300-1402416535.
So from the above analysis it shows that while doing previous() the fanIndex
gets changed for the row node and hence while doing the actual next() the
traversal of the trie structure ends up in the other leaf node attached to it.
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 29cb562fad564b468ea9d61a2d60e8b0, HFileAnalys.java,
TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096760#comment-14096760
]
ramkrishna.s.vasudevan commented on HBASE-11728:
The method
{code}
protected void followCurrentFan() {
int currentFanPosition = currentRowNode.getFanIndex();
followFan(currentFanPosition);
}
{code}
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 29cb562fad564b468ea9d61a2d60e8b0, HFileAnalys.java,
TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11731) Add option to only run a subset of the shell tests
[ https://issues.apache.org/jira/browse/HBASE-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11731: Attachment: HBASE-11731.patch Here's an implementation that allows matching the RUnit test cases (which should correspond to the names of test classes. It turns out RUnit had a (somewhat non-obvious) way to do this already, so I just hooked into that. One downside is that we have to use ruby-style regex literals to do the wildcard matching. Since this might be surprising, I added text to the book's dev section on testing. Add option to only run a subset of the shell tests -- Key: HBASE-11731 URL: https://issues.apache.org/jira/browse/HBASE-11731 Project: HBase Issue Type: Improvement Components: build, shell Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Labels: beginner Attachments: HBASE-11731.patch Right now, contributors to the shell can limit testing to just the shell tests but there's no way to limit to just a subset of them. It would be nice if I could just run the test for the thing I'm changing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11731) Add option to only run a subset of the shell tests
[ https://issues.apache.org/jira/browse/HBASE-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096769#comment-14096769 ] Sean Busbey commented on HBASE-11731: - I verified by trying out the commands given in the new docs. Unfortunately, the only way I can find to get the output of the actual ruby code is to insert a test that fails so that stdout/stderr will be placed in the XML surefire report. With that alteration in place, I can verify both the total test count when the shell.test param isn't given as well as the correct counts for specifying a test and a wildcard of tests. Add option to only run a subset of the shell tests -- Key: HBASE-11731 URL: https://issues.apache.org/jira/browse/HBASE-11731 Project: HBase Issue Type: Improvement Components: build, shell Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Labels: beginner Attachments: HBASE-11731.patch Right now, contributors to the shell can limit testing to just the shell tests but there's no way to limit to just a subset of them. It would be nice if I could just run the test for the thing I'm changing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11731) Add option to only run a subset of the shell tests
[ https://issues.apache.org/jira/browse/HBASE-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11731: Status: Patch Available (was: Open) Add option to only run a subset of the shell tests -- Key: HBASE-11731 URL: https://issues.apache.org/jira/browse/HBASE-11731 Project: HBase Issue Type: Improvement Components: build, shell Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Labels: beginner Attachments: HBASE-11731.patch Right now, contributors to the shell can limit testing to just the shell tests but there's no way to limit to just a subset of them. It would be nice if I could just run the test for the thing I'm changing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096760#comment-14096760
]
ramkrishna.s.vasudevan edited comment on HBASE-11728 at 8/14/14 9:23 AM:
-
The method is never used
{code}
protected void followCurrentFan() {
int currentFanPosition = currentRowNode.getFanIndex();
followFan(currentFanPosition);
}
{code}
{Hit enter too soon}
was (Author: ram_krish):
The method
{code}
protected void followCurrentFan() {
int currentFanPosition = currentRowNode.getFanIndex();
followFan(currentFanPosition);
}
{code}
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 29cb562fad564b468ea9d61a2d60e8b0, HFileAnalys.java,
TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096783#comment-14096783 ] Hudson commented on HBASE-11438: FAILURE: Integrated in HBase-0.98 #448 (See [https://builds.apache.org/job/HBase-0.98/448/]) HBASE-11438 - [Visibility Controller] Support UTF8 character as Visibility (ramkrishna: rev 47d1f137a2f3024836e585eaeb88ccde1db5c1b0) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionParser.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/CellVisibility.java * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/model/ScannerModel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/Authorizations.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsValidator.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionParser.java * hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestScan.java [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11742) Backport HBASE-7987 to 0.98
Esteban Gutierrez created HBASE-11742: - Summary: Backport HBASE-7987 to 0.98 Key: HBASE-11742 URL: https://issues.apache.org/jira/browse/HBASE-11742 Project: HBase Issue Type: Improvement Components: mapreduce, snapshots Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez HBASE-7987 improves how snapshots are handled via a manifest file. This requires reverting HBASE-11360 since introduces an alternate functionality that is not compatible with HBASE-7987. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[
https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096785#comment-14096785
]
Matteo Bertozzi commented on HBASE-11733:
-
{quote}Could this have performance implications, especially for hooks into hot
paths like preScannerNext(...), etc. We're creating one more object upon each
invocation now.{quote}
correct, the other alternative that I had in mind was to use reflection and
have something like execOperation(preCreate, arg1, arg2) so you avoid the
interface creation/scope-capture but you have reflection and strings around.
the problem is that the CoprocessorHost is sort of unmaintainable, if you look
at the code you have some method with slightly variations and some fix not
applied. Do you have other suggestions?
Avoid copy-paste in Master/Region CoprocessorHost
-
Key: HBASE-11733
URL: https://issues.apache.org/jira/browse/HBASE-11733
Project: HBase
Issue Type: Bug
Components: Coprocessors
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0
Attachments: HBASE-11733-v0.patch
all the methods of MasterCoprocessorHost and RegionCoprocessorHost are
copy-paste, and a couple of them have some missing fixes.
The body of each function is the same, the only difference is the call to the
specific coprocessor.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11742) Backport HBASE-7987 to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-11742: -- Attachment: HBASE-11742.v0.patch Backport HBASE-7987 to 0.98 --- Key: HBASE-11742 URL: https://issues.apache.org/jira/browse/HBASE-11742 Project: HBase Issue Type: Improvement Components: mapreduce, snapshots Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Attachments: HBASE-11742.v0.patch HBASE-7987 improves how snapshots are handled via a manifest file. This requires reverting HBASE-11360 since introduces an alternate functionality that is not compatible with HBASE-7987. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11742) Backport HBASE-7987 to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-11742: -- Status: Patch Available (was: Open) Backport HBASE-7987 to 0.98 --- Key: HBASE-11742 URL: https://issues.apache.org/jira/browse/HBASE-11742 Project: HBase Issue Type: Improvement Components: mapreduce, snapshots Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Attachments: HBASE-11742.v0.patch HBASE-7987 improves how snapshots are handled via a manifest file. This requires reverting HBASE-11360 since introduces an alternate functionality that is not compatible with HBASE-7987. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11742) Backport HBASE-7987 to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-11742: -- Affects Version/s: 0.98.5 Backport HBASE-7987 to 0.98 --- Key: HBASE-11742 URL: https://issues.apache.org/jira/browse/HBASE-11742 Project: HBase Issue Type: Improvement Components: mapreduce, snapshots Affects Versions: 0.98.5 Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Attachments: HBASE-11742.v0.patch HBASE-7987 improves how snapshots are handled via a manifest file. This requires reverting HBASE-11360 since introduces an alternate functionality that is not compatible with HBASE-7987. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11742) Backport HBASE-7987 to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-11742: -- Status: In Progress (was: Patch Available) Backport HBASE-7987 to 0.98 --- Key: HBASE-11742 URL: https://issues.apache.org/jira/browse/HBASE-11742 Project: HBase Issue Type: Improvement Components: mapreduce, snapshots Affects Versions: 0.98.5 Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Attachments: HBASE-11742.v0.patch HBASE-7987 improves how snapshots are handled via a manifest file. This requires reverting HBASE-11360 since introduces an alternate functionality that is not compatible with HBASE-7987. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096802#comment-14096802 ] Hudson commented on HBASE-11550: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #421 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/421/]) HBASE-11550 Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted (Gustavo Anatoly) (ndimiduk: rev 7e1e818bbc0bed59c12fbd8c003eaed702cf16ad) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted -- Key: HBASE-11550 URL: https://issues.apache.org/jira/browse/HBASE-11550 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 0.98.4, 0.98.5 Reporter: Ted Yu Assignee: Gustavo Anatoly Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, HBASE-11550.patch User can pass bucket sizes through hbase.bucketcache.bucket.sizes config entry. The sizes are supposed to be in increasing order. Validation should be added in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096809#comment-14096809
]
ramkrishna.s.vasudevan commented on HBASE-11728:
bq.So from the above analysis it shows that while doing previous() the fanIndex
gets changed for the row node and hence while doing the actual next() the
traversal of the trie structure ends up in the other leaf node attached to it.
I would like to rephrase the above sentence. The issue is clear now.
During the seek phase
{code}
[fan:0AB,token:a-b-,numCells:0,fanIndex:1(A),
fan:-,token:-1,numCells:1,fanIndex:0(-),
fan:29,token:14023,numCells:0,fanIndex:0(2),
fan:,token:9600-1402396277,numCells:1,fanIndex:-1]
{code}
and we have ended up in the right kv.
Now the previous() has now made the current row to be pointing to
{code}
fan:-,token:-1,numCells:1,fanIndex:0(-),
{code}
Which is correct.
Now on the next() call, ideally we should be traversing the same set of row
nodes but as the node
{code}
a-b-,numCells:0,fanIndex:1(A)
{code}
has more fanouts other than A (i.e B) the next() tries to track the nodes that
are attached to B and thus we end up in altogether new row. My feel is that in
this scenario where we have already formed the set of nodes we should traverse
the same set of nodes that we already created while seeking.
[~mcorgan] - What do you think?
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 29cb562fad564b468ea9d61a2d60e8b0, HFileAnalys.java,
TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096819#comment-14096819
]
ramkrishna.s.vasudevan commented on HBASE-11728:
In other encoding algos, while seeking using BufferedDataEncoder we do
moveToPrevious() by just ensuring that the data current kv starting offset is
stored in the seeker so that when the actual next() is called we could start
from that same position.
We should indicate in the PrefixTreeSeeker that whenever we do a next() after
seek() and we know that the seek() has done a previous() we should make the
advance() to follow the current fan out even though there could be other
fanouts.
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 29cb562fad564b468ea9d61a2d60e8b0, HFileAnalys.java,
TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-11728:
---
Component/s: Scanners
Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
--
Key: HBASE-11728
URL: https://issues.apache.org/jira/browse/HBASE-11728
Project: HBase
Issue Type: Bug
Components: Scanners
Affects Versions: 0.96.1.1, 0.98.4
Environment: ubuntu12
hadoop-2.2.0
Hbase-0.96.1.1
SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: 29cb562fad564b468ea9d61a2d60e8b0, HFileAnalys.java,
TestPrefixTree.java
Original Estimate: 72h
Remaining Estimate: 72h
In Scan case, i prepare some data as beflow:
Table Desc (Using the prefix-tree encoding) :
'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE',
TTL = '15552000'}
and i put 5 rows as:
(RowKey , Qualifier, Value)
'a-b-0-0', 'qf_1', 'c1-value'
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
corret result:
Test 1:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
'a-b-A-1', 'qf_1', 'c1-value'
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
and then i try next , scan to addColumn
Test2:
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_2));
scan.setStartRow(a-b-A-1.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nonthing. Then i update the addColumn for
scan.addColumn(Bytes.toBytes(cf_1) , Bytes.toBytes(qf_1)); and i got the
expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
then i do more testing... i update the case to modify the startRow greater
than the 'a-b-A-1'
Test3:
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
--
except:
'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
but actually i got nothing again. i modify the start row greater than
'a-b-A-1-1402329600-1402396277'
Scan scan = new Scan();
scan.setStartRow(a-b-A-1-140239.getBytes());
scan.setStopRow(a-b-A-1:.getBytes());
and i got the expect row as well:
'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
So, i think it may be a bug in the prefix-tree encoding.It happens after the
data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11742) Backport HBASE-7987 to 0.98
[
https://issues.apache.org/jira/browse/HBASE-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096832#comment-14096832
]
Matteo Bertozzi commented on HBASE-11742:
-
By applying the patch on 0.98 I get a param problem on the InputFormat
constructor, which seems to be related to HBASE-11137
{code}
mapred/TableSnapshotInputFormat.java:[60,22] error: no suitable constructor
found for InputSplit(String,ListString)
mapreduce/TableSnapshotInputFormat.java:[113,22] error: no suitable constructor
found for InputSplit(String,ListString)
{code}
HBASE-11185 must be applied too.
and HBASE-11326 is a nice to have.
other then that looks ok to me, I'll take another look and play with it a later
Backport HBASE-7987 to 0.98
---
Key: HBASE-11742
URL: https://issues.apache.org/jira/browse/HBASE-11742
Project: HBase
Issue Type: Improvement
Components: mapreduce, snapshots
Affects Versions: 0.98.5
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Attachments: HBASE-11742.v0.patch
HBASE-7987 improves how snapshots are handled via a manifest file. This
requires reverting HBASE-11360 since introduces an alternate functionality
that is not compatible with HBASE-7987.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096842#comment-14096842 ] Hudson commented on HBASE-11550: FAILURE: Integrated in HBase-TRUNK #5396 (See [https://builds.apache.org/job/HBase-TRUNK/5396/]) HBASE-11550 Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted (Gustavo Anatoly) (ndimiduk: rev c60cfbc999e6ae064632e297fd361c8c16e4) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted -- Key: HBASE-11550 URL: https://issues.apache.org/jira/browse/HBASE-11550 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 0.98.4, 0.98.5 Reporter: Ted Yu Assignee: Gustavo Anatoly Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, HBASE-11550.patch User can pass bucket sizes through hbase.bucketcache.bucket.sizes config entry. The sizes are supposed to be in increasing order. Validation should be added in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11731) Add option to only run a subset of the shell tests
[
https://issues.apache.org/jira/browse/HBASE-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096843#comment-14096843
]
Hadoop QA commented on HBASE-11731:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661681/HBASE-11731.patch
against trunk revision .
ATTACHMENT ID: 12661681
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10431//console
This message is automatically generated.
Add option to only run a subset of the shell tests
--
Key: HBASE-11731
URL: https://issues.apache.org/jira/browse/HBASE-11731
Project: HBase
Issue Type: Improvement
Components: build, shell
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Labels: beginner
Attachments: HBASE-11731.patch
Right now, contributors to the shell can limit testing to just the shell
tests but there's no way to limit to just a subset of them.
It would be nice if I could just run the test for the thing I'm changing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[ https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-11724: Attachment: HBASE-11724-v1.patch Talked with jon offline, an example with numbers is probably easier to understand. Also renaming from .share to .ratio should make clearer that the value is an interval between 0 and 1 Add to RWQueueRpcExecutor the ability to split get and scan handlers Key: HBASE-11724 URL: https://issues.apache.org/jira/browse/HBASE-11724 Project: HBase Issue Type: New Feature Components: IPC/RPC Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 2.0.0 Attachments: HBASE-11724-v0.patch, HBASE-11724-v1.patch RWQueueRpcExecutor has the devision between reads and writes requests, but we can split also small-reads and long-reads. This can be useful to force a deprioritization of scans on the RS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096877#comment-14096877 ] Hudson commented on HBASE-11438: FAILURE: Integrated in HBase-TRUNK #5397 (See [https://builds.apache.org/job/HBase-TRUNK/5397/]) HBASE-11438 - [Visibility Controller] Support UTF8 character as Visibility (ramkrishna: rev c1660a7962f9712881d2ee46d57cf781f6e1f36e) * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/Authorizations.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsValidator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/model/ScannerModel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/CellVisibility.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionParser.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionParser.java * hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestScan.java [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[
https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096940#comment-14096940
]
Hadoop QA commented on HBASE-11724:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661696/HBASE-11724-v1.patch
against trunk revision .
ATTACHMENT ID: 12661696
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10432//console
This message is automatically generated.
Add to RWQueueRpcExecutor the ability to split get and scan handlers
Key: HBASE-11724
URL: https://issues.apache.org/jira/browse/HBASE-11724
Project: HBase
Issue Type: New Feature
Components: IPC/RPC
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-11724-v0.patch, HBASE-11724-v1.patch
RWQueueRpcExecutor has the devision between reads and writes requests, but we
can split also small-reads and long-reads. This can be useful to force a
deprioritization of scans on the RS.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096954#comment-14096954 ] Hudson commented on HBASE-11438: FAILURE: Integrated in HBase-1.0 #101 (See [https://builds.apache.org/job/HBase-1.0/101/]) HBASE-11438 - [Visibility Controller] Support UTF8 character as Visibility (ramkrishna: rev ae2a944027bc146efd20e3fe3498e4c717b7baed) * hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestScan.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionParser.java * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/model/ScannerModel.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionParser.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/Authorizations.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/CellVisibility.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsValidator.java [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[
https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096958#comment-14096958
]
Jonathan Hsieh commented on HBASE-11724:
Thanks, the example helps immensely!
nit - fix this doc bug before commit.
{code}
+ a scan.ratio of 0.8 means that: 6 queues will contain only long-read
requests
+ and 6 queues will contain only short-read requests.
{code}
+1
Add to RWQueueRpcExecutor the ability to split get and scan handlers
Key: HBASE-11724
URL: https://issues.apache.org/jira/browse/HBASE-11724
Project: HBase
Issue Type: New Feature
Components: IPC/RPC
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-11724-v0.patch, HBASE-11724-v1.patch
RWQueueRpcExecutor has the devision between reads and writes requests, but we
can split also small-reads and long-reads. This can be useful to force a
deprioritization of scans on the RS.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11743) Add unit test for the fix that sorts custom value of BUCKET_CACHE_BUCKETS_KEY
Ted Yu created HBASE-11743: -- Summary: Add unit test for the fix that sorts custom value of BUCKET_CACHE_BUCKETS_KEY Key: HBASE-11743 URL: https://issues.apache.org/jira/browse/HBASE-11743 Project: HBase Issue Type: Test Reporter: Ted Yu Priority: Minor HBASE-11550 sorts the custom value of BUCKET_CACHE_BUCKETS_KEY such that there is no wastage in bucket allocation. This JIRA is to add unit test for the fix so that there is no regression in the future. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096975#comment-14096975
]
Carter commented on HBASE-11657:
How should the interfaces and MapReduce play together? I know certain classes
like TableInputFormatBase look at HRegionLocation. If that's not something
that should still be exposed, then to maintain appropriate encapsulation of
HBase from MR we'll have to explore an alternative. Any ideas?
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
HBASE_11657_v3.patch, HBASE_11657_v3.patch, HBASE_11657_v4.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11731) Add option to only run a subset of the shell tests
[ https://issues.apache.org/jira/browse/HBASE-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096978#comment-14096978 ] Sean Busbey commented on HBASE-11731: - test result support shows no failures, so I'm not sure what QA is complaining about. Add option to only run a subset of the shell tests -- Key: HBASE-11731 URL: https://issues.apache.org/jira/browse/HBASE-11731 Project: HBase Issue Type: Improvement Components: build, shell Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Labels: beginner Attachments: HBASE-11731.patch Right now, contributors to the shell can limit testing to just the shell tests but there's no way to limit to just a subset of them. It would be nice if I could just run the test for the thing I'm changing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[ https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-11724: Attachment: HBASE-11724-v2.patch Add to RWQueueRpcExecutor the ability to split get and scan handlers Key: HBASE-11724 URL: https://issues.apache.org/jira/browse/HBASE-11724 Project: HBase Issue Type: New Feature Components: IPC/RPC Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 2.0.0 Attachments: HBASE-11724-v0.patch, HBASE-11724-v1.patch, HBASE-11724-v2.patch RWQueueRpcExecutor has the devision between reads and writes requests, but we can split also small-reads and long-reads. This can be useful to force a deprioritization of scans on the RS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-11743) Add unit test for the fix that sorts custom value of BUCKET_CACHE_BUCKETS_KEY
[ https://issues.apache.org/jira/browse/HBASE-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly reassigned HBASE-11743: --- Assignee: Gustavo Anatoly Add unit test for the fix that sorts custom value of BUCKET_CACHE_BUCKETS_KEY - Key: HBASE-11743 URL: https://issues.apache.org/jira/browse/HBASE-11743 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Gustavo Anatoly Priority: Minor HBASE-11550 sorts the custom value of BUCKET_CACHE_BUCKETS_KEY such that there is no wastage in bucket allocation. This JIRA is to add unit test for the fix so that there is no regression in the future. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11730) Document release managers for non-deprecated branches
[ https://issues.apache.org/jira/browse/HBASE-11730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097011#comment-14097011 ] Sean Busbey commented on HBASE-11730: - At a minimum, I think having a ask Foo about inclusion in release X would be useful, in the same way we have an [explanation for and pointers to the component owners|http://hbase.apache.org/book.html#community.roles]. It gives new people a leg up on figuring out the existing social norm instead of requiring them to reverse engineer it from ping Foo for branch X notes in jira. Actually maybe this should be under the community roles section and not in codelines as I suggested earlier. Document release managers for non-deprecated branches - Key: HBASE-11730 URL: https://issues.apache.org/jira/browse/HBASE-11730 Project: HBase Issue Type: Task Components: documentation Reporter: Sean Busbey New development goes against trunk and is backported as desired to existing release branches. From what I have seen on the jira, it looks like each branch's release manager makes the call on backporting a particular issue. We should document both this norm and who the relevant release manager is for each branch. In the current docs, I'd suggest adding the RM list to the Codelines section (18.11.1) and add a brief explanation of pinging the RM as a new section after submitting a patch again (18.12.6). Post HBASE-4593, the note about pinging a prior branch RM should just go as a bullet in the patch workflow. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[ https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097010#comment-14097010 ] Jonathan Hsieh commented on HBASE-11724: +1 v2. Add to RWQueueRpcExecutor the ability to split get and scan handlers Key: HBASE-11724 URL: https://issues.apache.org/jira/browse/HBASE-11724 Project: HBase Issue Type: New Feature Components: IPC/RPC Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 2.0.0 Attachments: HBASE-11724-v0.patch, HBASE-11724-v1.patch, HBASE-11724-v2.patch RWQueueRpcExecutor has the devision between reads and writes requests, but we can split also small-reads and long-reads. This can be useful to force a deprioritization of scans on the RS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11627) RegionSplitter's rollingSplit terminated with / by zero, and the _balancedSplit file was not deleted properly
[
https://issues.apache.org/jira/browse/HBASE-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097017#comment-14097017
]
Sean Busbey commented on HBASE-11627:
-
bump. mind taking a look [~apurtell]?
RegionSplitter's rollingSplit terminated with / by zero, and the
_balancedSplit file was not deleted properly
---
Key: HBASE-11627
URL: https://issues.apache.org/jira/browse/HBASE-11627
Project: HBase
Issue Type: Bug
Components: Admin, util
Affects Versions: 0.96.2, 0.98.1
Environment: CentOS release 6.4 (Final) x86_64
hbase-0.98.1-cdh5.1.0
Reporter: louis hust
Assignee: Sean Busbey
Labels: beginner
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE_11627-v2.patch, HBASE_11627-v3.patch,
HBASE_11627-v4.patch, HBASE_11627-v5.patch, HBASE_11627-v6.patch,
HBASE_11627-v7.patch, HBASE_11627.patch
Original Estimate: 24h
Remaining Estimate: 24h
First create a table
{quote}
create 't1', 'f1'
{quote}
Then split the table with util:
{quote}
bin/hbase org.apache.hadoop.hbase.util.RegionSplitter -r -o 2 t1
UniformSplit
{quote}
*Finally get the following error :*
{quote}
12/11/08 19:21:12 DEBUG util.RegionSplitter: All regions have been
successfully split!
12/11/08 19:21:12 DEBUG util.RegionSplitter: TOTAL TIME = 30sec
12/11/08 19:21:12 DEBUG util.RegionSplitter: Splits = 0
Exception in thread main java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.hbase.util.RegionSplitter.rollingSplit(RegionSplitter.java:576)
at
org.apache.hadoop.hbase.util.RegionSplitter.main(RegionSplitter.java:349)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097028#comment-14097028 ] Hudson commented on HBASE-11438: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #422 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/422/]) HBASE-11438 - [Visibility Controller] Support UTF8 character as Visibility (ramkrishna: rev 47d1f137a2f3024836e585eaeb88ccde1db5c1b0) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionParser.java * hbase-server/src/main/java/org/apache/hadoop/hbase/rest/model/ScannerModel.java * hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestScan.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/CellVisibility.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsValidator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionParser.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/Authorizations.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11708) RegionSplitter incorrectly calculates splitcount
[
https://issues.apache.org/jira/browse/HBASE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097035#comment-14097035
]
Sean Busbey commented on HBASE-11708:
-
+1 LGTM.
RegionSplitter incorrectly calculates splitcount
Key: HBASE-11708
URL: https://issues.apache.org/jira/browse/HBASE-11708
Project: HBase
Issue Type: Bug
Components: Admin, util
Affects Versions: 0.96.2, 0.98.1
Reporter: Sean Busbey
Assignee: louis hust
Priority: Minor
Labels: beginner
Attachments: HBASE_11708-v2.patch, HBASE_11708-v3.patch,
HBASE_11708.patch
From discussion on HBASE-11627:
{quote}
And I also find another bug about the caculation of the variable splitCount
which is cause by the wrong caculation of variable splitCount.
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-11733: Attachment: HBASE-11733-0.98-v1.patch Avoid copy-paste in Master/Region CoprocessorHost - Key: HBASE-11733 URL: https://issues.apache.org/jira/browse/HBASE-11733 Project: HBase Issue Type: Bug Components: Coprocessors Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Trivial Fix For: 2.0.0 Attachments: HBASE-11733-0.98-v1.patch, HBASE-11733-v0.patch all the methods of MasterCoprocessorHost and RegionCoprocessorHost are copy-paste, and a couple of them have some missing fixes. The body of each function is the same, the only difference is the call to the specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[
https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097042#comment-14097042
]
Hadoop QA commented on HBASE-11733:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661722/HBASE-11733-0.98-v1.patch
against trunk revision .
ATTACHMENT ID: 12661722
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10434//console
This message is automatically generated.
Avoid copy-paste in Master/Region CoprocessorHost
-
Key: HBASE-11733
URL: https://issues.apache.org/jira/browse/HBASE-11733
Project: HBase
Issue Type: Bug
Components: Coprocessors
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0
Attachments: HBASE-11733-0.98-v1.patch, HBASE-11733-v0.patch
all the methods of MasterCoprocessorHost and RegionCoprocessorHost are
copy-paste, and a couple of them have some missing fixes.
The body of each function is the same, the only difference is the call to the
specific coprocessor.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Work started] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[ https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-7767 started by Andrey Stepachev. Get rid of ZKTable, and table enable/disable state in ZK - Key: HBASE-7767 URL: https://issues.apache.org/jira/browse/HBASE-7767 Project: HBase Issue Type: Sub-task Components: Zookeeper Affects Versions: 0.95.2 Reporter: Enis Soztutar Assignee: Andrey Stepachev As discussed table state in zookeeper for enable/disable state breaks our zookeeper contract. It is also very intrusive, used from the client side, master and region servers. We should get rid of it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097057#comment-14097057 ] Anoop Sam John commented on HBASE-11438: Same test case failures I am also getting locally. Investigating... [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11744) RpcServer code should not use a collection from netty internal
Nicolas Liochon created HBASE-11744: --- Summary: RpcServer code should not use a collection from netty internal Key: HBASE-11744 URL: https://issues.apache.org/jira/browse/HBASE-11744 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.99.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.99.0, 2.0.0 A side effect of the autocompletion... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11744) RpcServer code should not use a collection from netty internal
[ https://issues.apache.org/jira/browse/HBASE-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11744: Attachment: 11744.v1.patch RpcServer code should not use a collection from netty internal -- Key: HBASE-11744 URL: https://issues.apache.org/jira/browse/HBASE-11744 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.99.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.99.0, 2.0.0 Attachments: 11744.v1.patch A side effect of the autocompletion... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11744) RpcServer code should not use a collection from netty internal
[ https://issues.apache.org/jira/browse/HBASE-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11744: Status: Patch Available (was: Open) RpcServer code should not use a collection from netty internal -- Key: HBASE-11744 URL: https://issues.apache.org/jira/browse/HBASE-11744 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.99.0, 2.0.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.99.0, 2.0.0 Attachments: 11744.v1.patch A side effect of the autocompletion... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097069#comment-14097069 ] ramkrishna.s.vasudevan commented on HBASE-11438: Just saw the failures. Will update the patch with an addendum.the way did not report any failure. [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[
https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097102#comment-14097102
]
Hadoop QA commented on HBASE-11724:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661715/HBASE-11724-v2.patch
against trunk revision .
ATTACHMENT ID: 12661715
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10433//console
This message is automatically generated.
Add to RWQueueRpcExecutor the ability to split get and scan handlers
Key: HBASE-11724
URL: https://issues.apache.org/jira/browse/HBASE-11724
Project: HBase
Issue Type: New Feature
Components: IPC/RPC
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-11724-v0.patch, HBASE-11724-v1.patch,
HBASE-11724-v2.patch
RWQueueRpcExecutor has the devision between reads and writes requests, but we
can split also small-reads and long-reads. This can be useful to force a
deprioritization of scans on the RS.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11687) No need to abort on postOpenDeployTasks exception if region opening is cancelled
[ https://issues.apache.org/jira/browse/HBASE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11687: Priority: Minor (was: Major) No need to abort on postOpenDeployTasks exception if region opening is cancelled Key: HBASE-11687 URL: https://issues.apache.org/jira/browse/HBASE-11687 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 1.0.0 Attachments: hbase-11687.patch With ZK-less region assignment, if region opening is in postOpenDeployTasks step and the opening is cancelled, the region server will abort because it can't report the transition to the master. We should cancel postOpenDeployTasks instead. At least, there is no need to abort. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097113#comment-14097113 ] stack commented on HBASE-11550: --- [~ndimiduk] Why commit without the test? You start out with ... I'm not convinced the ticket is valid. -- something a test would clear up -- and then [c]an you add a test exercising the lack of sort as a problem? There is assertion that there is wastage being fixed by this patch. I'd be interested in assertions in code that this is actually the case. I ask Add tests to assert wastage. I'd also like to know by how much. I have been looking at this stuff of late and notice that bucketcache has a 'cost'. I'm interested in figuring where it comes from. This committed patch as is gives no insight when it seems like it could have some bearing. Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted -- Key: HBASE-11550 URL: https://issues.apache.org/jira/browse/HBASE-11550 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 0.98.4, 0.98.5 Reporter: Ted Yu Assignee: Gustavo Anatoly Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, HBASE-11550.patch User can pass bucket sizes through hbase.bucketcache.bucket.sizes config entry. The sizes are supposed to be in increasing order. Validation should be added in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[
https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097132#comment-14097132
]
ramkrishna.s.vasudevan commented on HBASE-11438:
I saw this iink
{code}
https://builds.apache.org/job/HBase-TRUNK/5397/testReport/
{code}
and thought no failures in the visibility. But on seeing the link it has not
run other visibility tests.
[Visibility Controller] Support UTF8 character as Visibility Labels
---
Key: HBASE-11438
URL: https://issues.apache.org/jira/browse/HBASE-11438
Project: HBase
Issue Type: Improvement
Components: security
Affects Versions: 0.98.4
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 1.0.0, 2.0.0, 0.98.6
Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch,
HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch,
HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch,
HBASE-11438_v8.patch, HBASE-11438_v9.patch
This would be an action item that we would be addressing so that the
visibility labels could have UTF8 characters in them. Also allow the user to
use a client supplied API that allows to specify the visibility labels inside
double quotes such that UTF8 characters and cases like , |, ! and double
quotes itself could be specified with proper escape sequence. Accumulo too
provides one such API in the client side.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11687) No need to abort on postOpenDeployTasks exception if region opening is cancelled
[ https://issues.apache.org/jira/browse/HBASE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097140#comment-14097140 ] Matteo Bertozzi commented on HBASE-11687: - +1 No need to abort on postOpenDeployTasks exception if region opening is cancelled Key: HBASE-11687 URL: https://issues.apache.org/jira/browse/HBASE-11687 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 1.0.0 Attachments: hbase-11687.patch With ZK-less region assignment, if region opening is in postOpenDeployTasks step and the opening is cancelled, the region server will abort because it can't report the transition to the master. We should cancel postOpenDeployTasks instead. At least, there is no need to abort. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097109#comment-14097109 ] Jimmy Xiang commented on HBASE-11689: - How about go with option 1 and make sure only master updates it? It doesn't help for the RS to update this znode since the meta is still not online anyway without letting the master know. Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Jimmy Xiang Assignee: Andrey Stepachev With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10834) Better error messaging on issuing grant commands in non-authz mode
[
https://issues.apache.org/jira/browse/HBASE-10834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-10834:
Resolution: Fixed
Fix Version/s: 0.94.23
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Integrated into 0.94. Thanks Srikanth for the patch.
Better error messaging on issuing grant commands in non-authz mode
--
Key: HBASE-10834
URL: https://issues.apache.org/jira/browse/HBASE-10834
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.94.17
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Priority: Trivial
Fix For: 0.94.23
Attachments: HBASE-10834.patch, HBASE-10834_v2.patch,
HBASE-10834_v3.patch, HBASE-10834_v4.patch
Running the below sequence of steps should give a better error messaging
rather than table not found error.
{code}
hbase(main):009:0 grant test, RWCXA
ERROR: Unknown table _acl_!
Here is some help for this command:
Grant users specific rights.
Syntax : grant user permissions [table [column family [column
qualifier]]
permissions is either zero or more letters from the set RWXCA.
READ('R'), WRITE('W'), EXEC('X'), CREATE('C'), ADMIN('A')
For example:
hbase grant 'bobsmith', 'RWXCA'
hbase grant 'bobsmith', 'RW', 't1', 'f1', 'col1'
{code}
Instead of ERROR: Unknown table _acl_!, hbase should give out a warning like
Command not supported in non-authz mode(as acl table is only created if
authz is turned on)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: HBASE_11699_v1.patch Region servers exclusion list to HMaster. - Key: HBASE-11699 URL: https://issues.apache.org/jira/browse/HBASE-11699 Project: HBase Issue Type: New Feature Components: Admin, Client, regionserver, Zookeeper Affects Versions: 0.98.3 Reporter: Gomathivinayagam Muthuvinayagam Priority: Minor Labels: patch Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11699.patch, HBASE_11699_v1.patch Original Estimate: 96h Remaining Estimate: 96h Currently HBase does not support adding set of region servers to be in the exclusion list. So that administrators can prevent accidental startups of some region servers to join the cluster. There was initially some work done, and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It was not done after that. I am planning to contribute it as a patch, and I would like to do some improvements as well. Instead of storing the exclusion entries on a file, I am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11689) Track meta in transition
[ https://issues.apache.org/jira/browse/HBASE-11689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097145#comment-14097145 ] Andrey Stepachev commented on HBASE-11689: -- [~jxiang] Making only master write this znode make sense. Lets go with 1. Track meta in transition Key: HBASE-11689 URL: https://issues.apache.org/jira/browse/HBASE-11689 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Jimmy Xiang Assignee: Andrey Stepachev With ZK-less region assignment, user regions in transition are tracked in meta. We need a way to track meta in transition too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097148#comment-14097148 ] Anoop Sam John commented on HBASE-11438: You can see failures in 0.98 branch test run [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 1.0.0, 2.0.0, 0.98.6 Attachments: HBASE-11438_v1.patch, HBASE-11438_v10.patch, HBASE-11438_v2.patch, HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, HBASE-11438_v9.patch This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097154#comment-14097154 ] Gomathivinayagam Muthuvinayagam commented on HBASE-11699: - I made the changes to store included/excluded hosts into a meta table in the reserved namespace. I have attached the patch for the same. I updated the reviewboard request as well. Could you review it? Region servers exclusion list to HMaster. - Key: HBASE-11699 URL: https://issues.apache.org/jira/browse/HBASE-11699 Project: HBase Issue Type: New Feature Components: Admin, Client, regionserver, Zookeeper Affects Versions: 0.98.3 Reporter: Gomathivinayagam Muthuvinayagam Priority: Minor Labels: patch Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11699.patch, HBASE_11699_v1.patch Original Estimate: 96h Remaining Estimate: 96h Currently HBase does not support adding set of region servers to be in the exclusion list. So that administrators can prevent accidental startups of some region servers to join the cluster. There was initially some work done, and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It was not done after that. I am planning to contribute it as a patch, and I would like to do some improvements as well. Instead of storing the exclusion entries on a file, I am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11702: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.98+ Better introspection of long running compactions Key: HBASE-11702 URL: https://issues.apache.org/jira/browse/HBASE-11702 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch, HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch For better introspection of long running compactions, periodically print compaction progress for a file at DEBUG level (thread name, file path, total compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097161#comment-14097161 ] Matteo Bertozzi commented on HBASE-11699: - instead of having just an allowedRS table that stores just booleans why not generalize it a bit as rsSettings or similar, that can be used in the future to tune rs settings.. like how many regions can stay on that RS or similar Region servers exclusion list to HMaster. - Key: HBASE-11699 URL: https://issues.apache.org/jira/browse/HBASE-11699 Project: HBase Issue Type: New Feature Components: Admin, Client, regionserver, Zookeeper Affects Versions: 0.98.3 Reporter: Gomathivinayagam Muthuvinayagam Priority: Minor Labels: patch Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE_11699.patch, HBASE_11699_v1.patch Original Estimate: 96h Remaining Estimate: 96h Currently HBase does not support adding set of region servers to be in the exclusion list. So that administrators can prevent accidental startups of some region servers to join the cluster. There was initially some work done, and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It was not done after that. I am planning to contribute it as a patch, and I would like to do some improvements as well. Instead of storing the exclusion entries on a file, I am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11687) No need to abort on postOpenDeployTasks exception if region opening is cancelled
[ https://issues.apache.org/jira/browse/HBASE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11687: Resolution: Fixed Fix Version/s: (was: 1.0.0) 2.0.0 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Integrated into branch 1 and master. Thanks Matteo for the review. No need to abort on postOpenDeployTasks exception if region opening is cancelled Key: HBASE-11687 URL: https://issues.apache.org/jira/browse/HBASE-11687 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.99.0, 2.0.0 Attachments: hbase-11687.patch With ZK-less region assignment, if region opening is in postOpenDeployTasks step and the opening is cancelled, the region server will abort because it can't report the transition to the master. We should cancel postOpenDeployTasks instead. At least, there is no need to abort. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11740) RegionStates.getRegionAssignments() gets stuck on clone
[ https://issues.apache.org/jira/browse/HBASE-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11740: -- Status: Patch Available (was: Open) RegionStates.getRegionAssignments() gets stuck on clone --- Key: HBASE-11740 URL: https://issues.apache.org/jira/browse/HBASE-11740 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-11740.patch This happens in processDeadServersAndRegionsInTransition() on Master startup. RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds a tree from sorted data (Order of n). There were a million entries in RegionAssigments and from jstack, thread was stuck in TreeMap.buildFromSorted() for couple of hours. Instead of this shadow clone, wrapping as an unmodifiable Map should be enough. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11740) RegionStates.getRegionAssignments() gets stuck on clone
[ https://issues.apache.org/jira/browse/HBASE-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11740: -- Attachment: HBASE-11740.patch Doing the same for RegionStates.getRegionsInTransition(). Also checked that no other callers were modifying this map. RegionStates.getRegionAssignments() gets stuck on clone --- Key: HBASE-11740 URL: https://issues.apache.org/jira/browse/HBASE-11740 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-11740.patch This happens in processDeadServersAndRegionsInTransition() on Master startup. RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds a tree from sorted data (Order of n). There were a million entries in RegionAssigments and from jstack, thread was stuck in TreeMap.buildFromSorted() for couple of hours. Instead of this shadow clone, wrapping as an unmodifiable Map should be enough. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11610) Enhance remote meta updates
[ https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097183#comment-14097183 ] Jimmy Xiang commented on HBASE-11610: - With threadlocal, how is the HTable closed? The purpose of these HConnections is just to talk to the meta region. They share the same network connection, access the same region on the same region server. With more connections, we can achieve some parallelism at the network/RPC layer. It is another configuration, isn't it? Personally, I prefer your initial approach: share the HConnection from the server. Enhance remote meta updates --- Key: HBASE-11610 URL: https://issues.apache.org/jira/browse/HBASE-11610 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Virag Kothari Attachments: HBASE-11610.patch Currently, if the meta region is on a regionserver instead of the master, meta update is synchronized on one HTable instance. We should be able to do better. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097185#comment-14097185 ] Andrew Purtell commented on HBASE-11733: I tend to agree with Matteo that the maintainability improvement is really big here. The copy-paste wasn't bad in the beginning but the pre/post-amble of coprocessor hook invocations has grown over time, and inconsistently (bad) Use of reflection will cost us more than an additional object allocation, guaranteed. Thanks for the 0.98 patch. +1 for 0.98 and up Avoid copy-paste in Master/Region CoprocessorHost - Key: HBASE-11733 URL: https://issues.apache.org/jira/browse/HBASE-11733 Project: HBase Issue Type: Bug Components: Coprocessors Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Trivial Fix For: 2.0.0 Attachments: HBASE-11733-0.98-v1.patch, HBASE-11733-v0.patch all the methods of MasterCoprocessorHost and RegionCoprocessorHost are copy-paste, and a couple of them have some missing fixes. The body of each function is the same, the only difference is the call to the specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11732) No need to force unassign a region
[ https://issues.apache.org/jira/browse/HBASE-11732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097188#comment-14097188 ] Matteo Bertozzi commented on HBASE-11732: - switching from regionOffline(region) to regionStates.updateRegionState(region, State.OFFLINE) will result in a missing notification right? regionOffline() is calling sendRegionClosedNotification() probably a test should be added to verify the case where the server is offline No need to force unassign a region -- Key: HBASE-11732 URL: https://issues.apache.org/jira/browse/HBASE-11732 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 2.0.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: hbase-11732.patch Clean up force region unassignment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097189#comment-14097189 ] Matteo Bertozzi commented on HBASE-11733: - yeah just to follow up with some numbers, reflection is too expensive while the new Interface() takes only few nanoseconds per call. I've tested it with jit disabled calling the direct method vs calling a method with the interface that calls the direct method.. and by dividing the total time by the numbers of call I'm around 20nanosec. Avoid copy-paste in Master/Region CoprocessorHost - Key: HBASE-11733 URL: https://issues.apache.org/jira/browse/HBASE-11733 Project: HBase Issue Type: Bug Components: Coprocessors Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Trivial Fix For: 2.0.0 Attachments: HBASE-11733-0.98-v1.patch, HBASE-11733-v0.patch all the methods of MasterCoprocessorHost and RegionCoprocessorHost are copy-paste, and a couple of them have some missing fixes. The body of each function is the same, the only difference is the call to the specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11740) RegionStates.getRegionAssignments() gets stuck on clone
[ https://issues.apache.org/jira/browse/HBASE-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097191#comment-14097191 ] Jimmy Xiang edited comment on HBASE-11740 at 8/14/14 4:46 PM: -- If the original is changed, will the unmodifiable Map get some exception if it is in the middle of iteration? was (Author: jxiang): It the original is changed, will the unmodifiable Map get some exception if it is in the middle of iteration? RegionStates.getRegionAssignments() gets stuck on clone --- Key: HBASE-11740 URL: https://issues.apache.org/jira/browse/HBASE-11740 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-11740.patch This happens in processDeadServersAndRegionsInTransition() on Master startup. RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds a tree from sorted data (Order of n). There were a million entries in RegionAssigments and from jstack, thread was stuck in TreeMap.buildFromSorted() for couple of hours. Instead of this shadow clone, wrapping as an unmodifiable Map should be enough. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11740) RegionStates.getRegionAssignments() gets stuck on clone
[ https://issues.apache.org/jira/browse/HBASE-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097191#comment-14097191 ] Jimmy Xiang commented on HBASE-11740: - It the original is changed, will the unmodifiable Map get some exception if it is in the middle of iteration? RegionStates.getRegionAssignments() gets stuck on clone --- Key: HBASE-11740 URL: https://issues.apache.org/jira/browse/HBASE-11740 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Virag Kothari Assignee: Virag Kothari Attachments: HBASE-11740.patch This happens in processDeadServersAndRegionsInTransition() on Master startup. RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds a tree from sorted data (Order of n). There were a million entries in RegionAssigments and from jstack, thread was stuck in TreeMap.buildFromSorted() for couple of hours. Instead of this shadow clone, wrapping as an unmodifiable Map should be enough. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097190#comment-14097190 ] Nick Dimiduk commented on HBASE-11550: -- I committed v4 of this patch because it ensures the placement semantics are identical for the default setting and user-provided value, something I inadvertently missed in HBASE-10641. Yes, I think a test would be good. Wastage will depend on the size of buckets, the placement strategy used, and the sequence of block sizes inserted. The difficulty with a test is that memory allocation strategies are not black-and-white. It could well be that there's a better strategy for most of our users than the current greedy approach. I guess there's now HBASE-11743 that will look at this. Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted -- Key: HBASE-11550 URL: https://issues.apache.org/jira/browse/HBASE-11550 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 0.98.4, 0.98.5 Reporter: Ted Yu Assignee: Gustavo Anatoly Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, HBASE-11550.patch User can pass bucket sizes through hbase.bucketcache.bucket.sizes config entry. The sizes are supposed to be in increasing order. Validation should be added in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11745) FilterAllFilter should return ReturnCode.SKIP
Andrew Purtell created HBASE-11745:
--
Summary: FilterAllFilter should return ReturnCode.SKIP
Key: HBASE-11745
URL: https://issues.apache.org/jira/browse/HBASE-11745
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 2.0.0, 0.98.6
FilterAllFilter is a filter in hbase-server's test packages used by
PerformanceEvaluation to filter all results at the server when given the
--filterAll command line argument. There is an unfortunate bug in versions
greater than 0.94 (where I rewrote this filter and didn't notice the issue in
later versions at the time):
{code}
@Override
public ReturnCode filterKeyValue(Cell v) throws IOException {
return ReturnCode.INCLUDE;
}
{code}
This is obviously wrong, and results in the creation of KeyValue objects that
we will almost immediately throw away when processing filterRow for backwards
compatibility, which FilterAllFilter implements as:
{code}
@Override
public boolean filterRow() throws IOException {
return true;
}
{code}
We noticed this problem when comparing object allocations during RPC processing
between 0.94 and 0.98.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11744) RpcServer code should not use a collection from netty internal
[
https://issues.apache.org/jira/browse/HBASE-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097199#comment-14097199
]
Hadoop QA commented on HBASE-11744:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661724/11744.v1.patch
against trunk revision .
ATTACHMENT ID: 12661724
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10435//console
This message is automatically generated.
RpcServer code should not use a collection from netty internal
--
Key: HBASE-11744
URL: https://issues.apache.org/jira/browse/HBASE-11744
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.99.0, 2.0.0
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Minor
Fix For: 0.99.0, 2.0.0
Attachments: 11744.v1.patch
A side effect of the autocompletion...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11745) FilterAllFilter should return ReturnCode.SKIP
[
https://issues.apache.org/jira/browse/HBASE-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11745:
---
Attachment: HBASE-11745.patch
Attached fix for the reported problem plus removal of code that doesn't do
anything useful in parseFrom.
FilterAllFilter should return ReturnCode.SKIP
-
Key: HBASE-11745
URL: https://issues.apache.org/jira/browse/HBASE-11745
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11745.patch
FilterAllFilter is a filter in hbase-server's test packages used by
PerformanceEvaluation to filter all results at the server when given the
--filterAll command line argument. There is an unfortunate bug in versions
greater than 0.94 (where I rewrote this filter and didn't notice the issue in
later versions at the time):
{code}
@Override
public ReturnCode filterKeyValue(Cell v) throws IOException {
return ReturnCode.INCLUDE;
}
{code}
This is obviously wrong, and results in the creation of KeyValue objects that
we will almost immediately throw away when processing filterRow for backwards
compatibility, which FilterAllFilter implements as:
{code}
@Override
public boolean filterRow() throws IOException {
return true;
}
{code}
We noticed this problem when comparing object allocations during RPC
processing between 0.94 and 0.98.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11745) FilterAllFilter should return ReturnCode.SKIP
[
https://issues.apache.org/jira/browse/HBASE-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11745:
---
Status: Patch Available (was: Open)
FilterAllFilter should return ReturnCode.SKIP
-
Key: HBASE-11745
URL: https://issues.apache.org/jira/browse/HBASE-11745
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11745.patch
FilterAllFilter is a filter in hbase-server's test packages used by
PerformanceEvaluation to filter all results at the server when given the
--filterAll command line argument. There is an unfortunate bug in versions
greater than 0.94 (where I rewrote this filter and didn't notice the issue in
later versions at the time):
{code}
@Override
public ReturnCode filterKeyValue(Cell v) throws IOException {
return ReturnCode.INCLUDE;
}
{code}
This is obviously wrong, and results in the creation of KeyValue objects that
we will almost immediately throw away when processing filterRow for backwards
compatibility, which FilterAllFilter implements as:
{code}
@Override
public boolean filterRow() throws IOException {
return true;
}
{code}
We noticed this problem when comparing object allocations during RPC
processing between 0.94 and 0.98.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11610) Enhance remote meta updates
[ https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097205#comment-14097205 ] Virag Kothari commented on HBASE-11610: --- bq. With threadlocal, how is the HTable closed? Good question. If we were to close it, we would need to store references to it which is expensive. So, instead if we skip the close it should be fine because the way htable is created, it wont shutdown the pool or close the connection (Both cleanupPoolOnClose and cleanupConnectionOnClose is false). So the only thing it would do is flushCommits but put() will take care of that as autoflush is true by default. bq. With more connections, we can achieve some parallelism at the network/RPC layer. Yeah, more TCP connections are helping. I have seen the write throughput on meta region increase to around 450K/min from around 380k/min (~15%) when trying to assign 1M regions. Tweaking this configuration would help us when we to scale even further. Enhance remote meta updates --- Key: HBASE-11610 URL: https://issues.apache.org/jira/browse/HBASE-11610 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Virag Kothari Attachments: HBASE-11610.patch Currently, if the meta region is on a regionserver instead of the master, meta update is synchronized on one HTable instance. We should be able to do better. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11745) FilterAllFilter should return ReturnCode.SKIP
[
https://issues.apache.org/jira/browse/HBASE-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097208#comment-14097208
]
Ted Yu commented on HBASE-11745:
+1
FilterAllFilter should return ReturnCode.SKIP
-
Key: HBASE-11745
URL: https://issues.apache.org/jira/browse/HBASE-11745
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11745.patch
FilterAllFilter is a filter in hbase-server's test packages used by
PerformanceEvaluation to filter all results at the server when given the
--filterAll command line argument. There is an unfortunate bug in versions
greater than 0.94 (where I rewrote this filter and didn't notice the issue in
later versions at the time):
{code}
@Override
public ReturnCode filterKeyValue(Cell v) throws IOException {
return ReturnCode.INCLUDE;
}
{code}
This is obviously wrong, and results in the creation of KeyValue objects that
we will almost immediately throw away when processing filterRow for backwards
compatibility, which FilterAllFilter implements as:
{code}
@Override
public boolean filterRow() throws IOException {
return true;
}
{code}
We noticed this problem when comparing object allocations during RPC
processing between 0.94 and 0.98.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11745) FilterAllFilter should return ReturnCode.SKIP
[
https://issues.apache.org/jira/browse/HBASE-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097210#comment-14097210
]
Andrew Purtell commented on HBASE-11745:
Thanks Ted. Will commit shortly.
FilterAllFilter should return ReturnCode.SKIP
-
Key: HBASE-11745
URL: https://issues.apache.org/jira/browse/HBASE-11745
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11745.patch
FilterAllFilter is a filter in hbase-server's test packages used by
PerformanceEvaluation to filter all results at the server when given the
--filterAll command line argument. There is an unfortunate bug in versions
greater than 0.94 (where I rewrote this filter and didn't notice the issue in
later versions at the time):
{code}
@Override
public ReturnCode filterKeyValue(Cell v) throws IOException {
return ReturnCode.INCLUDE;
}
{code}
This is obviously wrong, and results in the creation of KeyValue objects that
we will almost immediately throw away when processing filterRow for backwards
compatibility, which FilterAllFilter implements as:
{code}
@Override
public boolean filterRow() throws IOException {
return true;
}
{code}
We noticed this problem when comparing object allocations during RPC
processing between 0.94 and 0.98.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097209#comment-14097209
]
Nick Dimiduk commented on HBASE-11657:
--
The MapReduce classes are considered internal API. A user doesn't need to
know where a region lives in order to submit a MR job.
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
HBASE_11657_v3.patch, HBASE_11657_v3.patch, HBASE_11657_v4.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097215#comment-14097215
]
Nick Dimiduk commented on HBASE-11657:
--
bq. Should RegionLocator extend Closeable?
I think so yes. This is consistent with Admin also being closable, despite
being retrieved from the closable HConnection.
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
HBASE_11657_v3.patch, HBASE_11657_v3.patch, HBASE_11657_v4.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11732) Should not preemptively offline a region
[ https://issues.apache.org/jira/browse/HBASE-11732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11732: Summary: Should not preemptively offline a region (was: No need to force unassign a region) Should not preemptively offline a region Key: HBASE-11732 URL: https://issues.apache.org/jira/browse/HBASE-11732 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 2.0.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: hbase-11732.patch Clean up force region unassignment. -- This message was sent by Atlassian JIRA (v6.2#6252)
