[
https://issues.apache.org/jira/browse/HBASE-22969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973555#comment-16973555
]
Udai Bhan Kashyap commented on HBASE-22969:
-------------------------------------------
On submit of latest patch following tests failed:
{color:#de350b}*[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time
elapsed: 322.33 s <<< FAILURE! - in
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover*{color}
[ERROR]
testSecondaryRegionKill(org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover)
Time elapsed: 44.909 s <<< FAILURE!
java.lang.AssertionError: Failed verification of row :0
at
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover.testSecondaryRegionKill(TestRegionReplicaFailover.java:240)
{color:#de350b}*[ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time
elapsed: 2.746 s <<< FAILURE! - in
org.apache.hadoop.hbase.io.hfile.bucket.TestBucketCacheRefCnt*{color}
[ERROR]
testBlockInRAMCache(org.apache.hadoop.hbase.io.hfile.bucket.TestBucketCacheRefCnt)
Time elapsed: 0.087 s <<< FAILURE!
java.lang.AssertionError: expected:<3> but was:<2>
at
org.apache.hadoop.hbase.io.hfile.bucket.TestBucketCacheRefCnt.testBlockInRAMCache(TestBucketCacheRefCnt.java:118)
However, both of these tests pass on both master and patch branch.
Also, please note that the only difference between the last successful patch,
HBASE-22969.0011.patch, submitted on 10/06/2019 and patches submitted later is
'{color:#57d9a3}*documentation*{color}' as suggested by [~clayb] , and
[~busbey] .
Will any comitter be kind enough to look and provide some guidance on what I am
missing?
> A new binary component comparator(BinaryComponentComparator) to perform
> comparison of arbitrary length and position
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-22969
> URL: https://issues.apache.org/jira/browse/HBASE-22969
> Project: HBase
> Issue Type: New Feature
> Components: Filters
> Reporter: Udai Bhan Kashyap
> Assignee: Udai Bhan Kashyap
> Priority: Minor
> Attachments: HBASE-22969.0003.patch, HBASE-22969.0004.patch,
> HBASE-22969.0005.patch, HBASE-22969.0006.patch, HBASE-22969.0007.patch,
> HBASE-22969.0008.patch, HBASE-22969.0009.patch, HBASE-22969.0010.patch,
> HBASE-22969.0011.patch, HBASE-22969.0012.patch, HBASE-22969.0013.patch,
> HBASE-22969.0014.patch, HBASE-22969.HBASE-22969.0001.patch,
> HBASE-22969.master.0001.patch
>
>
> Lets say you have composite key: a+b+c+d. And for simplicity assume that
> a,b,c, and d all are 4 byte integers.
> Now, if you want to execute a query which is semantically same to following
> sql:
> {{"SELECT * from table where a=1 and b > 10 and b < 20 and c > 90 and c < 100
> and d=1"}}
> The only choice you have is to do client side filtering. That could be lots
> of unwanted data going through various software components and network.
> Solution:
> We can create a "component" comparator which takes the value of the
> "component" and its relative position in the key to pass the 'Filter'
> subsystem of the server:
> {code}
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL);
> int bOffset = 4;
> byte[] b10 = Bytes.toBytes(10);
> Filter b10Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(b10,bOffset));
> filterList.addFilter(b10Filter);
> byte[] b20 = Bytes.toBytes(20);
> Filter b20Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(b20,bOffset));
> filterList.addFilter(b20Filter);
> int cOffset = 8;
> byte[] c90 = Bytes.toBytes(90);
> Filter c90Filter = new RowFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(c90,cOffset));
> filterList.addFilter(c90Filter);
> byte[] c100 = Bytes.toBytes(100);
> Filter c100Filter = new RowFilter(CompareFilter.CompareOp.LESS,
> new BinaryComponentComparator(c100,cOffset));
> filterList.addFilter(c100Filter);
> in dOffset = 12;
> byte[] d1 = Bytes.toBytes(1);
> Filter dFilter = new RowFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComponentComparator(d1,dOffset));
> filterList.addFilter(dFilter);
> //build start and end key for scan
> int aOffset = 0;
> byte[] startKey = new byte[16]; //key size with four ints
> Bytes.putInt(startKey,aOffset,1); //a=1
> Bytes.putInt(startKey,bOffset,11); //b=11, takes care of b > 10
> Bytes.putInt(startKey,cOffset,91); //c=91,
> Bytes.putInt(startKey,dOffset,1); //d=1,
> byte[] endKey = new byte[16];
> Bytes.putInt(endKey,aOffset,1); //a=1
> Bytes.putInt(endKey,bOffset,20); //b=20, takes care of b < 20
> Bytes.putInt(endKey,cOffset,100); //c=100,
> Bytes.putInt(endKey,dOffset,1); //d=1,
> //setup scan
> Scan scan = new Scan(startKey,endKey);
> scan.setFilter(filterList);
> //The scanner below now should give only desired rows.
> //No client side filtering is required.
> ResultScanner scanner = table.getScanner(scan);
> {code}
> The comparator can be used with any filter which makes use of
> ByteArrayComparable. Most notably it can be used with ValueFilter to filter
> out KV based on partial comparison of 'values' :
> {code}
> byte[] partialValue = Bytes.toBytes("partial_value");
> int partialValueOffset =
> Filter partialValueFilter = new
> ValueFilter(CompareFilter.CompareOp.GREATER,
> new BinaryComponentComparator(partialValue,partialValueOffset));
> {code}
> Which in turn can be combined with RowFilter to create a poweful predicate:
> {code}
> RowFilter rowFilter = new RowFilter(GREATER, new
> BinaryComponentComparator(Bytes.toBytes("a"),1);
> FilterLiost fl = new FilterList
> (MUST_PASS_ALL,rowFilter,partialValueFilter);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
