[
https://issues.apache.org/jira/browse/HBASE-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008101#comment-14008101
]
Anoop Sam John edited comment on HBASE-11234 at 5/24/14 10:54 AM:
------------------------------------------------------------------
After debugging both 98 and trunk, I got the issue
In PrefixTreeArrayScanner.populateNonRowFieldsAndCompareTo() it calls
CellComparator.compareStatic(this, key). This comparison compares even mvcc
also. In 98 version HFileScanner.seekTo() takes key byte[] to be seeked. So
this will contain the key part alone.
Later in PrefixTreeSeeker#seekToOrBeforeUsingPositionAtOrAfter KV is created
from key
KeyValue kv = KeyValue.createKeyValueFromKey(keyOnlyBytes, offset, length);
So this kv will have an mvcc 0.
The seek operation is seeking to the kv that we are looking for (rk, cf, q, ts,
type) But then the mvcc compare says the seeked kv is less than the seek key
and so we go ahead one more cell.
In trunk Ram's patch changed the HFileScanner.seekTo() signature to take a KV.
As per the test what we pass is the actual KVs written to file (means it will
contain mvcc also.) So we will use that kv only in the code also and so it
works.
But in actual execution scenario what we get is key alone and we will create
KVs (KeyOnlyKeyValue). So this will create same issue what we see in 98 test,
even in trunk also.
To fix we need to make sure the cell compare compares key alone as part of
PrefixTreeArrayScanner.populateNonRowFieldsAndCompareTo()
Also we need to change the test case to pass a seekKey which will not contain
mvcc. (mvcc = 0)
Ping [~mcorgan]
was (Author: anoop.hbase):
After debugging both 98 and trunk, I got the issue
In PrefixTreeArrayScanner.populateNonRowFieldsAndCompareTo() it calls
CellComparator.compareStatic(this, key). This comparison compares even mvcc
also. In 98 version HFileScanner.seekTo() takes key byte[] to be seeked. So
this will contain the key part alone.
Later in PrefixTreeSeeker#seekToOrBeforeUsingPositionAtOrAfter KV is created
from key
KeyValue kv = KeyValue.createKeyValueFromKey(keyOnlyBytes, offset, length);
So this kv will have an mvcc 0.
The seek operation is seeking to the kv that we are looking for (rk, cf, q, ts,
type) But then the mvcc compare says the seeked kv is less than the seek key
and so we go ahead one more cell.
In trunk Ram's patch changed the HFileScanner.seekTo() signature to take a KV.
As per the test what we pass is the actual KVs written to file (means it will
contain mvcc also.) So we will use that kv only in the code also and so it
works.
But in actual execution scenario what we get is key alone and we will create
KVs (KeyOnlyKeyValue). So this will create same issue what we see in 98 test,
even in trunk also.
To fix we need to make sure the cell compare compares key alone as part of
PrefixTreeArrayScanner.populateNonRowFieldsAndCompareTo()
Also we need to change the test case to pass a seekKey which will not contain
mvcc. (mvcc = 0)
> FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
> ------------------------------------------------------------
>
> Key: HBASE-11234
> URL: https://issues.apache.org/jira/browse/HBASE-11234
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Priority: Critical
> Fix For: 0.99.0, 0.96.3, 0.94.21, 0.98.4
>
> Attachments: 11234-94.patch, 11234-96.patch, HBASE-11234.patch
>
>
> As Ted found,
> With this change:
> {noformat}
> Index:
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> ===================================================================
> ---
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> (revision 1596579)
> +++
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> (working copy)
> @@ -51,6 +51,7 @@
> import org.apache.hadoop.hbase.filter.FilterList.Operator;
> import org.apache.hadoop.hbase.filter.PageFilter;
> import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
> +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
> import org.apache.hadoop.hbase.io.hfile.CacheConfig;
> import org.apache.hadoop.hbase.io.hfile.HFileContext;
> import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
> @@ -90,6 +91,7 @@
> CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
> HFileContextBuilder hcBuilder = new HFileContextBuilder();
> hcBuilder.withBlockSize(2 * 1024);
> + hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
> HFileContext hFileContext = hcBuilder.build();
> StoreFile.Writer writer = new StoreFile.WriterBuilder(
> TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
> {noformat}
> I got:
> java.lang.AssertionError:
> expected:<testRow0197/testCf:testQual0000/1400712260004/Put/vlen=13/mvcc=5>
> but was:<testRow0198/testCf:testQual0000/1400712260004/ Put/vlen=13/mvcc=0>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
> at
> org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
> After debugging, it seems the method of
> FastDiffDeltaEncoder#getFirstKeyInBlock become broken. And it will cause
> hfilescanner#seekBefore returns wrong result.
> The solution is simple, see the patch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
