Jeongmin Kim created HBASE-30537:
------------------------------------
Summary: PrefixFilter compares bytes past the end of a row shorter
than the prefix
Key: HBASE-30537
URL: https://issues.apache.org/jira/browse/HBASE-30537
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 2.6.7, 3.0.0
Reporter: Jeongmin Kim
HBASE-28621 removed the row length check (if (length < prefix.length) return
true;)
from PrefixFilter.filterRowKey. Since then the filter compares prefix.length
bytes
starting at the row offset, even when the row is shorter than the prefix.
This shows up in two ways.
1. Wrong scan results (reproduced on master)
In a KeyValue the row is followed by the family length and the family, and those
bytes are compared as if they were part of the row.
Table with family "f" and rows "ab", "ab\x00x", "ab\x01f1":
* PrefixFilter("ab\x01f") returns [ab, ab\x01f1], expected [ab\x01f1].
The bytes after row "ab" are the family length (1) and "f", which match the
rest of the prefix, so the row is included.
* PrefixFilter("ab\x00") returns [], expected [ab\x00x].
The family length byte (1) is greater than \x00, so row "ab" is taken to be
past the prefix and the scan ends.
2. ArrayIndexOutOfBoundsException (reproduced on 2.6.6)
When the backing array holds nothing but the row, the comparison runs past the
end
of the array. A scan with explicit columner than the
prefix hits this on store files without delete family markers:
StoreFileScanner.requestSeek skips the se built from the
scan start row, and RegionScannerImpl.nextInternal hands it to filterRowKey.
{code}
java.lang.ArrayIndexOutOfBoundsException: length 5
at
org.apache.hadoop.hbase.util.Bytes$LexicosafeComparer.compareTo(Bytes.java:1579)
at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:1285)
at org.apache.hadoop.hbase.filter.PrefixFilter.java:88)
at
org.apache.hadoop.hbase.filter.FilterListWithOR.filterRowKey(FilterListWithOR.java:345)
at org.apache.hadoop.hbase.regionserverowKey(RegionScannerImpl.java:716)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.nextInternal(RegionScannerImpl.java:497)
{code}
Reproduction: put rows "row-a/1", "row-a/
withStartRow("row-a"), addColumn(f, q) and PrefixFilter("row-a/").
HBASE-27691 stopped filters from seeing that fake cell, so this scan no longer
fails on branches that have it. PrefixFilows when it is
given such a cell directly.
Fix
Compare no more than the row itself: min(rowLength, prefix.length) bytes of the
row
against the prefix. A row shorter than thprefix as far as
it goes then sorts before the prefix, so a forward scan seeks to the prefix with
the hint added by HBASE-28621 and a rever
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
