[
https://issues.apache.org/jira/browse/HBASE-28043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761089#comment-17761089
]
Becker Ewing edited comment on HBASE-28043 at 9/1/23 2:56 PM:
--------------------------------------------------------------
I've added a new "reverseScan" command to the "hbase pe" tool to assist with
benchmarking. I used the following methodology:
# Start a local HBase cluster with "hbase master start --localRegionServers=1"
# Prepare the table state for benchmarking with "hbase pe --nomapred=true
--valueSize=30 --blockEncoding=PREFIX --compress=GZ randomWrite 1" (this gives
us a table that has many rows in a block)
# Run "hbase pe --nomapred=true reverseScan 1" to benchmark reverse scan
performance by scanning over all rows in this table in reverse
I got the following results:
||Benchmark||Revision||Time (ms)||Throughput (rows / sec)||
|reverse scan run #1|master|52617|19928|
|reverse scan run #2|master|53621|19555|
|reverse scan run #1|patch|34990|29968|
|reverse scan run #2|patch|37270|28135|
|scan run #1 (just for fun :))|master|4509|232552|
So this patch looks to bring the theorized ~50% performance improvement.
was (Author: JIRAUSER301708):
I've added a new "reverseScan" command to the "hbase pe" tool to assist with
benchmarking. I used the following methodology:
# Start a local HBase cluster with "hbase master start --localRegionServers=1"
# Prepare the table state for benchmarking with "hbase pe --nomapred=true
--valueSize=30 --blockEncoding=PREFIX --compress=GZ randomWrite 1" (this gives
us a table that has many rows in a block)
# Run "hbase pe --nomapred=true reverseScan 1" to benchmark reverse scan
performance by scanning over all rows in this table in reverse
I got the following results:
||Benchmark||Revision||Time (ms)||Throughput (rows / sec)||
|reverse scan run #1|master|52617|19928|
|reverse scan run #2|master|53621|19555|
|reverse scan run #1|patch|34990|29968|
|reverse scan run #2|patch|37270|28135|
|scan run #1 (just for fun :))|master|4509|232552|
So this patch looks to bring about a 30% performance improvement.
> Reduce seeks from beginning of block in StoreFileScanner.seekToPreviousRow
> --------------------------------------------------------------------------
>
> Key: HBASE-28043
> URL: https://issues.apache.org/jira/browse/HBASE-28043
> Project: HBase
> Issue Type: Improvement
> Reporter: Becker Ewing
> Assignee: Becker Ewing
> Priority: Major
> Attachments: Current_SeekToPreviousRowBehavior.png,
> Proposed_SeekToPreviousRowBehavior.png
>
>
> Currently, for non-RIV1 DBE encodings, each call to
> [StoreFileScanner.seekToPreviousRow|https://github.com/apache/hbase/blob/89ca7f4ade84c84a246281c71898543b6161c099/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L493-L506]
> (a common operation in reverse scans) results in two seeks:
> # Seek from the beginning of the block to before the given row to find the
> prior row
> # Seek from the beginning of the block to the first cell of the prior row
> So if there are "N" rows in a block, a reverse scan through each row results
> in seeking past 2(N-1)! rows.
>
> This is a particularly expensive operation for tall tables that have many
> rows in a block.
>
> By introducing a state variable "previousRow" to StoreFileScanner, I believe
> that we could modify the seeking algorithm to be:
> # Seek from the beginning of the block to before the given row to find the
> prior row
> # Seek from the beginning of the block to before the row that is before the
> row that was just seeked to (i.e. 2 rows back). _Save_ this as a hint for
> where the prior row is in "previousRow"
> # Reseek from "previousRow" (2 rows back from start) to 1 row back from
> start (to the actual previousRow)
> Then the rest of the calls where a "previousRow" is present, you just need to
> seek to the beginning of the block once instead of twice, i.e.
> # seek from the beginning of the block to right before the beginning of your
> "previousRow" marker. Save this as the new "previousRow" marker
> # Reseek to the next row (i.e. your previous "previousRow" marker)
>
> If there are "N" rows in a block, a reverse scan from row N to row 0 results
> in seeking past approximately (N-1)! rows i.e. 50% less than the current
> behavior.
>
> See the attached diagrams for the current and proposed behavior.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
