[jira] [Comment Edited] (HBASE-17339) Scan-Memory-First Optimization for Get Operation
[
https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783540#comment-15783540
]
Eshcar Hillel edited comment on HBASE-17339 at 12/28/16 7:19 PM:
-
We can use existing TRT, no need for a new one. Only need to remember the
maximal flushed TS so far and make sure the range in TRT is beyond this TS.
The protocol including step-0 is for each operation:
{code}
0. check monotonicity in all relevant stores
if monotonicity is maintained and the optimization is applicable (given list of
columns) then
1. open all relevant *memory* scanners
2. get results
ONLY if result is not complete then
3. open all scanners
4. get results
else
5. open all scanners
6. get results
{code}
possible executions are 0-1-2, 0-1-2-3-4, 0-5-6, the operation returns the
result of step 2 (access only memory) or steps 4/6 (access memory and HFiles)
was (Author: eshcar):
We can use existing TRT, no need for a new one. Only need to remember the
maximal flushed TS so far and make sure the range in TRT is beyond this TS.
The protocol including step-0 is for each step:
{code}
0. check monotonicity in all relevant stores
if monotonicity is maintained and the optimization is applicable (given list of
columns) then
1. open all relevant *memory* scanners
2. get results
ONLY if result is not complete then
3. open all scanners
4. get results
else
5. open all scanners
6. get results
{code}
possible executions are 0-1-2, 0-1-2-3-4, 0-5-6, the operation returns the
result of step 2 (access only memory) or steps 4/6 (access memory and HFiles)
> Scan-Memory-First Optimization for Get Operation
>
>
> Key: HBASE-17339
> URL: https://issues.apache.org/jira/browse/HBASE-17339
> Project: HBase
> Issue Type: Improvement
>Reporter: Eshcar Hillel
> Attachments: HBASE-17339-V01.patch
>
>
> The current implementation of a get operation (to retrieve values for a
> specific key) scans through all relevant stores of the region; for each store
> both memory components (memstores segments) and disk components (hfiles) are
> scanned in parallel.
> We suggest to apply an optimization that speculatively scans memory-only
> components first and only if the result is incomplete scans both memory and
> disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-17339) Scan-Memory-First Optimization for Get Operation
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780751#comment-15780751 ] Phil Yang edited comment on HBASE-17339 at 12/27/16 4:44 PM: - bq. In particular, replication support was deemed to be a blocker at the time. Now we have HBASE-9465, the order can be guaranteed. But if clients also Put Cells into peer cluster, we have two region servers generating timestamp. Unless we change the TS in entries to local TS when peer cluster's RS receives the Cells, we still can not keep monotonicity. But replication is asynchronous which means maybe delay a lot time. So the only solution is synchronous replication or prevent writing to peer cluster. And we may need to distinguish normal client writing and ReplicationSink client writing. The second one can contains a TS while the first one can't. was (Author: yangzhe1991): bq. In particular, replication support was deemed to be a blocker at the time. Now we have HBASE-9465, the order can be guaranteed. But if clients also Put Cells into peer cluster, we have two region servers generating timestamp. Unless we change the TS in entries to local TS when peer cluster's RS receives the Cells, we still can not keep monotonicity. But replication is asynchronous which means maybe delay a lot time. So the only solution is synchronous replication or prevent writing to peer cluster. And we may need to distinguish normal client writing and ReplicationSink client writing. The second one can contains a TS while the first one can't. > Scan-Memory-First Optimization for Get Operation > > > Key: HBASE-17339 > URL: https://issues.apache.org/jira/browse/HBASE-17339 > Project: HBase > Issue Type: Improvement >Reporter: Eshcar Hillel > Attachments: HBASE-17339-V01.patch > > > The current implementation of a get operation (to retrieve values for a > specific key) scans through all relevant stores of the region; for each store > both memory components (memstores segments) and disk components (hfiles) are > scanned in parallel. > We suggest to apply an optimization that speculatively scans memory-only > components first and only if the result is incomplete scans both memory and > disk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
