[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377404#comment-14377404
]
Lars Hofhansl commented on HBASE-13082:
---------------------------------------
Here's another idea. I think it has been suggested before, but anyway... What
if we had a versioned data structure that kept of the _currently_ visible
files? Each time we compact/flush/bulkload, we create a new version of this. In
the background we retire older versions that are no longer in use by any
scanner. If that leads to any HFile no longer being used by any scanner we
archive them.
Scanners only need to reference the current version upon creation and and
release the reference upon closing (or leas expiry). Since everything is
versioned flushes/compaction can proceed immediately. New scanner will see the
new data, old scanner will work on the old files. The only downside is that
we'll hold on to older files until all scanners using them have finished.
> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
> Key: HBASE-13082
> URL: https://issues.apache.org/jira/browse/HBASE-13082
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt,
> 13082-v4.txt, 13082.txt, 13082.txt, gc.png, gc.png, gc.png, hits.png,
> next.png, next.png
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to
> the lock already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make
> the cores wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking
> contract. For example Phoenix does not lock RegionScanner.nextRaw() and
> required in the documentation (not picking on Phoenix, this one is my fault
> as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load.
> RegionScanner operations would keep getting the locks and the
> flushes/compactions would not be able finalize the set of files.
> I'll have a patch soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
