[
https://issues.apache.org/jira/browse/HBASE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-9272:
---------------------------------
Attachment: 9272-0.94.txt
So here's a sample patch against 0.94. It does the following:
# An API to parallelize a single Scan.
# Round robin across RegionServers
# Builds its own task queue in order not to rely on a specifically configured
thread pool (i.e. the HTable's pool can be used)
# explores ways of automated scaling. The parallelism is controlled by a
scaling factor that takes the number of a region server touched by the scan
into account
# An alternate API where the caller can pass in a set of Splits (in form of
Scans) and then those are executed on the pool
# limits all thread synchronization to the a BlockingQueue, which (in theory)
allows the reader and the writer to lock independently
# to avoid other synchronization, marker objects are passed to indicate when
the thread is done or encountered an exception
# Also hooked this up with HTable (which is the only questionable - IMHO - part
of this, since it changes HTableInterface and could break client application
that directly implement HTableInterface). This part is not strictly needed,
ParallelClientScanner can be used on its own.
# Pushes a bit more common code into AbstractClientScanner.
Please let me know what you think. If direction is good I'll add tests and make
a trunk patch.
> A simple parallel, unordered scanner
> ------------------------------------
>
> Key: HBASE-9272
> URL: https://issues.apache.org/jira/browse/HBASE-9272
> Project: HBase
> Issue Type: New Feature
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Priority: Minor
> Attachments: 9272-0.94.txt, ParallelClientScanner.java,
> ParallelClientScanner.java
>
>
> The contract of ClientScanner is to return rows in sort order. That limits
> the order in which region can be scanned.
> I propose a simple ParallelScanner that does not have this requirement and
> queries regions in parallel, return whatever gets returned first.
> This is generally useful for scans that filter a lot of data on the server,
> or in cases where the client can very quickly react to the returned data.
> I have a simple prototype (doesn't do error handling right, and might be a
> bit heavy on the synchronization side - it used a BlockingQueue to hand data
> between the client using the scanner and the threads doing the scanning, it
> also could potentially starve some scanners long enugh to time out at the
> server).
> On the plus side, it's only a 130 lines of code. :)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
