[
https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997232#comment-14997232
]
stack commented on HBASE-12790:
-------------------------------
Thanks for the eloquence [~apurtell] summarizing where we are at and how we got
here using illustration from our community past. I also appreciate and agree
with your outline of the positions involved and for hitting the reset button
(should have been done long ago).
When we reset, can the discussion be more grounded?
For example, "Opening a new HConnection takes three seconds..." I just measured
it against a cluster. Total connection setup takes 20ms... less if I let the
JVM warm up. Its 3.5 seconds to start a cold JVM, hit about 40 regions, then
completely shut all down again. Improving Connection setup is an issue many
could benefit from so would be worth doing in general. We can work on sharing
cache/zk over Connections too.
Or, "It's not feasible to pool enough HConnections to support the concurrency
we see over the various types of queries."... Please substantiate your claim
with numbers and use case.
> Support fairness across parallelized scans
> ------------------------------------------
>
> Key: HBASE-12790
> URL: https://issues.apache.org/jira/browse/HBASE-12790
> Project: HBase
> Issue Type: New Feature
> Reporter: James Taylor
> Assignee: ramkrishna.s.vasudevan
> Labels: Phoenix
> Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch,
> HBASE-12790_1.patch, HBASE-12790_5.patch, HBASE-12790_callwrapper.patch,
> HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip
>
>
> Some HBase clients parallelize the execution of a scan to reduce latency in
> getting back results. This can lead to starvation with a loaded cluster and
> interleaved scans, since the RPC queue will be ordered and processed on a
> FIFO basis. For example, if there are two clients, A & B that submit largish
> scans at the same time. Say each scan is broken down into 100 scans by the
> client (broken down into equal depth chunks along the row key), and the 100
> scans of client A are queued first, followed immediately by the 100 scans of
> client B. In this case, client B will be starved out of getting any results
> back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead
> of the standard FIFO queue. The queue to be used could be (maybe it already
> is) configurable based on a new config parameter. Using this queue would
> require the client to have the same identifier for all of the 100 parallel
> scans that represent a single logical scan from the clients point of view.
> With this information, the round robin queue would pick off a task from the
> queue in a round robin fashion (instead of a strictly FIFO manner) to prevent
> starvation over interleaved parallelized scans.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
