[
https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15001806#comment-15001806
]
James Taylor commented on HBASE-12790:
--------------------------------------
bq. and make sure if you have internal notions of "this is for query A" and
"that is for query B" that you interleave calls to scanner.next for A and B.
[~apurtell] We already do that, but it won't mitigate the case when a number of
clients run only a single large scan. The RS is the ultimate arbitrator.
bq. Your requirement changes every time you comment
[~stack] Not true. I've stated it clearly above. I've just given you more
examples of why the round-robin-on-connection won't solve this issue. Here's
another one: Apache Drill + Phoenix. There will be large scans parallelized
across many distributed client nodes. It's useful for the RS scheduler to know
that they're linked so that it can round-robin between them. It's similar to
the MR job, but these wouldn't be batch queries, but interactive queries.
When I said it was simple, I'm talking from the users POV. It really is just
having an extra optional attribute on an operation that links them together as
one "virtual operation". I agree that the implementation is kind of a mess.
[~apurtell] - those idea you floated are all good potential workarounds that we
may need to resort to. Would be good to start with a good test case and a
design IMHO. Maybe that's something we can all agree on.
> Support fairness across parallelized scans
> ------------------------------------------
>
> Key: HBASE-12790
> URL: https://issues.apache.org/jira/browse/HBASE-12790
> Project: HBase
> Issue Type: New Feature
> Reporter: James Taylor
> Assignee: ramkrishna.s.vasudevan
> Labels: Phoenix
> Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch,
> HBASE-12790_1.patch, HBASE-12790_5.patch, HBASE-12790_callwrapper.patch,
> HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip
>
>
> Some HBase clients parallelize the execution of a scan to reduce latency in
> getting back results. This can lead to starvation with a loaded cluster and
> interleaved scans, since the RPC queue will be ordered and processed on a
> FIFO basis. For example, if there are two clients, A & B that submit largish
> scans at the same time. Say each scan is broken down into 100 scans by the
> client (broken down into equal depth chunks along the row key), and the 100
> scans of client A are queued first, followed immediately by the 100 scans of
> client B. In this case, client B will be starved out of getting any results
> back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead
> of the standard FIFO queue. The queue to be used could be (maybe it already
> is) configurable based on a new config parameter. Using this queue would
> require the client to have the same identifier for all of the 100 parallel
> scans that represent a single logical scan from the clients point of view.
> With this information, the round robin queue would pick off a task from the
> queue in a round robin fashion (instead of a strictly FIFO manner) to prevent
> starvation over interleaved parallelized scans.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
