[
https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379912#comment-14379912
]
ramkrishna.s.vasudevan commented on HBASE-12790:
------------------------------------------------
bq.seems burdensome expecting that clients identify their own scan 'group';
can't we pick an identifier internal to the scan and not bother users with such
detail?
This is based on the Phoenix use case. Pls take a look at
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=blob;f=phoenix-core/src/main/java/org/apache/phoenix/job/JobManager.java;h=31ef7424d4ad04ddbd286c32ef27f187266a7728;hb=a94a6f4195af2867379803f19c90045eb3943c2d.
Phoenix currently tries to use its own thread pool executor to execute the task
in parallel which are grouped.
Currently in HBase we have only FIFO model for which we use the
LinkedBlockingQueue that java provides and for the deadline based prioirty we
use the BoundedPrioirtyQueue impl.
In order to implement the round robin impl based on the Grouping the current
RPC executor does not have any data structure.
Regarding the change to Scan - we could actually do that setting the
Attributes. That was initially thought as the best way. But the thing is that
inside the Executor inorder to identify the Attribute corresponding to the
groupId we have to iterate the list of attributes because from the Protobuf we
don't get a map. If that is fine I can remove the changes to Scan.java and the
patch will not have protobuf changes.
But regarding the round robin nature I think we need a generic queue impl that
would do the round robin work like how a new BoundedPriorityQueue was
implemented just to have the FIFO nature.
> Support fairness across parallelized scans
> ------------------------------------------
>
> Key: HBASE-12790
> URL: https://issues.apache.org/jira/browse/HBASE-12790
> Project: HBase
> Issue Type: New Feature
> Reporter: James Taylor
> Assignee: ramkrishna.s.vasudevan
> Labels: Phoenix
> Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch,
> HBASE-12790_1.patch
>
>
> Some HBase clients parallelize the execution of a scan to reduce latency in
> getting back results. This can lead to starvation with a loaded cluster and
> interleaved scans, since the RPC queue will be ordered and processed on a
> FIFO basis. For example, if there are two clients, A & B that submit largish
> scans at the same time. Say each scan is broken down into 100 scans by the
> client (broken down into equal depth chunks along the row key), and the 100
> scans of client A are queued first, followed immediately by the 100 scans of
> client B. In this case, client B will be starved out of getting any results
> back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead
> of the standard FIFO queue. The queue to be used could be (maybe it already
> is) configurable based on a new config parameter. Using this queue would
> require the client to have the same identifier for all of the 100 parallel
> scans that represent a single logical scan from the clients point of view.
> With this information, the round robin queue would pick off a task from the
> queue in a round robin fashion (instead of a strictly FIFO manner) to prevent
> starvation over interleaved parallelized scans.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
