[
https://issues.apache.org/jira/browse/HBASE-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285496#comment-15285496
]
Enis Soztutar commented on HBASE-15765:
---------------------------------------
[~chenheng] with this patch, the handlers still do not steal from the queues in
the same tier, but a different "shard" (the config controlled by
{{hbase.ipc.server.callqueue.handler.factor}} ). The reason we have these
shards is that, the single blocking queue becomes the bottleneck for 10+
readers a producers and 100+ handlers as consumers. I believe previous perf
tests by others proved that having sharding here helps a lot.
Plus, we can steal 1 RPC every ~3seconds if I read the patch right. This will
not have any affect on latency tail smoothing other than potentially working
around deadlock-type cases a la HBASE-15613.
I am not sure whether we can find a good data structure to give us a highly
concurrent multi-priority queue, where every handler type can only take from
its queue, or higher priority ones.
> RPC handlers should steal from other queues in the same or higher priority
> --------------------------------------------------------------------------
>
> Key: HBASE-15765
> URL: https://issues.apache.org/jira/browse/HBASE-15765
> Project: HBase
> Issue Type: Improvement
> Reporter: Enis Soztutar
> Attachments: HBASE-15765.patch, HBASE-15765_v1.patch
>
>
> With the simple rpc scheduler model by default, a request first falls in one
> type of queue (priority, general, replication) and it will get "sharded" in a
> queue within that priority level (for example there are 2 queues for the high
> priority level). The handlers can only take from their specific queues, and
> nothing else.
> Even if a request is waiting in a high priority queue, the handlers within
> the same priority level or a lower priority level will not see these requests
> (which is the root cause for HBASE-15613). We should be able to steal from
> other queues, without losing the prioritization and sharding benefits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
