[
https://issues.apache.org/jira/browse/HBASE-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287179#comment-15287179
]
Enis Soztutar commented on HBASE-15765:
---------------------------------------
bq. It is not steal task every 3 seconds. In the patch, we firstly get task
from our original queue, if timeout (3s), then we will try to steal task from
other queue.
So we block on our queue for 3 seconds before going to the higher priority
queue. Let's say that the high priority queue is all full and our queue is
always empty. We can still process 1 RPC from the higher priority queue every 3
seconds with the patch.
bq. Sorry, i am not sure what is "queues in the same tier"?
In the same priority level, we have (numHandlers / 10) queues by default. So if
numHandlers=60, then we have 6 different queues in the same priority level. The
handler is assigned a queue based on hash, and the handler can only take RPC
requests from its own queue, not the other 5 queues. See this code from
RpcExecutor:
{code}
protected void startHandlers(final String nameSuffix, final int numHandlers,
final List<BlockingQueue<CallRunner>> callQueues,
final int qindex, final int qsize, final int port) {
final String threadPrefix = name + Strings.nullToEmpty(nameSuffix);
for (int i = 0; i < numHandlers; i++) {
final int index = qindex + (i % qsize);
Thread t = new Thread(new Runnable() {
@Override
public void run() {
consumerLoop(callQueues.get(index));
}
});
....
{code}
> RPC handlers should steal from other queues in the same or higher priority
> --------------------------------------------------------------------------
>
> Key: HBASE-15765
> URL: https://issues.apache.org/jira/browse/HBASE-15765
> Project: HBase
> Issue Type: Improvement
> Reporter: Enis Soztutar
> Attachments: HBASE-15765.patch, HBASE-15765_v1.patch
>
>
> With the simple rpc scheduler model by default, a request first falls in one
> type of queue (priority, general, replication) and it will get "sharded" in a
> queue within that priority level (for example there are 2 queues for the high
> priority level). The handlers can only take from their specific queues, and
> nothing else.
> Even if a request is waiting in a high priority queue, the handlers within
> the same priority level or a lower priority level will not see these requests
> (which is the root cause for HBASE-15613). We should be able to steal from
> other queues, without losing the prioritization and sharding benefits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
