[
https://issues.apache.org/jira/browse/HADOOP-10282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Li updated HADOOP-10282:
------------------------------
Attachment: HADOOP-10282.patch
Hi Arpit,
1. LinkedBlockingQueue will be on by default, this can be enabled in the config
2. Because there can be multiple RPC servers, we need to have the configs
specify which server. It was [~sureshms]'s idea to use port number to do this.
3. I'm not sure where the right place will be to document this.
4. Whoops, is fixed in this new patch. Turns out that dropping calls is a
terrible thing for the client.
5. Do you mean to .put() on the high priority queue? That would allow the
current user to be serviced the fastest, but it might cause starvation of other
requests if the queue is constantly waiting to put on queue 0
6. Thanks
7. Suppose so... I think prior versions of this did use that lock recursively.
Any preferences?
8, 9. Actually take() will never be used in hadoop(), maybe it shouldn't even
be implemented? The CallQueueManager uses poll() with timeout. I don't remember
why take was so bad, it might be possible to take(), block, and never get a
result even though something is in the queue...
10. Sounds good.
11. As you know I can't spell :)
Uploaded new patch to fix some of these issues
> Create a FairCallQueue: a multi-level call queue which schedules incoming
> calls and multiplexes outgoing calls
> --------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-10282
> URL: https://issues.apache.org/jira/browse/HADOOP-10282
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Chris Li
> Assignee: Chris Li
> Attachments: HADOOP-10282.patch, HADOOP-10282.patch
>
>
> The FairCallQueue ensures quality of service by altering the order of RPC
> calls internally.
> It consists of three parts:
> 1. a Scheduler (`HistoryRpcScheduler` is provided) which provides a priority
> number from 0 to N (0 being highest priority)
> 2. a Multi-level queue (residing in `FairCallQueue`) which provides a way to
> keep calls in priority order internally
> 3. a Multiplexer (`WeightedRoundRobinMultiplexer` is provided) which provides
> logic to control which queue to take from
> Currently the Mux and Scheduler are not pluggable, but they probably should
> be (up for discussion).
> This is how it is used:
> // Production
> 1. Call is created and given to the CallQueueManager
> 2. CallQueueManager requests a `put(T call)` into the `FairCallQueue` which
> implements `BlockingQueue`
> 3. `FairCallQueue` asks its scheduler for a scheduling decision, which is an
> integer e.g. 12
> 4. `FairCallQueue` inserts Call into the 12th queue:
> `queues.get(12).put(call)`
> // Consumption
> 1. CallQueueManager requests `take()` or `poll()` on FairCallQueue
> 2. `FairCallQueue` asks its multiplexer for which queue to draw from, which
> will also be an integer e.g. 2
> 3. `FairCallQueue` draws from this queue if it has an available call (or
> tries other queues if it is empty)
> Additional information is available in the linked JIRAs regarding the
> Scheduler and Multiplexer's roles.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
