[
https://issues.apache.org/jira/browse/HADOOP-10282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Li updated HADOOP-10282:
------------------------------
Attachment: HADOOP-10282.patch
Hi [~arpitagarwal] sorry for the delay, I wanted to fix up the tests for more
isolation from the mux and scheduler.
I remember why I made that comment now, it's because poll() without timeunit
and peek() don't acquire the takeLock, so it is possible for poll()/peek() to
return null even though something is in the queue. This doesn't affect hadoop
since hadoop uses poll(long, TimeUnit) exclusively.
I wanted to get this patch out asap, but I think it would also be good to have
pluggable schedulers and multiplexers in the FCQ itself, as it enables some
cool stuff like HADOOP-10598. I will open another issue for making pluggable
schedulers/muxes.
> Create a FairCallQueue: a multi-level call queue which schedules incoming
> calls and multiplexes outgoing calls
> --------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-10282
> URL: https://issues.apache.org/jira/browse/HADOOP-10282
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Chris Li
> Assignee: Chris Li
> Attachments: HADOOP-10282.patch, HADOOP-10282.patch,
> HADOOP-10282.patch
>
>
> The FairCallQueue ensures quality of service by altering the order of RPC
> calls internally.
> It consists of three parts:
> 1. a Scheduler (`HistoryRpcScheduler` is provided) which provides a priority
> number from 0 to N (0 being highest priority)
> 2. a Multi-level queue (residing in `FairCallQueue`) which provides a way to
> keep calls in priority order internally
> 3. a Multiplexer (`WeightedRoundRobinMultiplexer` is provided) which provides
> logic to control which queue to take from
> Currently the Mux and Scheduler are not pluggable, but they probably should
> be (up for discussion).
> This is how it is used:
> // Production
> 1. Call is created and given to the CallQueueManager
> 2. CallQueueManager requests a `put(T call)` into the `FairCallQueue` which
> implements `BlockingQueue`
> 3. `FairCallQueue` asks its scheduler for a scheduling decision, which is an
> integer e.g. 12
> 4. `FairCallQueue` inserts Call into the 12th queue:
> `queues.get(12).put(call)`
> // Consumption
> 1. CallQueueManager requests `take()` or `poll()` on FairCallQueue
> 2. `FairCallQueue` asks its multiplexer for which queue to draw from, which
> will also be an integer e.g. 2
> 3. `FairCallQueue` draws from this queue if it has an available call (or
> tries other queues if it is empty)
> Additional information is available in the linked JIRAs regarding the
> Scheduler and Multiplexer's roles.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
