sunchao commented on a change in pull request #1977:
URL: https://github.com/apache/hadoop/pull/1977#discussion_r416001217
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/FairCallQueue.java
##########
@@ -101,11 +111,18 @@ public FairCallQueue(int priorityLevels, int capacity,
String ns,
this.queues = new ArrayList<BlockingQueue<E>>(numQueues);
this.overflowedCalls = new ArrayList<AtomicLong>(numQueues);
- int queueCapacity = capacity / numQueues;
- int capacityForFirstQueue = queueCapacity + (capacity % numQueues);
+ int totalWeights = 0;
+ for (int i = 0; i < capacityWeights.length; i++) {
Review comment:
what if user passes in negative numbers as weights? should we add a safe
guard against that?
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java
##########
@@ -343,6 +347,36 @@ private static int parseNumLevels(String ns, Configuration
conf) {
return retval;
}
+ /**
+ * Read the weights of capacity in callqueue and pass the value to
+ * callqueue constructions.
+ */
+ private static int[] parseCapacityWeights(
+ int priorityLevels, String ns, Configuration conf) {
+ int[] weights = conf.getInts(ns + "." +
+ CommonConfigurationKeys.IPC_CALLQUEUE_CAPACITY_WEIGHTS_KEY);
+ if (weights.length == 0) {
+ weights = getDefaultQueueCapacityWeights(priorityLevels);
+ } else if (weights.length != priorityLevels) {
+ throw new IllegalArgumentException(
+ CommonConfigurationKeys.IPC_CALLQUEUE_CAPACITY_WEIGHTS_KEY + " must "
+ + "specify " + priorityLevels + " capacity weights: one for each
"
+ + "priority level");
+ }
+ return weights;
+ }
+
+ /**
+ * By default, queue capacity is the same for all priority levels.
Review comment:
nit: add a blank line between this and the first parameter line.
##########
File path:
hadoop-common-project/hadoop-common/src/site/markdown/FairCallQueue.md
##########
@@ -126,6 +126,7 @@ omitted.
|:---- |:---- |:---- |:--- |
| backoff.enable | General | Whether or not to enable client backoff when a
queue is full. | false |
| callqueue.impl | General | The fully qualified name of a class to use as the
implementation of a call queue. Use `org.apache.hadoop.ipc.FairCallQueue` for
the Fair Call Queue. | `java.util.concurrent.LinkedBlockingQueue` (FIFO queue) |
+| callqueue.capacity.weights | General | The capacity allocation weights among
all subqueues. A int array whose length is equal to the
`scheduler.priority.levels` is expected where each int is the relative weight
out of total capacity. i.e. if a queue with capacity weight `w`, its queue
capacity is `capacity * w/sum(weights)` |
Review comment:
maybe make it explicit only positive numbers are allowed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
