[
https://issues.apache.org/jira/browse/HBASE-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985685#comment-13985685
]
Jonathan Hsieh commented on HBASE-10993:
----------------------------------------
Instead of boolean could we make this a string and have some validation it?
'fifo' and 'deadline' seems like reasonable names for the two versions. This
would allow for newer queues to be added without adding a new wonky configs and
invalid config patterns.
{code}
+ /** If set to true, uses a priority queue and deprioritize long-running
scans */
+ public static final String DEADLINE_CALL_QUEUE_CONF_KEY =
"ipc.server.use.deadline.callqueue";
+
{code}
This should have one more consistent tiebreaker -- deadlineA and deadlineB
could be equal (see
http://docs.oracle.com/javase/7/docs/api/java/util/Comparator.html).. hm..
revisiting after reading BoundedPriorityBlockingQueue -- need to put warning on
this saying that ties are explicitly allowed and handled by special queue.
{code}
+ @Override
+ public int compare(CallRunner a, CallRunner b) {
+ RpcServer.Call callA = a.getCall();
+ RpcServer.Call callB = b.getCall();
+ long deadlineA = priority.getDeadline(callA.getHeader(), callA.param);
+ long deadlineB = priority.getDeadline(callB.getHeader(), callB.param);
+ deadlineA = callA.timestamp + Math.min(deadlineA, maxDelay);
+ deadlineB = callB.timestamp + Math.min(deadlineB, maxDelay);
+ return (int)(deadlineA - deadlineB);
+ }
{code}
It isn't how long the scan request lives but how many times it is used (which
is captured by vtime), right?
{code}
+ // get the 'virtual time' of the scanner, and applies sqrt() to get a
+ // nice curve for the delay. longer a scan request lives the less
priority it gets.
+ // The weight is used to have more control on the delay.
{code}
nice, I like how the queue logic is separate from locking behavior of the
queue. (would have been nice to have some javadoc about this pattern on the
private static class PriorityQueue<E> (I thought this was racy until I got to
the next section).
nit: two ';'s
+ // Lock used for all operations
+ private final ReentrantLock lock = new ReentrantLock();;
nit: why this (multiple times). is this an artifact of previous iteration?
+
+ final ReentrantLock lock = this.lock;
+ lock.lock();
this is basically the right idea, but I think the equation is a little off --
shouldn't it the sum of time each call was waiting as opposed to the more
costly 2x penalization of subsequent calls?
{code}
+ int cumsum = 0;
+ for (int i = 0; i < work.size(); ++i) {
+ LOG.debug("Request i=" + i + " value=" + work.get(i));
+ cumsum += cumsum + work.get(i);
+ }
+ LOG.debug("Time cumulative sum=" + cumsum);
+
+ // -> [small small small huge small large small small]
+ // -> NO REORDER [10 10 10 100 10 50 10 10] -> 4150 (FIFO Queue)
10 20 30 130 140 190 200 210 -> 930
// this instead of this sequence
10 30 70 240 490 1030 2070 4150
+ // -> WITH REORDER [10 10 10 10 10 10 50 100] -> 2720 (Deadline Queue)
10 20 30 40 50 60 110 210 -> 530
// this instead of this sequence.
10 30 70 150 310 630 1310 2720
{code}
> Deprioritize long-running scanners
> ----------------------------------
>
> Key: HBASE-10993
> URL: https://issues.apache.org/jira/browse/HBASE-10993
> Project: HBase
> Issue Type: Sub-task
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-10993-v0.patch, HBASE-10993-v1.patch,
> HBASE-10993-v2.patch
>
>
> Currently we have a single call queue that serves all the "normal user"
> requests, and the requests are executed in FIFO.
> When running map-reduce jobs and user-queries on the same machine, we want to
> prioritize the user-queries.
> Without changing too much code, and not having the user giving hints, we can
> add a “vtime” field to the scanner, to keep track from how long is running.
> And we can replace the callQueue with a priorityQueue. In this way we can
> deprioritize long-running scans, the longer a scan request lives the less
> priority it gets.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
