I'm investigating extended ThreadPoolExecutor to get an Executor that
bases the number of threads on the mean number of tasks over an
interval. If successful, that would flatten out sudden peaks in the task
load, using the queue as a buffer, but would make the number of threads
match a sustained load, so that there would be no unnecessary bottleneck.
The current TaskManager allows specification of e.g. one thread per
three tasks, which does result in a lower rate of change but can leave
the number of threads a fraction of the number of tasks in a sustained
workload.
Here is a general outline of a design:
Methods that change the number of tasks would log the new number of
tasks and a timestamp on a concurrent queue. As a refinement, there may
be an option to use a ThreadLocal Random to log only with a specified
probability to bound the number of messages on the queue.
A thread (more about which thread later) would periodically drain the
queue into a local TreeSet, getting them sorted, and then calculate the
mean number of tasks and adjust the thread count accordingly.
The big design problem is which thread does this work. Here are the
options I'm considering:
1. Add it to the tasks, so that a ThreadPoolExecutor worker thread does
it. The downside is that all the existing worker threads could get busy
on long running tasks and tasks are building up in the queue. Nothing
would be done about it until a worker thread finishes its current task.
2. Do it from the execute method, in whatever thread is adding the task.
Again, we could get into situations in which nothing is done about a
situation that could be improved by adding threads. The effect would be
to make the execute method randomly slow from the point of view of its
caller.
3. Use a dedicated thread that sleeps until it is time to reconsider the
number of tasks, processes the queue of log messages, does any
adjustment, and then goes back to sleep. This is the most reliable in
terms of getting adjustments made when they are needed, but costs the
overhead of an extra thread that is just managing the thread pool.
Any views, alternatives?
Thanks,
Patricia
