I announce a pre-alpha version of TaskPoolKit:
TaskPoolKit is a package that mantains a TaskPool, a ThreadPool and Workers.
The Workers process tasks, the ThreadPool maintains a dynamic number of
workers,
and TaskPool maintain a queue of tasks to be processed.
The public interface of the package is TaskPool, the other are of
intrapackage use only.
TaskPool has this public methods:
addTask(task, *args, **kargs): add a task to be procesed by a worker
asynchronously.
if there are to many tasks
in the queue, wait, if you can't
add even waiting then raise
Full
addTaskNoWait: the same as addTask but raises Full without retrying
asyncCall(function, *args, **kargs): return a ReturnValue instance:
returnvalue.done(): return
true if the work is finished
returnvalue() return the
result
The function is procesed
asynchronously
The number of workers are dynamic, they are maintained between minThread and
maxThreads.
In the __init__ file is a fast hack that adds a TaskPool instance to
application:
you can do:
application.addTask(tas, *arg, **kargs) and the other methods
You can process long jobs in your servlet (for example indexing a database,
do ftp, start sending emails, ...)
It is simple, write in your code:
app = self.application()
app.addTask(self.doLongJob, arg1, arg2)
A worker will execute doLongJob(arg1, arg2)
My idea is that WebKit can use internally TaskPool ( self.addTask
request.handleRequest) ) .
I think that this is the first step to a MultiAppServer arquitecture. Each
AppServer invokes
addTask from a unique taskPool. Then all the threads are shared between all
the AppServers.
TaskPool.py has a test() function, try it.
@@TODO
Make a better integration of Application and TaskPool:
include in Application self._taskPool and the public methods of TaskPool
directly accesible from Application.
Include TaskPool.shutDown() in Application.shutDown()
Include TaskPool parameters in config file
Others
What do you think? I need feedback.
Can this be included in Webware?
If all is correct I will work in a MultiAppServer Webware. My idea is:
AppServer ------>
AppServer------> Application <---- TaskPool
AppServer------>
And application has multiple appservers. Each appserver listen a port
in an eternal cicle.
Each appserver add tasks to the TaskPool, the workers execute the task.
All the AppServers share the same TaskPool, and thus they share
the same threads.
TaskPoolKit.ZIP
