Hello,
I have a list of tasks/items that I want handed off to threads/processes to
complete. (I would like to stick with process if I could, since there is some
CPU work here. )
Each task involves some calculations and a call to a remote server over
urllib2/HTTP.
The time to complete each task varies from 1 to 20 seconds depending on a
number of factors including variable delay on the remote server.
I would like to:
1) Have a maximum of 20 "in-flight" tasks. (thus worker processes?)
2) Not overload the external server that each task is calling. No more than "3
new tasks" per second. More "waiting" tasks may be OK, i need to test it.
3) Certain tasks in my list must be processed in the correct order. (I guess
the asignment logic must somehow tag those to by done by the same worker?)
Do any of you have suggestions? Can someone point me in the direction of sample
code for this?
Thank you,
:)
_______________________________________________
Tutor maillist - [email protected]
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
