Hello,
I have been developing a medium/large scale application using twisted
and very happy about it. It is composed of two parts, an application
server and a PyQt based client. Perspective broker is used for all
communication between client and server. Server mainly talks with a
database, formats results and sends to clients (roughly).
Most formatting is done in simple for loops (there are many different
kinds of operations which a client can choose):
@defer.inlineCallbacks
def operation():
dbQueryResult = yield runSomeSQL()
for item in dbQueryResult:
process(item)
Again, most of the time these are light operations which complete in
sub-seconds. However, rarely, results may be so large that it takes
several seconds to process, and during this period all other server
processing stops (i.e. accepting new client connections).
I've written a utility function, deferredSleep(), which I call
occasionally to avoid blocking:
def deferredSleep(seconds=0.01):
d = defer.Deferred()
reactor.callLater(seconds, d.callback, None)
return d
And I use it like below to avoid blocking for very large result sets:
@defer.inlineCallbacks
def operation():
dbQueryResult = yield runSomeSQL()
for i, item in enumerate(dbQueryResult):
if i % 1000 == 0:
# Breathe after a thousand iterations
yield deferredSleep(0.01)
process(item)
So my question is, is this an acceptable solution? And is there a more
generic way, i.e.
for item in magicalIterator(dbQueryResult):
process(item)
I have lots of similar loops all over the place and a more generic
solution would be great. I am also aware of deferToThread but I am
trying to avoid using threads.
Regards,
--
Umit
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
