On 02:27 pm, [email protected] wrote:
Hello,
I've been trying to implement a simple notification system via long
polling
using Twisted 14.0.0. On the client side, a web page makes Ajax calls
to an
"update" resource, which will deliver the latest status message known
to
the application. On the server side, a thread feeds a queue with status
events and another one polls this queue and sends the latest value to
any
pending request, trying to keep the request open for 30 seconds, or
until a
new status event arrives, whichever happens sooner (this is the long
polling).
[snip]
class CounterUpdateResource(Resource):
def __init__(self, msg_queue):
Resource.__init__(self)
self.msg_queue = msg_queue
self.msg = {}
self.etag = None
self.pending_requests = []
t = threading.Thread(target=self._poll_messages)
t.daemon = True
t.start()
def _poll_messages(self):
while True:
try:
self.msg = self.msg_queue.get(True, 30)
self.etag = '"%s"' % (uuid.uuid4(),)
print "-> Got new message: %s" % (self.msg,)
except Queue.Empty:
pass
json_str = json.dumps(self.msg)
json_len = len(json_str)
while True:
try:
request = self.pending_requests.pop()
self._actual_render(request, json_str, json_len)
except IndexError:
break
def _actual_render(self, request, json_msg, json_len):
try:
request.setETag(self.etag)
request.setHeader("Content-Type", "application/json")
request.setHeader("Content-Length", json_len)
request.write(json_msg)
request.finish()
except:
pass
`_poll_messages` is run in a thread. It calls `_actual_render` - so
`_actual_render` is run in a thread. It calls these `request` methods -
so they are run in a thread.
Twisted APIs are not thread-safe - except for one or two that are
explicitly marked as being thread-safe.
Since this code calls Twisted APIs (the request methods) in a thread,
they have undefined behavior.
If you change this so that `_actual_render` (or all of the request
methods) are called in the reactor thread then I expect it will start to
work reliably.
Stepping back, you might consider getting rid of the multithreaded in-
process polling and instead make your application fully event-driven.
Instead of having one part of your program put messages into a queue and
another part look at the queue from time to time and do work if it finds
an item, have one part of your program call a method implemented by
another part of your program which does the work. No threads, no
polling, hooray.
Stepping back even further, there are existing libraries for doing this
kind of thing with Twisted already. There's Divmod Nevow's Athena and
there's Minerva. There are also other approaches to "real time" web -
such as websockets (see Autobahn and txwebsocket). You may not actually
have to implement this low-level plumbing yourself.
Jean-Paul
_______________________________________________
Twisted-web mailing list
[email protected]
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
