Hello everybody,
I wasn't able to find any information on how to run Pylons app that
uses SQLAlchemy ORM on top of single-thread coroutine based WSGI
server like gevent.pywsgi.WSGIServer so I decided this information
could be usable for somebody else.
I was about to start small backoffice project when I decided to gevent
package a try. Main benefits I was expecting by using it were:
* no locking for shared obejcts
* ability to use long-polling controller actions
* no thread switching and GIL overhead
* fun (IMHO, that one is a must otherwise any project is doomed)
After reading gevent docs and googling for a while I realized there
was no readily available recipe for what I wanted to do. I found some
posts that lead me to solution that was successfully deployed today.
First of all, database engine has to be "green". In other words it has
to support non-blocking exec(). Luckily, my long time favorite
PostgreSQL has psycopg2 engine that has such a support. It just needed
to be configured properly:
# myproject.lib.gevent
from gevent.socket import wait_read, wait_write
from psycopg2.extensions import set_wait_callback, POLL_OK, POLL_READ,
POLL_WRITE
from psycopg2 import OperationalError
def gevent_wait_callback(conn, timeout=None):
fd = conn.fileno()
while 1:
state = conn.poll()
if state == POLL_OK:
return
elif state == POLL_READ:
wait_read(fd, timeout=timeout)
elif state == POLL_WRITE:
wait_write(fd, timeout=timeout)
else:
raise OperationalError('Unknown connection state %s' %
state)
def make_psycopg2_green():
set_wait_callback(gevent_wait_callback)
Second, threading.local has to be replaced/monkeypatched by
gevent.local in order to make Pylons properly handle single-thread
multiple-request conditions. I couldn't find better solution for that
than patching paster script (I used virtualenv so there was local copy
of it). I just put "from gevent import monkey; monkey.patch_thread()"
as the very first Python code.
Third, there was no paste.server_runner available so I implemented it
and used it in "[server.main]" section of whatever.ini.
# myproject.lib.gevent
def server_runner(app, global_conf, host, port, spawn='default',
**kwargs):
WSGIServer((host, int(port)), app, host=host, port=int(port),
spawn=spawn, **kwargs).run_forever()
The above function can be user as following:
[server.main]
paste.server_runner = myproject.lib.gevent:server_runner
Or, if you add paste.server_runner entry point to your project:
[server.main]
use = myproject#entry_point_name
At that point I was able to do something crazy in controller action:
def action(self):
# next call could "block" on gevent.event.wait() for pretty long
time until content is changed by some other request/activity
content = model.get_content(since=request.if_modified_since,
timeout=WHATEVER_YOU_LIKE)
if not content:
abort(304, 'Not modified')
response.last_modified = model.content_modification_time()
return jsonify(content)
Not to mention I was able to run my existing Pylons + SQLAlchemy
project by making above modifications to it. I have not tried to
measure performance difference because my main goal was not to
increase it but to be able to write long-poll Ajax application using
my beloved Pylons/SQLAlchemy/PostgreSQL combination.
Well, this is probably not the best way to do it but that's what
worked for me. I hope this could save some time to somebody else who
is looking for similar solution.
Cheers,
Alexander
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en.
