I'm looking forward to Pep3156 (http://www.python.org/dev/peps/pep-3156/) finally producing a working integration between Twisted and Gevent. Or whatever Nick Coghlan wants to come up with, since he among anyone has the most crossover knowledge of the async / ORM-related worlds I've seen.
On Jun 12, 2013, at 4:36 PM, Richard Gerd Kuesters <[email protected]> wrote: > I've been dealing with Twisted since the beggining of my ages with Python :) > > What I can say: it's not an easy job. Here's what I see working: > Use Twisted Perspective Brokers to do the database job for you, AFAIK is the > most used combination of Twisted + SQA; > Use other types of message brokers (AMQP, for instance); > Implement your own "Query" class as a Deferred -- that's what I use, but is > far from complete, tested and documented to become any kind of project (yet). > Inspired on https://github.com/lunant/SQLAlchemy-Future > Apart this, every single application that I used scoped_session with > deferreds ends up hanging sometime :) > > If anyone, of course, knows a rock solid implementation ... > > > On 06/12/2013 05:17 PM, Michael Bayer wrote: >> On Jun 12, 2013, at 2:47 PM, writes_on <[email protected]> wrote: >> >>> I'm building a Twisted application in Python 2.7 and am trying to use >>> SqlAlchemy to interact with the database. I've got a working application >>> that is leaking memory, and am not sure how to find the leaks. As a "maybe >>> this is the problem" I'm asking if how I'm using SqlAlchemy might be the >>> source of the leaks. I've written a decorator to create the sessions I use >>> to interact with the database. The decorator is wrapped around a function >>> that accepts the session as a parameter and is called in a thread like this: >>> >>> @inlineCallbacks >>> def update_db(data) >>> @db_scoped_session >>> def process(session): >>> # do SqlAlchemy work here >>> >>> # call process in thread >>> result = yield threads.deferToThread(process) >>> defer.returnValue(result) >>> where process is the function wrapped by the decorator. Here is my >>> decorator code that is creating the session: >>> >>> def db_scoped_session(engine, auto_commit=True): >>> def decorator(func): >>> @functools.wraps(func) >>> def wrapper(*args, **kwargs): >>> results = None >>> db_sessionmaker = >>> scoped_session(sessionmaker(expire_on_commit=True, bind=engine)) >>> db_session = db_sessionmaker() >>> try: >>> results = func(db_session, *args, **kwargs) >>> # should we rollback for safety? >>> if not auto_commit: >>> db_session.rollback() >>> except: >>> db_session.rollback() >>> raise >>> finally: >>> db_session.commit() >>> return results >>> return wrapper >>> return decorator >>> >>> Does anyone see anything wrong with what I'm doing here? >> from my understanding, you definitely can't use an out-of-the-box >> scoped_session with Twisted, as Twisted doesn't have a simple mapping of >> work to be done with threads. A scoped_session() uses thread locals. But >> I'm not deeply familiar with the mechanics of defer_to_thread in this regard. >> > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
