That's cool. I made some tests with a "GeventReactor" with SQLAlchemy
and the psycopg "green" implementation. The results were ~20% faster in
almost all benchs I made, but I didn't tested them in production yet.
On 06/12/2013 05:48 PM, Michael Bayer wrote:
I'm looking forward to Pep3156
(http://www.python.org/dev/peps/pep-3156/) finally producing a working
integration between Twisted and Gevent. Or whatever Nick Coghlan
wants to come up with, since he among anyone has the most crossover
knowledge of the async / ORM-related worlds I've seen.
On Jun 12, 2013, at 4:36 PM, Richard Gerd Kuesters
<[email protected] <mailto:[email protected]>> wrote:
I've been dealing with Twisted since the beggining of my ages with
Python :)
What I can say: it's not an easy job. Here's what I see working:
1. Use Twisted Perspective Brokers to do the database job for you,
AFAIK is the most used combination of Twisted + SQA;
2. Use other types of message brokers (AMQP, for instance);
3. Implement your own "Query" class as a Deferred -- that's what I
use, but is far from complete, tested and documented to become
any kind of project (yet). Inspired on
https://github.com/lunant/SQLAlchemy-Future
Apart this, every single application that I used scoped_session with
deferreds ends up hanging sometime :)
If anyone, of course, knows a rock solid implementation ...
On 06/12/2013 05:17 PM, Michael Bayer wrote:
On Jun 12, 2013, at 2:47 PM, writes_on<[email protected]> wrote:
I'm building a Twisted application in Python 2.7 and am trying to use SqlAlchemy to
interact with the database. I've got a working application that is leaking memory, and am
not sure how to find the leaks. As a "maybe this is the problem" I'm asking if
how I'm using SqlAlchemy might be the source of the leaks. I've written a decorator to
create the sessions I use to interact with the database. The decorator is wrapped around
a function that accepts the session as a parameter and is called in a thread like this:
@inlineCallbacks
def update_db(data)
@db_scoped_session
def process(session):
# do SqlAlchemy work here
# call process in thread
result = yield threads.deferToThread(process)
defer.returnValue(result)
where process is the function wrapped by the decorator. Here is my decorator
code that is creating the session:
def db_scoped_session(engine, auto_commit=True):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
results = None
db_sessionmaker =
scoped_session(sessionmaker(expire_on_commit=True, bind=engine))
db_session = db_sessionmaker()
try:
results = func(db_session, *args, **kwargs)
# should we rollback for safety?
if not auto_commit:
db_session.rollback()
except:
db_session.rollback()
raise
finally:
db_session.commit()
return results
return wrapper
return decorator
Does anyone see anything wrong with what I'm doing here?
from my understanding, you definitely can't use an out-of-the-box
scoped_session with Twisted, as Twisted doesn't have a simple mapping of work
to be done with threads. A scoped_session() uses thread locals. But I'm not
deeply familiar with the mechanics of defer_to_thread in this regard.
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
