On Tue, 14 Feb 2006 19:20:04 -0500, Rick Morrision <[EMAIL PROTECTED]> wrote:
I've been thinking about this a bit lately, and I no longer think the larger
issue is a simple Twisted integration problem, but a more general problem of
simply getting queries to run in threads, whether those threads are Twisted
threads.deferToThread(), or are simulated "greenlets" threads, or the new
generator-based simulated threads that we'll maybe see soon, or whatever.
I don't know how useful it is to run queries in greenlets or generator-based
threads since the query would block and of course this is not acceptable for a
system that needs a fast response time for every user (and not just the first
one). But given this aim the best strategy is not doing anything at all IMHO.
Every framework has its own way of running stuff in a separate X and all the
frameworks will evolve to integrate with SQLAlchemy.
If we had a generic "execute this in a thread" functionality in the core
engine code that could simply be subclassed for the details of each
threading mechanism's setup/launch/callback particulars, this would make the
library a lot easier to integrate into *any* framework like Twisted, Django,
TurboGears & etc.
Here I disagree. I integrated twisted and SQLalchemy in 50 lines or so. Ed used quite a lot more space and the ending code, since longer is also more complex. Being threading aware rather than threading based is the best choice that SQLAlchemy developers had (IMHO).
Any thread-based query execution is going to involve the idea of a callback
function to handle the completion of the .execute() call, and some way to
guard against the mutability of the parameters. I think your pickle stuff is
a nice approach to solving the mutability issue, and the addition of
callback I think could be added via a small API extension to the various
.execute() functions, perhaps as a keyboard parameter like:
statement.execute(callback = mycallbackfunc)
For now, it seems enough to me to handle just the .execute(), and leave
thorny issues like thread-based result fetching until later, if at all.
Thoughts?
Right here you are already proposing something that Twisted won't accept at
all. In Twisted you have deferreds to deal with callbacks and nobody is going
to accept such an API to register callbacks.
Taking any decision about this issue means that you are making harder for the
others to use SQLAlchemy.
pickling is too much IMHO. Just a deepcopy is enough (and not needed when the
argument is immutable). I like SQLAlchemy like it is, it could maybe ship with
the various 'tricks' to integrate with the various frameworks (like my
decorator and so on).
The main point about the decorator approach is that you have a complete control
over the transaction and the developer is exposed to this control because he is
the person that has to think about this (SQLAlchemy exposes the transaction
mechanism too in some way or another, another great point about it) and I don't
think it's good to hide the transaction machinery. Note that the decorator
approach is not twisted specific. You could easily keep the same structure and
use another threading approach instead (only one line in the decorator is
twisted specific, everything else only depends on SQLAlchemy).
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Sqlalchemy-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users
