I had inquired a few days ago about Twisted integration, and had just recently come across Valentino Volonghi's function decorator code (it was buried in a thread about changes to RowProxy).
I've been thinking about this a bit lately, and I no longer think the larger issue is a simple Twisted integration problem, but a more general problem of simply getting queries to run in threads, whether those threads are Twisted threads.deferToThread(), or are simulated "greenlets" threads, or the new generator-based simulated threads that we'll maybe see soon, or whatever.
If we had a generic "execute this in a thread" functionality in the core engine code that could simply be subclassed for the details of each threading mechanism's setup/launch/callback particulars, this would make the library a lot easier to integrate into *any* framework like Twisted, Django, TurboGears & etc.
Any thread-based query execution is going to involve the idea of a callback function to handle the completion of the .execute() call, and some way to guard against the mutability of the parameters. I think your pickle stuff is a nice approach to solving the mutability issue, and the addition of callback I think could be added via a small API extension to the various .execute() functions, perhaps as a keyboard parameter like:
statement.execute(callback = mycallbackfunc)
For now, it seems enough to me to handle just the .execute(), and leave thorny issues like thread-based result fetching until later, if at all.
Thoughts?
Rick
On 2/14/06, Ed Suominen <[EMAIL PROTECTED]> wrote:
I am a new but very enthusiastic user of sqlalchemy and have been
happily banging out some code to use it. Like Valentino Volonghi (but by
no means in the same league), I am also a big fan of Twisted and want to
integrate my use of sqlalchemy into Twisted apps.
Before I get too deep into unit testing (yes, I wrote the code first,
shame on me), I'd like to get some feedback from both sqlalchemy and
twisted users about what they think of the approach in my module
database.py, which I have posted to http://edsuom.pastebin.com/555036.
A use case, for my persistent graph package that's going to rely on all
this, is posted to http://edsuom.pastebin.com/555040. Note how some
database operations are simply encapsulated into a self.transact() call
while others, involving several lines of code, are put into a local
function and encapsulated into a self.transact(thisTransaction) call at
the end of the method.
Some assorted coolness:
* You can use Twisted's deferred result mechanism or block, even on a
case-by-case basis.
* Engines are defined at the class level, for all tables and contexts.
Tables are defined, also at the class level, for all contexts. Cached
selects are defined, however, on a context-by-context basis.
* In all three cases, thread-local storage permits even cranky databases
like sqlite from complaining about connection objects created in one
thread being used in a different thread.
* Overridden attribute access is used to construct, or retrieve from
cache, the needed goodies on the fly. Advantageously, those goodies are
constructed in the context not only of the current Table (in the case of
cached select() objects) but also in the current thread.
Comments welcome -- positive, critical, or utterly disdainful!
- Ed Suominen
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Sqlalchemy-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users
