OK, responding to D. Miller's great ideas and taking far longer than 10 minutes this time, heres #2...which actually does not involve much of a deep change to things, its largely a naming/rearrangement type of thing with a few extra tricks...so thats a good thing ! as i am not so down with totally rewriting stuff at this point.

The SQLEngine architecture, as well as the objectstore and everything else, heavily relies upon thread-local storage to share resources. So far, there has not really been much concept of a "connection" outside of the pooling module since it was all built in such a way theres usually nowhere to pass a "connection" around, its just implicitly available when needed (and implicitly returned to the pool when not). But it is nice to say, "conn = engine.connect()", with a nice verb and a tangible thing to hold onto. So this proposal, while not ditching the basic "implicit" behavior of SQL statement execution as well as Session construction that I think is pretty popular, introduces some more "explicitness" when its desired to use concurrent SQL connections or transactions, or concurrent Sessions, or a combination of both. It also simplifies the Session interface and removes all transaction-like terminology from the ORM package. there is actually very little new here, its mostly a rearrangement of terminology.

Part I.  SQL construction, transactions, engines. note that we
arent dealing with the SA ORM at all, no mappers, objectstores,
they are not not involved here.
==============

# 1. engine. nothing new here. it represents a particular connection URL using # the dialect (to use a Hibernate term) of a particular vendor's database.

    e = create_engine('foo')

# SQL statements constructed against the engine use it implicitly to # execute themselves, connections are pulled from the pool behind the scenes and released.
    mytable.update().execute()

# 2. transaction, the implicit way, slightly new but not much.

    trans = e.begin()
    try:
        mytable.update().execute()
        trans.commit()
    except:
        trans.rollback()

the "trans" object is called a SQLTransaction, which doesnt do much except provide the "commit" and "rollback" function, but there you go, the "tangible transaction object" to hold onto.

# 3. the connection. this is "new" (its currenly called a SQLSession...but that name changes to ConnectionProxy) generally used to say "do this with this explicit resource". this is to make everyones needs for what I was calling "nested" stuff possible, but youre right its not really "nested", so now theres a tangible thing to hold onto (well there has been for several weeks now, but now its the primary way to do it).

# this is a ConnectionProxy object. it represents a single unique connection from the # pool, which it grabs when needed, or when a transaction is started it holds onto it. # this object is used automatically with the implicit stuff in #1, #2
    conn = e.connection()

# 3a. transactions on the connection. same deal, same SQLTransaction object:

    trans = conn.begin()
    trans.commit()

# 3b. but heres the new thing. since we have this nice tangible ConnectionProxy, lets tangibly use it. # We stick "using()" on the base ClauseElement (which is the base of all Table/Select/Alias/etc objects),
# so you can use different connections all over the place.

    mytable.using(conn).select().execute()
    mytable.select().using(conn).execute()

# or maybe we do the keyword thing
    mytable.select(connection=conn).execute()

# 3c. or if you want the "implicit" stuff to use this ConnectionProxy, to bypass current transactional state, introduce new tranasctional state, or even connect to some other database, and have it affect everything going forward (on the current thread), do this:

    engine.set_current(connection)  # now everyone uses 'connection'
    engine.restore_previous()   # put the old one back.

# 3d. 'using' can take engines or connections:

    e1 = create_engine('postgres://foo')
    e2 = create_engine('mysql://bar')

    t = Table('mytable', Column('col1', ...) )

    t.select().using(e2).execute()

    t.using(e1.connection()).select().execute()

# those SQL constructs are looking pretty "engine-independent" now arent they ? (this behavior has been available all along, just not as succinctly)

# 3e. switch the current engine to use a different engine's connection. this is a hacky thing. There is a user who is writing his own "clustered database" solution involving data stored in redundant tables across multiple DBs, he needed something exactly like this:

    e1 = create_engine('postgres://foo')
    e2 = create_engine('postgres://bar')

e1.set_current(e2.connection()) # now everyone that uses e1, secretly uses e2
    e1.restore_previous()  # put it back

# in theory, things should be designed so that the "dialect" of e2 gets used also if you swap it into e1. might need a little more internal rearranging for that.

# 4. natural nesting behavior

As always, begin/commit/rollback uses the "outermost nested" idea, where "nested" calls to begin/commit dont affect the single "outermost" transaction.

e = create_engine('foo')
trans = e.begin()
try:
    mytable.update().execute()

    # then in some other function..
    trans2 = e.begin()
    try:
        myothertable.update.execute()
        trans2.commit()   # doesnt commit
    except:
        trans2.rollback()   # rolls trans2, trans

    trans.commit()   # commits mytable, myothertable
except:
    trans.rollback()  # rolls back trans

So nesting just means that kind of nesting. The other thing i was calling nesting, yes, its just two concurrent connections, its not really "nesting", for that you just use a second ConnectionProxy, and you can optionally use set_current/restore_previous to make it take over the "implicit" scope.

So to sum up Part I, heres the full vocabulary to learn regarding connections, engines, transactions: SQLEngine, ConnectionProxy, SQLTransaction. begin(), connection(), using(), commit(), rollback (), set_current(), restore_previous().

====================================

Part II. Objectstore stuff. The ORM is a system for emitting SQL. it uses the underlying SQL construction to do its work. There is the objectstore module, the Mapper object and friends, and the Session object, and thats mostly it. The Session object gets simplified, begin()/commit() go away (theyll hang around in deprecated state), and with regards to scoping, you get the basic terminology flush(), Session(), mapper.using(), set_current() and restore_previous(), and thats about it. so all the transactional terminology is removed from objectstore/session since Session doesnt have much to do with tranasctions, except that it uses one for its own flush() operation. if you want your own SQL transaction demarcation, you use the engine stuff as normal, so only one set of concepts to learn. The Session is no more than a "bag of objects" which can "flush" SQL statements to the database.

# 5.  simple objectstore work

obj = mapper.get(1)
obj.foo = 'bar'
objectstore.flush()  # flush everything

obj2 = mapper.get(2)
obj2.foo = 'bar'
objectstore.flush(obj, obj2)  # flush just two objects

# 6. multiple objectstore sessions. these are just two separate "bags of objects". they arent doing anything explicit with the underlying SQLEngines so are therefore using whatever ConnectionProxy's are current for them.

obj1 = mapper.get(1)
obj1.foo = 'bar'

sess = Session()  # second session
obj2 = mapper.using(sess).get(2)
obj2.foo = 'bar'
sess.flush()  # saves obj2

objectstore.flush()  # saves obj1

# 7. multiple objectstore sessions, sharing the same identity map space. this is similar to when we were using "begin()/commit()" with the Session.

obj1 = mapper.get(1)
obj1.foo = 'bar'

sess = Session(new_imap=False) # shares the identity map from the current thread's session
obj2 = mapper.using(sess).get(2)
obj2.foo = 'bar'
sess.flush()  # saves obj2

objectstore.flush()  # saves obj1

# 8. setting a new Session to be the current thread's session, so that all subsequent calls use it automatically.

obj1 = mapper.get(1)
obj1.foo = 'bar'
sess = Session()  # new session
objectstore.set_current(sess)  # set it as current
try:
    obj2 = mapper.get(2)
    obj2.foo = 'bar'
    objectstore.flush() # saves obj2
finally:
    objectstore.restore_previous()
objectstore.flush()  # saves obj1

# so really, what before was session.begin()/commit(), is now more or less like objectstore.set_current(Session(new_imap=False))/ objectstore.flush()/objectstore.restore_previous(). I guess the use case for this was so that you can write a function that works with some objects and then saves them, without affecting the objects that were manipulated outside of the function...but probably easier to just use the mapper.using() syntax. if people want *exactly* what was begin/commit before, within a simple pair of functions as opposed to this more verbose style, we can do that too...but they are *not* going to be called begin/commit. too confusing !

============================

transactions + objectstore stuff

# 9. the Session is just an object that knows how to spit out a bunch of UPDATE etc. statements. Transactions are used via the engine as from Part I:

e = create_engine('foo')
trans = e.begin()
try:
    obj1 = mapper.get(1)
    obj1.foo = 'bar'
    objectstore.flush()
    trans.commit()
except:
    trans.rollback()

# 10. A shortcut for when you want a particular ConnectionProxy (or set of ConnectionProxies) to go into effect corresponding to a particular Session. This ConnectionProxy gets used for all operations with this Session which involve the given ConnectionProxy's underlying engine:

e = create_engine('...')
conn = e.connection()
sess = Session(connections=[conn])

obj = mapper.using(sess).get(3) # all SQL for conn's engine will use the conn above for this operation
sess.flush()  # commit using conn

objectstore.set_current(sess) # now all ORM operations on both Mapper and Session use conn implicitly
objectstore.restore_previous()


- mike




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Sqlalchemy-users mailing list
Sqlalchemy-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users

Reply via email to