dan -
just to make sure, we arent calling for a rewrite to the whole API
for SQLAlchemy. While the internals of the using() function are
hacking around the global session registry, i am as interested as you
are in making it more inline, since at the very least it would
operate more efficiently. I am definitely interested in improving
SA's support for explicit sessions throughout, and making the Session
registry a totally optional thing, if one wants to specify explicit
Sessions everywhere. the code is very close to being able to do this
anyway, there are only a few places in Mapper where it uses
get_session() which can be wrapped with some extra contextual logic,
which will be similar to the Query idea youre getting at, and also
fixes the lazyloader thing youre having...im adding a few more
"session" arguments to some mapper methods right now that should get
at least that part working.
but if I may defend the concept of a registry, its a very well
accepted pattern, and thread-bound transactions are used by
Hibernate /JTA to eliminate the need for transactional objects to be
passed to all method calls everywhere. I dont really think the "they
taught you in school that global variables are bad" argument is very
pertinent here. heres fowlers description of it: http://
www.martinfowler.com/eaaCatalog/registry.html .
of course a global registry is entirely inapprorpriate for your
application, and restructuring SA to not be dependent on it is
something we agree on (even though I could make it all work along the
lines of the using() idea, its simply wasteful of processing time to
have to push/pop onto a stack for every method call).
On Apr 6, 2006, at 1:47 PM, dmiller wrote:
As you may know if you've been reading this list for a while, I'm
working on a GUI application that can have multiple "document"
editors open at the same time. Since it's not a web application, I
don't have user interactions nicely diced into isolated requests.
Rather, a user may be editing a document in one window, then load a
new window and drag an item from the new window back to the
original window. Since the user may be editing many different
"documents" at the same time (each of which consists of a graph of
related data entities), I decided to give each editor (window) it's
own SQLAlchemy session. Objects in each session are effectively
isolated from eachother and can be edited and saved without
affecting other open documents.
SQLAlchemy's heavy dependence on the thread-local session pattern
is giving me bad headaches. I need to be able to guarantee that all
data entities belonging to a given document are loaded in a
specific session, but the thread-local pattern doesn't always allow
me to do that (it will silently load some objects in the wrong
session and I'll have strange bugs as a result).
Since SQLAlchemy has been written very well, it is almost possible
to do what I want by monkey-patching the parts of SA that deal with
thread local. However, I really want to avoid that because it would
most likely make very painful experience each time I upgrade to a
new version of SQLAlchemy.
[Aside] Yesterday I updated my SA source from r1025 to HEAD. I
haven't found anything broken yet, which is a huge testament to the
quality of development used on this project! Way to go Mike!!
I had tried to do a session-per-window pattern by monkey-patching
objectstore.session_registry to consult my application for the
active window's session. However, this broke down with drag/drop
between windows. If an object is dragged from the active window
(we'll call it window_1) and dropped on a non-active window
(window_2), then objects loaded from the database by window_2 in
response to the drop are loaded in window_1's session. Ouch. In
fact, this will always happen when two windows need to communicate
with eachother: only one window can be active at a time. It's even
possible that neither window involved in the communication is
active, which means that both windows involved in the communication
will be loading their objects in some other window's session.
Double ouch.
Luckily I have confined the data access code to a few specific
areas, so whatever solution I come up with will not require me to
rewrite the entire application.
So I set out to find out exactly what SA is doing behind the
scenes. First, I monkey-patched objectstore.session_registry to
disable thread-local sessions. This allowed me to find out when the
default thread-local session was being used:
class NotImplementedProxy(object):
def __getattr__(self, name):
raise NotImplementedError("thread-local sessions are disabled")
def __setattr__(self, name, value):
raise NotImplementedError("thread-local sessions are disabled")
sa.objectstore.session_registry = sa.util.ScopedRegistry
(NotImplementedProxy)
This forced me to execute every mapper statement "using(session)",
which was exactly what I wanted anyway. The problem arises when I
load an object object with lazy attributes. When I do obj.lazy_attr
I get a "thread-local disabled" exception:
Traceback (most recent call last):
...
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/
attributes.py", line 62, in __get__
return self.manager.get_list_attribute(obj, self.key)
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/
attributes.py", line 345, in get_list_attribute
return self.get_history(obj, key, **kwargs)
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/
attributes.py", line 414, in get_history
return self.get_unexec_history(obj, key).history(**kwargs)
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/
attributes.py", line 256, in history
value = self.callable_()
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/mapping/
properties.py", line 618, in lazyload
result = self.mapper.select_whereclause(self.lazywhere,
order_by=order_by, params=params)
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/mapping/
mapper.py", line 557, in select_whereclause
return self._select_statement(statement, params=params)
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/mapping/
mapper.py", line 577, in _select_statement
return self.instances(statement.execute(**params), **kwargs)
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/mapping/
mapper.py", line 317, in instances
self._instance(row, imap, result,
populate_existing=populate_existing)
File "/Users/dmiller/Code/SQLAlchemy/lib/sqlalchemy/mapping/
mapper.py", line 871, in _instance
if sess.has_key(identitykey):
File "<path to NotImplementedProxy>", line 3, in __getattr__
raise NotImplementedError("thread-local sessions are disabled")
NotImplementedError: thread-local sessions are disabled
The problem is with Mapper._instance(...), which calls
objectstore.get_session() to get a session to load the lazy
attribute. It should check for obj._sa_session and use that
instead. If that session is not valid, the mapper should raise an
exception about the object not being associated with a session
(this is what Hibernate does).
Note that this problem could happen to anyone who loads some
objects, pushes a new session, then loads lazy attributes of the
original objects. The child objects loaded by lazy attributes will
be loaded in the new session. In most cases we want child objects
to be loaded in the same session as the parent object.
I hope I have communicated that I have a use case where thread-
local is not appropriate. Now I will try to show that the session-
resolution mechanism can be abstracted to a well-defined interface.
The thread-local pattern can simply be the default implementation,
but it will be pluggable so any pattern may be used.
The main thing that needs to go is the notion that there is a
global session somewhere. This is a problem with the current mapper
implementation. Remember way back in CS1 when they told you not to
use global variables? This is why. Whether you like it or not,
thread-local is a global variable (within the context of a thread).
It's handy because a resource can be stored there and any function
that needs it can just use it. This is where the beauty of object
oriented programming comes in. The concept of an object allows
initialization code to put some stuff on the object and then later
it can use that stuff without referring to a global context. This
is what needs to happen with the mapper.
The "using" thing is a hack that says "swap the thread-local
session for a moment while I do this thing". In theory this will
work, but in practice (when lazy attributes are loaded) the mapper
tries to use the thread-local session when I'm not around to say
"use this session".
This could be much simpler if there was a concept of a session/
engine-bound Query. This Query object would execute SQL constructed
by a mapper on an engine and store the results in a session. It
would decouple the mapper from the engine and the thread-local
session. It would become the job of the query to execute(...)
mapper-generated-SQL on it's engine and then associate all
resulting mapped objects with it's session. Session-bound objects
would use their session to load lazy attributes (exactly what
Hibernate does, except in Hibernate the reference to the session is
stored in the lazy attribute proxy instead of on the parent
object). A thread-local implementation of this Query object would
be trivial to write, and that implementation could be the default
used by everyone who doesn't want to worry about session scoping.
I realize this covers some of the same material I have already
covered in the "Decoupling Tables/Mappers from engine" thread. I
wrote that on my own spare time for the benefit of SQLAlchemy. I'm
writing this from work where I actually need a real solution (I'm
not complaining or saying the community must provide one--I know
that's not what open-source is about). In the short term I'll try
to hack my own solution as best I can. I'll try to contribute
patches as I come up with them.
Thanks for your help.
~ Daniel
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting
language
that extends applications into web and mobile media. Attend the
live webcast
and join the prime developer group breaking into this new coding
territory!
http://sel.as-us.falkag.net/sel?
cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Sqlalchemy-users mailing list
Sqlalchemy-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Sqlalchemy-users mailing list
Sqlalchemy-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users
