Ahh...sorry, I keep emailing you directly Mike...I need to remember
to always check the "To" address before sending...
On Feb 9, 2006, at 5:03 PM, Michael Bayer wrote:
dmiller wrote:
question: does having each object maintain a reference to a
unitofwork
object impact its serializabilty (oh, i guess you mentioned
that...)?
No. If an object is serialized it should not retain the reference to
the unitofwork.
OK, but this is a problem since the Pickle process can't easily be
customized. I want an object to be able to be dropped into
pickle.dumps()
without any extra operations. hence some string-based identifier
would be
better since it doesnt get in the way of a pickle operation.
I do not
know how pickle handles weak references, do you?
My guess would be that it raises an exception.
Yes, I just tried it and it does...bummer. This works though:
<code>
import pickle
import weakref
def set_weakref(obj, attr, value):
setattr(obj, attr, weakref.ref(value))
oldgetstate = getattr(obj, "__getstate__", None)
def __getstate__(self):
if oldgetstate is not None:
d = dict(oldgetstate())
else:
d = dict(self.__dict__)
d.pop(attr, None)
d.pop("__getstate__", None)
return d
obj.__getstate__ = __getstate__.__get__(obj)
class A(object):
pass
a = A()
b = A()
set_weakref(a, "b_ref", b)
s = pickle.dumps(a)
c = pickle.loads(s)
print a.__dict__ # b_ref is here
print c.__dict__ # b_ref is gone
</code>
- a given object cannot be associated with more than one unit of
work
(is this a problem? are there use cases for associating an object
with more than one uow concurrently?)
not concurrently, but the unit of work begin/commit pattern
creates a
second child unit of work.
that shouldn't be a problem
well, it implies the objects are keeping a reference to some
contextual
object that knows about the parent unit of work, or something.
also, it is supported behavior that the
current thread-local unit of work can be changed at any time. for
this
reason, I think associating objects with a unit of work would have
to be
done through some intermediary scope object, along the lines of your
previous patch.
This brings up another thing: it would be possible for an object to
reference a unit of work that did not know about the object. Is that
a problem?
the unit of work contains an Identity Map which represents all
database-identified objects being operated upon. Any object that
wishes
to deal with a particular unitofwork must be in this map. The only
two
ways a database identified object gets called to the scene is from
being
loaded/saved via a mapper, which puts it in the current identity
map, or
from the import_instance method. How exactly would an object
reference
the unit of work but not be in its identity map ? (other than, they
deserialized and did not use import_instance) ?
There would be no other way, which is good. With the above
set_weakref code serialization shouldn't cause a problem.
I agree about the mapper interface. Maybe we could do something like
mapper.with(uow).select(...)
I am getting the impression youre inside of some context where you
get an
event and the current containing context first, and you want the
contextual object to be inline with your mapping calls, rather than an
additional "switch context" statement. Hibernate solves this by
having
pretty much all the calls be off of a Session object, which you
have to
obtain first:
sess.createQuery(
"select kitten, mother from Cat kitten join kitten.mother
mother")
.list()
.iterator();
So they are doing something similar, i.e. there is a session object
obtained from somewhere when is then explicitly bound to operations.
Im not thrilled about mapper.some_contextual_function(uow).select()
since
it means the source code throughout has to be constructed
differently than
an application that uses the default scoping rules. But any other
way, in
your case means more effort on your part to devise some other way to
associate objects with uow objects...which I dont think is very much
effort, but it is a greater learning curve than just a single proxying
function off the mapper (which would probably do the same thing -
set the
unit of work in the thread-local context first, then do the operation,
then restore the context afterwards. try/finally will be used to
insure
its atomic.).
if "with" is going to be a reserved word then we clearly arent
going to
use that.
Agreed. How about "using(uow)"
In fact, you could probably construct this right now via a custom
MapperExtension that decorates the initial creation of objects, and
putting a special scoping object in objectstore.uow() (with perhaps
those
changes to ScopedRegistry).
I had initially looked into that. The problem is resolving the
unitofwork with the special scoping object. I couldn't think of a way
to tell which object is being operated on (hence which uow to use),
which is necessary when loading lazy attributes.
yes youd have to get into some dictionary-type schemes or something
exotic
which is beyond the level of effort a programmer should need for this
issue.
I think this issue should be solvable with just a little bit of
extra effort:
1. every unitofwork object gets some kind of hashkey (probably its
in-memory ID) and placed into a weakkeydictionary, so that we can
associate anything with a unitofwork in a decoupled fashion that
does not
impact serialization.
See above. I really like the weakref idea because it avoids the extra
overhead of having to maintain a global dictionary of object-to-
unitofwork pointers.
2. mapper/unitofwork system, when it normally says "uow()" to find the
current unit of work, will do something like "uow(object)" that
looks for
this hashkey on the object itself to locate its specific unit of work.
this single dictionary lookup will in most cases probably replace the
current single "thread-local" dictionary lookup, except in the case
when a
newly loaded or created object needs to find its initial unitofwork
(the
uow(object) function will probably just return the thread-local
unitofwork
if no key is present on the object...and maybe bind the key to that
object
at that point).
That's a great idea. The lookup mechanism does not really matter (a
weakref would actually make the lookup simpler--does this object
reference a unit of work? yes? then use it. no? then resolve normally).
I also have some ideas about cleaning up the UnitOfWork/
ScopedRegistry/ObjectStore mechanism. I had some problems figuring
out which one was to be used for which task. Particularly confusing
was the objectstore.uow which is actually a scoped registry. I think
we could reduce it to ObjectStore and UnitOfWork (in most cases
UnitOfWork would be totally encapsulated by ObjectStore)--the patch I
submitted the other day starts to address these issues. This is
getting off topic so I'll start another thread for that when I get
around to it.
Instead of mapper.using(uow) I think it should be mapper.using
(objectstore). It would be redundant to use the default objectstore:
from sqlalchemy import *
# create mapper here ...
mapper.using(objectstore).select(...)
# is the same as
mapper.select(...)
But when using a custom objectstore (i.e. one that resolves to a
specific unitofwork):
from sqlalchemy import *
objstr = ObjectStore(UnitOfWork()) # UnitOfWork param could be optional
# create mapper here ...
mapper.using(objstr).select(...)
The mapper uses the unitofwork associated with the given objectstore.
This makes the api simpler (no need to wonder if we have a UnitOfWork
or a ScopedRegistry... or a RottenEgg :)
3. mappers will get a function such as "mapper.bind(uow)" which
returns an
anonymous proxying object, that establishes the given unit of work
for an
operation and then performs any operations performed on it via the
originating mapper. setting up the unit of work internally within the
bind() call probably would be via the same thread-local
methodology, i.e.
so that the mapper just calls uow(object) as normal after the bind
function has set it up, since its consistent with everything else and
probably more reliable.
I think that will work. Just to be clear, I tend to stay away from
the thread-local pattern when possible (probably due to some
irrational fear that it's not thread safe...or that using globals
causes regret). I suppose this is a good place to use it.
~ Daniel
For the record, here's the original email I sent to mike (I meant to
send it to the list):
On Feb 9, 2006, at 2:39 PM, Michael Bayer wrote:
does this pre-empt the patch that you sent me for "Session-izing" the
objectstore ? I havent gotten a chance to look at that yet.
No. In fact, I think that patch may be necessary for some of this.
question: does having each object maintain a reference to a
unitofwork
object impact its serializabilty (oh, i guess you mentioned that...)?
No. If an object is serialized it should not retain the reference to
the unitofwork. The only case I can think of where this might not
happen is if the (previously referenced before serialization)
unitofwork still existed, in which case the object would still point
to that unitofwork (which might be a problem if unitofwork instances
were being pooled or something, but that sounds obscure). I do not
know how pickle handles weak references, do you? Normally, I would
think the user would "import_instance" into a new unitofwork anyway,
which would overwrite the old reference.
that is a key goal, if we go forth with an idea like this there
would have
to be some indirection going on, such as attaching a hashkey or some
identifying string to the object instead. this string is discarded
upon
import_instance.
As I mentioned above, the reference would be overwritten on
import_instance.
- a given object cannot be associated with more than one unit of work
(is this a problem? are there use cases for associating an object
with more than one uow concurrently?)
not concurrently, but the unit of work begin/commit pattern creates a
second child unit of work.
that shouldn't be a problem
also, it is supported behavior that the
current thread-local unit of work can be changed at any time. for
this
reason, I think associating objects with a unit of work would have
to be
done through some intermediary scope object, along the lines of your
previous patch.
This brings up another thing: it would be possible for an object to
reference a unit of work that did not know about the object. Is that
a problem?
It also might be nice to have this object attribute be
optional if the regular scoped operation is desired (the default),
I think everything would behave exactly like it did before with a few
less quirks (even if the attribute was added to every new object).
and
also might be nice to have the initial association of this object to a
session be handled by some callable or other argument passed to the
initial mapper() construction (or probably MapperExtension is the
place
for this to be). I would very much like to avoid adding arguments to
every single mapper operation.
I agree about the mapper interface. Maybe we could do something like
>>> mapper.with(uow).select(...)
where the object returned by "with(uow)" would use the given
unitofwork for its operations. While this seems like a good
compromise, it does seem like a lot of the mapper logic would be
either duplicated (big-time no-no) or moved into the "with" (the
mapper would be come a proxy that would simply use the normal scope
unit of work. Note: "with" may be a bad name since I believe a new
"with" keyword has been proposed for Python (will that affect this?).
In fact, you could probably construct this right now via a custom
MapperExtension that decorates the initial creation of objects, and
putting a special scoping object in objectstore.uow() (with perhaps
those
changes to ScopedRegistry).
I had initially looked into that. The problem is resolving the
unitofwork with the special scoping object. I couldn't think of a way
to tell which object is being operated on (hence which uow to use),
which is necessary when loading lazy attributes.
Just some things to think about, Im a little time-starved this week
and
havent gotten to play with this stuff yet.
- mike
Thanks for your input. I've started looking into this since I'm going
to need it for the project I'm working on. I'll try to post a patch
soon.
~ Daniel
And the one before that...
Proposal:
- allow a unit of work to be passed into mapper.get/select operations
(fallback on objectstore.uow if not provided)
Implications:
- once an object is associated with a given uow any objects loaded
via relationships on that object will be loaded in the same uow
- each SA-managed object will retain a (weak?) reference to the unit
of work in which it was loaded
- a given object cannot be associated with more than one unit of work
(is this a problem? are there use cases for associating an object
with more than one uow concurrently?)
- an object's reference to a unit of work should be cleared if the
object is serialized (how are weakrefs handled with regard to
serialization?)
- it should be possible to "move" an object from one uow to another
(I believe this is already possible using import_instance. do all
related objects move as well? probably not. maybe the object's
relationships should be "cleared" so they are reloaded in the new uow
if called for)
This proposal, if implemented, would allow the use of multiple
UnitOfWork objects concurrently. Currently, this is possible in a
very inflexible way: the global uow must be switched each time the
local context switches.
Example use case:
A rich client app could load support data in a global uow to be used
in navigation controls, dropdown lists, etc. Then it could use one or
more different uow's for data that is being edited. Each uow could be
committed separately without affecting other uow's.
I'd be willing to work on a patch (maybe a branch would make more
sense for this). Thoughts?
~ Daniel
On Feb 7, 2006, at 6:09 PM, Michael Bayer wrote:
dmiller wrote:
2. Switch the active UOW each time a window gains focus (error prone,
messy)
Actually this is probably your best option. because, its not just
passing
the UOW to all explicit mapper operations, the UOW is used every
time you
set an attribute on an object or append/delete from a list attribute.
If your "sessions" really do correspond with "window focus" then
you would
have to match them up that way. Just like the widgets on the screen
correspond to the window, so the individual data objects they
represent
correspond to the UOW. I gather your object-modify and commit()
operations are occuring within an event loop ? Maybe you could
extend the
event-loop system itself to always insure the correct UOW is
selected ?
- mike
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Sqlalchemy-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users
