On Feb 27, 2006, at 6:17 AM, Florent Guillaume wrote:
1. I'd like the possibility of having the identity map not cleared
on commit. This turns identity maps into proper cross-requests
caches in the context of web applications. This is what ZODB does,
and it's extremely beneficial to have non-modified objects kept in
caches associated to a pool.
its not cleared on commit. its only cleared if you do
objectstore.clear(). if you are using just commits, or explicit
begin/commits, the identity map stays in place.
2. I'd like to replace the identity map implementation with another
one derived from Zope's ZODB (the persistent.PickleCache, coded in C).
If you really wanted this, Id provide some kind of hook into
objectstore where alternate implementations can take place. But I
dont think this is what youre looking for. a picklecache sounds
like its serializing objects and putting them inside of a file, is it
cross-process accessibility youre looking for ? keep in mind the
identity map is not an object cache, thats not its purpose at
all....its strictly to maintain the identity of objects as unique
within a particular session. if youre looking for caching that would
be something else.
PickleCache has a number of very interesting features, most notably
automatic size management with a target maximum size, the rest may
be garbage collected (LRU eviction).
now I know you are talking about a cache. this is not approrpriate
for an identity map...it must maintain a reference to all objects
that are in memory, else duplicate instances can occur. the current
identity map is a WeakValueDictionary so no additional objects are
stored that are not already present in an application's in-memory
namespace, so it incurs no memory overhead at all right now (other
than weakref containers) (maybe thats why you thought it clears out
after a commit?).
3. In some cases, I'd like the option of assigning a mapper to a
class differently, without any class modification. I would inform
the history/attributemanager system that something changed on the
object by other means than monkey-patched properties.
You can do this now. You can change the primary mapper for a class,
and you can also make as many secondary mappers as you like against a
class. only the primary mapper handles the saving and commit
dependencies for the class, but any mapper can load instances of the
class. the only restriction is that those secondary mappers cannot
define a different set of properties for the class than the current
primary mapper; this is simply because those properties will not be
properly managed by a commit. its possible you want an extra twist
on this, so show me a more specific example of what youre looking for.
4. Another interesting feature of the persistent.Persistent class
is the notion of "ghost". A ghost is a stub for an object that's
lazily loaded. It has a very small memory footprint. On any
attribute access, its manager (_p_jar) is called to fetch the
object's state. Conversely, when the PickleCache is purged by the
LRU mechanism, old entries are not deleted but turned back into
ghosts.
this could be built on top of an SA-managed set of classes and
probably could be provided as an extension option. not totally sure
how this is used ? if I ask the mapper "select all objects where
x=12", it hits the DB anyway, is it just that you dont want to fetch
the full column set ? deferred properties can already be used for
this ? (i.e. you can have all object attributes load as "deferred" in
a single lazy query, http://www.sqlalchemy.org/docs/
adv_datamapping.myt#adv_datamapping_deferred )
5. I'll have to add two-phase commit to SA because this is
essential to Zope's transaction mechanism (which may manage several
databases).
is this compatible with DBAPI ? if you can show me how DBAPI is used
within a two-phase commit we can begin to think how such a feature
might be implemented. if the whole notion occurs externally to
DBAPI, its possible that you would need only provide a custom
implementation of the Session to provide this (its worth looking at
the source code to the Session and UnitOfWork objects in
objectstore.py, they are pretty simple. the supporting objects used
during an actual commit to sort dependencies, i.e.UOWTransaction on
down, much more complicated, dont read too far down in the
module :) ...maybe I should break those out into a separate module.)
6. I'd like to hook somewhere to provide filesystem storage of some
classes' attributes instead of having them mapped to the RDB. I'd
use this for images and attached files, where I'd rather use NFS
storage than transmit a blob through SQL.
you could probably provide a custom MapperProperty object.
MapperProperty is inside of lib/sqlalchemy/mapping/mapper.py, and
five subclasses are present in the neighboring properties.py.
MapperProperty defines all mapping operations for a particular
attribute and could be made to:
1. not append any column to the originating SELECT statement (like
DeferredProperty does)
2. load its value from a file, either upon execution (i.e. heres a
row, give me your attribute, like ColumnProperty does) or lazily (a
loader function is attached to the attribute, like DeferredProperty)
3. we would have to put one additional hook into Mapper, so that
upon saving the object it can be instructed to "ignore" the property
when determining the INSERT/UPDATE parameters and can instead execute
a custom save/delete operation on the property.
this is a valuable feature that you should probably add an
enhancement ticket for.
I'm not familiar enough (yet) with SA's code to know if these
points are complex to do or already possible. Advice welcome.
I can provide more details about ZODB's features if needed.
the two-phase commit sounds the most dicey, only because i dont have
experience with that kind of thing.
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Sqlalchemy-users mailing list
Sqlalchemy-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users
