good on yer!
one thing that i sorta failed is making a clean set of tests for this.
if u aren't too far in the usage, save some major future headaches and
do yourself a favour, make such test set. All the things u've tried
has to be (simple) testcases - it will be the _spec_ of how the thing
works / what u want from it. Anyone else on the same path could also
use it then ;-)
i hope that once u do that u'll get an idea of where exactly u have
failed and fix it.
as of expiration, it also did me some trouble. i have not stepped on
that session-gone one but i can see how it happens. The (expected)
life-times of SA-touched things aren't that obvious, and aren't
documented. e.g. Objects living longer than session or in an extreme
case, the mappers, might be troublesome.
what i can suggest is, put debug statements and run with echo=True and
watch what goes on behind and how SA-activities intertwist with
traits' /your ones.
ciao
svilen
On Sunday 22 March 2009 04:00:27 Christiaan Putter wrote:
> Hi svilen,
>
> Thanks for your advice, going through your code helped a lot in
> understanding how SA works.
>
> I've gotten traited classes to behave like classes generated by
> SA's declarative extension. After a lot of stepping through code I
> realised the main problem was that SA removes values from an
> instance __dict__ once they have been expired. Traits smartly (or
> not so smartly) picks this up and on the next access to those
> attributes and returns their default values. Whereas SA would
> normally refresh the values from SQL. This caused some serious
> headaches, and really strange behaviour.
>
> My solution was to the overwrite the class's __getattribute__ with:
>
> def __getattribute__(self, key):
> try:
> dict = super(HasTraitsORM,
> self).__getattribute__('__dict__')
>
> if ('_sa_state' in dict):
> state = dict['_sa_state']
> if key in state.manager.keys():
> return state.get_impl(key).get(state)
>
> except Exception, e:
> print "Exception in SA getattr for %s: %s" % (key, e)
> pass
>
> return super(HasTraitsORM, self).__getattribute__(key)
>
>
> Which tries to first get an attribute from the instance's state
> (which will issue SQL if needed), and if that fails pass the
> request along to its super class where Traits will do it's magic if
> needed.
>
> This seems to work in the tests if been trying. Most of the work
> is done by SA declarative, and I've just added a thin layer on top
> to get it to behave well with Traited classes and map traits to SA
> columns.
>
> One issue I'm still having though is that after commits all
> attributes get expired. If you close the session the instance was
> attached to the data you just commited can't be accessed any more.
> This is also the behaviour of plain declarative classes. This is
> rather unintuitive I believe since once you've comitted an instance
> and closed the session you can't use the data you just set any
> more. In my case this means records being displayed in a gui get
> corrupted.
>
> Though what's strange is that if you access the attributes after
> the commit and before you close the session, and only then close
> the session, the values remain in the instance's __dict__ and the
> class behaves normally again. Is this intended for some reason?
> Is there a workaround? Personally I don't think quantum physics
> should apply to the bits of my programs and thus merely observing
> them should not change their behaviour. From going through the
> source I found a 'dont_expire_missing' attribute on attribute
> implementations, though this does unfortunately not do what I'd
> like.
>
> So what I'm doing now is keeping a session open on the gui's
> thread. Which causes some problems when trying to modify said
> instances from other threads and commiting the new values.
>
> How's this usually done is SA? Right now I've put a lock on the
> gui thread's session and allow other threads to issue a commit on
> that main thread if need be. I can't imagine this is the best way
> to do it...
>
> Hope you're all having a great weekend,
>
> Christian
>
> 2009/2/8 <[email protected]>:
> > afaiknow these traits are like my own static_types, i.e.
> > descriptors holding metadata and applying it to attribute access.
> > i have been combining SA with static_type for more than 2 years
> > now, since SA 3.0.
> >
> > The approach i did in the beginning was to replace the object's
> > __dict__ by something smart that is static_type-aware. When
> > InstrumentionManager framework came into place, i did not find it
> > any different, as SA still uses the __dict__ for data access. The
> > difference is that now my __dict__ replacement is created once
> > and not at every attr.access.
> >
> > i did suggest one patch about replaceing the SA's
> > obj.__dict__.whatever usage with an explicit set of methods (so
> > one knows what access to mime), and that was making SA 1-2%
> > faster, but it wasn't accepted.
> >
> > basicaly now there's a thin layer on top of SA, then SA itself,
> > then a thick layer underneath managing the data (the fake
> > __dict__).
> >
> > declarative+traits... u'll end up where i was. dbcook.sf.net is
> > doing that - since beginning. and it's switchable on/off.
> > It all works well and stable, in project with 250-300 classes,
> > although about 15% slower than without it (-:)
> >
> > The sa2static code:
> > svn co
> > http://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk/dbcook/usa
> >ge/static_type/
> >
> > The static_type itself:
> > svn co
> > https://dbcook.svn.sourceforge.net/svnroot/dbcook/static_type
> >
> > whole dbcook:
> > svn co https://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk
> >
> > i have quite some experience fighting this field, ask if u want.
> >
> > ciao
> > svilen
> > www.svilendobrev.com
> >
> > On Sunday 08 February 2009 01:51:20 cputter wrote:
> >> Hi guys and girls,
> >>
> >> I've recently discovered the joys of using sqlalchemy and would
> >> love to using it together with Traits. A few months back there
> >> was an attempt to integrate sqlalchemy into traits, though it
> >> wasn't really comprehensive in exploiting all of sqlalchemy's
> >> potential.
> >>
> >> So I'm trying to work on that and combine ext.Declarative with
> >> traits. The basic idea is to use the DeclarativeMeta type to
> >> generate Columns from Traits and pass those on for the
> >> Declarative extension to do its magic. This would allow mixing
> >> of sqlalchemy attributes and trait attributes in a single class
> >> so that we could still make use of all the relational setup
> >> sqlalchemy does in any case.
> >>
> >> Reading through several threads and looking at Elixir's SA
> >> integration helped me a bit though I couldn't find any
> >> documentation on how to implement the InstrumentationManager
> >> interface. I'm assuming this would be essential for letting
> >> Traits and SQLAlchemy play well together.
> >>
> >> There's still a lot of work to do, and I'm not really sure what
> >> needs to be done for everything to work properly. Would really
> >> appreciate it if someone could help me out.
> >>
> >>
> >> Here's an example of how it's working at the moment, I'll add
> >> the actual implementation at the end.
> >>
> >> #####################################
> >> class User(HasTraitsORM):
> >> __tablename__ = 'users'
> >>
> >> id = Column('id', Integer, primary_key=True)
> >> name = Str(sqldb=True)
> >>
> >> def _name_changed(self, old, new):
> >> print 'Changed name from %s to %s.' % (old, new)
> >>
> >> def __repr__(self):
> >> return '<User(%s, %s)>' % (self.id, self.name)
> >>
> >> people = ['John', 'Charls','Steve','Smith','Jane']
> >>
> >> for per in people:
> >> obj = User(name=per)
> >> sess = sqlservice.Session()
> >> sess.add(obj)
> >> sess.commit()
> >> sess.close()
> >> print obj
> >>
> >> session = sqlservice.Session()
> >> print '\nQuery all users\n'
> >> for user in session.query(User).order_by(User.name).all():
> >> print user
> >>
> >> session.close()
> >>
> >>
> >> ############################
> >>
> >> Which spits out:
> >> ###############################
> >> Changed name from to John.
> >> <User(users.id, )>
> >> Changed name from to Charls.
> >> <User(users.id, )>
> >> Changed name from to Steve.
> >> <User(users.id, Steve)>
> >> Changed name from to Smith.
> >> <User(users.id, Smith)>
> >> Changed name from to Jane.
> >> <User(users.id, Jane)>
> >>
> >> Query all users
> >>
> >> <User(2, Charls)>
> >> <User(1, John)>
> >>
> >> ##############################
> >>
> >> Which is really strange behaviour. There's obviously something
> >> wrong in my implementation of HasTraitsORM but why the different
> >> results within the same loop??? Why add only two instances?
> >>
> >> Totally baffles me.
> >>
> >> Here's the rest of my code, hope somehow can help me out. It's
> >> very messy, I've been hacking at it like crazy with no success
> >> :-)
> >>
> >> Hope you're all having a great weekend.
> >> -Chris
> >>
> >>
> >> ##############################
> >>
> >> # Standard library imports.
> >> import logging
> >>
> >> # Enthought library imports
> >> from enthought.preferences.api import Preferences
> >> from enthought.traits.api import \
> >> HasTraits, MetaHasTraits, Int, Str, Bool, Float, Any,\
> >> String, Enum, Python, \
> >> on_trait_change, TraitListObject
> >>
> >> # Package imports
> >> import sqlalchemy
> >> from sqlalchemy import Column, Integer
> >> from sqlalchemy.schema import MetaData
> >> from sqlalchemy.orm.interfaces import MapperProperty,
> >> InstrumentationManager
> >> from sqlalchemy.orm.attributes import get_attribute,
> >> set_attribute, is_instrumented
> >> from sqlalchemy.orm.collections import InstrumentedList,
> >> collection_adapter
> >> from sqlalchemy.ext.declarative import _as_declarative
> >>
> >>
> >> # Setup a logger for this module.
> >> logger = logging.getLogger(__name__)
> >>
> >>
> >> TRAIT_MAPPING = {
> >> Int : 'sqlalchemy.Integer',
> >> Str : 'sqlalchemy.Text',
> >> Enum : 'sqlalchemy.Text',
> >> String : 'sqlalchemy.Text',
> >> Float : 'sqlalchemy.Float',
> >> Bool : 'sqlalchemy.Boolean',
> >> }
> >>
> >>
> >> class HasTraitsORMState(InstrumentationManager):
> >> def __init__(self, cls):
> >> self.states = {}
> >>
> >> def instrument_attribute(self, class_, key, attr):
> >> pass
> >>
> >> def install_descriptor(self, class_, key, attr):
> >> pass
> >>
> >> def uninstall_descriptor(self, class_, key, attr):
> >> pass
> >>
> >> def instrument_collection_class(self, class_, key,
> >> collection_class):
> >> return ObjectCollection
> >>
> >> def get_instance_dict(self, class_, instance):
> >> return instance.__dict__
> >>
> >> def initialize_instance_dict(self, class_, instance):
> >> instance.reset_traits()
> >>
> >> def initialize_collection(self, key, state, factory):
> >> data = factory()
> >> return ObjectCollectionAdapter(key, state, data), data
> >>
> >> def install_state(self, class_, instance, state):
> >> self.states[id(instance)] = state
> >>
> >> def state_getter(self, class_):
> >> def find(instance):
> >> return self.states[id(instance)]
> >> return find
> >>
> >>
> >> class ObjectCollectionAdapter(object):
> >> """
> >> An adapter for SQLAlchemy for TraitsListObject which is
> >> the collection
> >> we use for storing instances of classes within
> >> attributes of other
> >> classes.
> >>
> >> TODO: Think of a way to get this to behave like a
> >> normal traited
> >> attribute of objects.
> >> At the moment collection attributes are set through the
> >> mapper by SA
> >> when using relations or backref.
> >>
> >> """
> >> def __init__(self, key, state, collection):
> >> self.key = key
> >> self.state = state
> >> self.collection = collection
> >> setattr(collection, '_sa_adapter', self)
> >>
> >> def unlink(self, data):
> >> setattr(data, '_sa_adapter', None)
> >>
> >> def adapt_like_to_iterable(self, obj):
> >> return iter(obj)
> >>
> >> def append_with_event(self, item, initiator=None):
> >> self.collection.append(item, emit=initiator)
> >>
> >> def append_without_event(self, item):
> >> self.collection.append(item, emit=False)
> >>
> >> def remove_with_event(self, item, initiator=None):
> >> self.collection.remove(item, emit=initiator)
> >>
> >> def remove_without_event(self, item):
> >> self.collection.remove(item, emit=False)
> >>
> >> def clear_with_event(self, initiator=None):
> >> for item in list(self):
> >> self.remove_with_event(item, initiator)
> >>
> >> def clear_without_event(self):
> >> for item in list(self):
> >> self.remove_without_event(item)
> >>
> >> def __iter__(self):
> >> return iter(self.collection)
> >>
> >> def fire_append_event(self, item, initiator=None):
> >> if initiator is not False and item is not None:
> >> self.state.get_impl(self.key).fire_append_event
> >> (self.state, item,
> >>
> >> initiator)
> >>
> >> def fire_remove_event(self, item, initiator=None):
> >> if initiator is not False and item is not None:
> >> self.state.get_impl(self.key).fire_remove_event
> >> (self.state, item,
> >>
> >> initiator)
> >>
> >> def fire_pre_remove_event(self, initiator=None):
> >> self.state.get_impl(self.key).fire_pre_remove_event
> >> (self.state,
> >>
> >> initiator)
> >>
> >> class ObjectCollection(TraitListObject):
> >> __emulates__ = list
> >>
> >> def __init__( self, trait, object, name, value ):
> >> pass
> >>
> >> def __init__(self):
> >> self.members = list()
> >> def append(self, object, emit=None):
> >> self.members.append(object)
> >> #collection_adapter(self).fire_append_event(object,
> >> emit) def remove(self, object, emit=None):
> >> #collection_adapter(self).fire_pre_remove_event(object)
> >> self.members.remove(object)
> >> #collection_adapter(self).fire_remove_event(object,
> >> emit) def __getitem__(self, index):
> >> return self.members[index]
> >> def __iter__(self):
> >> return iter(self.members)
> >> def __len__(self):
> >> return len(self.members)
> >> def __repr__(self):
> >> if len(self.members) == 0:
> >> return '[empty collection]'
> >> s = '['
> >> for item in self.members[0:max(5, len(self.members))]:
> >> s += '%s,' % item
> >> s += '...]'
> >> return s
> >>
> >>
> >> class DeclarativeMetaTraits(MetaHasTraits):
> >> pass
> >>
> >> # def __new__(cls, classname, bases, dict_):
> >> # return MetaHasTraits.__new__(cls, classname, bases,
> >> dict_)
> >>
> >> def __init__(cls, classname, bases, dict_):
> >>
> >> if '_decl_class_registry' in cls.__dict__:
> >> super(DeclarativeMetaTraits, cls).__init__(cls,
> >> classname, bases, dict_)
> >> return
> >>
> >> if '__tablename__' not in dict_:
> >> setattr(cls, '__tablename__', cls.__name__)
> >>
> >> # create sql columns from flagged traits
> >> if '__class_traits__' in cls.__dict__:
> >> traits = cls.__dict__['__class_traits__']
> >> for key, trait in traits.iteritems():
> >> if getattr( trait, 'sqldb' ):
> >> args = getattr( trait, 'col_args' ) or ()
> >> kwargs = getattr( trait, 'col_kwargs' ) or
> >> {} if 'name' not in kwargs:
> >> kwargs['name'] = key
> >> if 'type_' not in kwargs:
> >> kwargs['type_'] =
> >> eval(TRAIT_MAPPING[type (trait.trait_type)])
> >>
> >> if getattr(trait, 'sqlpk'):
> >> kwargs['primary_key'] = True
> >>
> >> c = Column(*args, **kwargs)
> >> dict_[key] = c
> >> setattr(cls, key, c)
> >>
> >> # Pass this along to SA's declarative mapper stuff
> >> _as_declarative(cls, classname, dict_)
> >>
> >> super(DeclarativeMetaTraits, cls).__init__(cls,
> >> classname, bases, dict_)
> >>
> >>
> >> class HasTraitsORM(HasTraits):
> >> __metaclass__ = DeclarativeMetaTraits
> >>
> >> # some SA hacking
> >> __sa_instrumentation_manager__ = HasTraitsORMState
> >>
> >> _decl_class_registry = dict()
> >> metadata = MetaData()
> >>
> >> # Any implicit traits added by SQLAlchemy are transient and
> >> should not be
> >> # copied through .clone_traits(), copy.copy(), or pickling.
> >> _ = Python(transient=True)
> >>
> >>
> >> # For some reason we need to override these, our superclass
> >> screws things up
> >>
> >> def __getattr__(self, key):
> >> if is_instrumented(self, key):
> >> return get_attribute(self, key)
> >> else:
> >> try:
> >> return getattr(self, key)
> >> except:
> >> pass
> >>
> >> def __setattr__(self, key, value):
> >> if is_instrumented(self, key):
> >> set_attribute(self, key, value)
> >> else:
> >> setattr(self, key, value)
> >>
> >>
> >> #################################################
>
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
