On Wed, Mar 9, 2011 at 7:46 AM, Michael Bayer <[email protected]> wrote: > > On Mar 9, 2011, at 4:57 AM, Yang Zhang wrote: > >> I have a model with a field: >> >> class Obj(Base): >> ... >> meta = Column(PickleType) >> >> For some reason, when I do >> >> obj = session.query(Obj).get(id) >> obj2 = session.query(Obj2).... >> >> I get a dirty UPDATE being attempted on the first object (noticed this >> due to read-only access to a Postgresql DB): >> >> InternalError: (InternalError) transaction is read-only >> 'UPDATE intel.opportunity_runs SET meta=%(meta)s WHERE >> intel.opportunity_runs.id = %(intel_opportunity_runs_id)s' {'meta': >> <psycopg2._psycopg.Binary object at 0x2f8fe18>, >> 'intel_opportunity_runs_id': '21da68c8-499d-11e0-a5c9-12313f00dc12'} >> >> I'm using 0.6.4. I haven't been able to repro this issue outside the >> app. Just wondering if there's any high-level tips on how to best go >> about debugging this. Thanks in advance. > > > It would appear that the comparison of the pickled data is coming up as > "modified", which is a common occurrence. To debug this, you'd do a pdb or > print statement after "obj" is loaded, assuming that's the one being marked > dirty, then do an "obj in session.dirty" which would probably return True, > then if still unsure that it's "meta" do an "attributes.get_history(obj, > "meta")" and inspect the history object. Within get_history() the comparison > is actually performed. By default the "==" operator is used, as defined by > the "compare_values()" method of the column's type object. The History > object returns a tuple with ([new value], (), [previous value]), the > "previous value" being a copy of the original. If "new_value != > previous_value", that is the issue. > > Going forward, just generally, if you see UPDATEs coming out for no reason > and you see a PickleType, its already obvious that this is the issue. > "mutable" types are headed towards deprecation in favor of the "mutable > scalars" extension in 0.7. The strategies, in order of preference are: > > 1. set mutable=False on the PickleType. Your CPU will thank you as > "mutable=True" is extremely inefficient. > > 2. If "in place mutation detection" is definitely needed, if possible (I > know, its probably not, just go to #3), upgrade to 0.7 and use the new > mutable scalars extension to detect in-place change events. > > 3. add a "comparator" function to your PickleType, or implement an __eq__() > method on the object that you're pickling. > > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/sqlalchemy?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > >
Ah, after some more experimentation, I think I understand what you're saying. I can repro this problem outside the app by trying to pickle something whose instances don't implement __eq__ (e.g., object()). For posterity, my guess from reading the docs on MutableType (correct me if I'm wrong) is that the ORM does loads(meta) twice, once for the historic instance and once for the instance the app uses/mutates (not really sure where in the code these take place), and it compares the two whenever does a check for anything dirty to flush (which may be pretty frequently, e.g. on every read). And it sounds like in 0.7 the mutable scalars approach is to define/use your own data structures that know when they've been mutated and inform Mutable.changed(). Thanks! -- Yang Zhang http://yz.mit.edu/ -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
