Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-20 Thread Stefan H. Holek
One thing I seem to remember is that changes made in __setstate__ do not mark 
the object as dirty. This is because after being loaded an object is up-to-date 
by definition. 

I suggest going with zope.generations.

Stefan


On 19.06.2012, at 23:42, Malthe Borch wrote:

 On 19 June 2012 23:38, Claudiu Saftoiu csaft...@gmail.com wrote:
 Is there any hook to call *after* the instance attributes get set/loaded
 from the database?
 
 True. I guess you'll have to go with Marius' suggestion – ``__setstate__``.

-- 
Stefan H. Holek
ste...@epy.co.at

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Malthe Borch
On 19 June 2012 19:54, Claudiu Saftoiu csaft...@gmail.com wrote:
 That is, a function called whenever the object is loaded, that does all the
 necessary backwards-compatibility
 work right there. It separates the backwards-compat code cleanly, and also
 only updates the objects
 as-needed... though still a minor performance hit as it does the check each
 time the object is loaded.

 Is there a way to do that last option? What's the best practice for this
 sort of thing, in general?

You'll need to override ``__new__``. That's your hook. It's called
when the database instantiates the object. Note that this is always
true for Python. The ``__new__`` method is always called before an
object is instantiated.

\malthe
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Marius Gedminas
On Tue, Jun 19, 2012 at 01:54:21PM -0400, Claudiu Saftoiu wrote:
 Ideally I could do something like this:
 
 class Foo(Persistent):
 def __init__(self, a):
 self.a = a
 self.b_cache = PersistentDict()
 
 def __just_loaded__(self):
 if not hasattr(self, 'b_cache'): self.b_cache = PersistentDict()
 
 def calc_it(self, b):
 
 if b in self.b_cache: return self.b_cache[b]
 res = expensive_function(a, b)
 self.b_cache[b] = res
 return res
 
 That is, a function called whenever the object is loaded, that does
 all the necessary backwards-compatibility work right there. It
 separates the backwards-compat code cleanly, and also only updates the
 objects as-needed... though still a minor performance hit as it does
 the check each time the object is loaded.
 
 Is there a way to do that last option?

Yes, you can override __setstate__:

def __setstate__(self, state):
super(Foo).__setstate__(self, state)
if not hasattr(self, 'b_cache'):
self.b_cache = PersistentDict()

This assumes you haven't been defining other pickling protocol
functions like __reduce__, and so cPickle -- which is the foundation
of ZODB -- uses the usual __getstate__/__setstate__ pair.

 What's the best practice for this sort of thing, in general?

It depends. :-)

If the new attribute is immutable, you can set it as a class attribute.

If it's easy to locate all Foo objects in your DB, you may want to write
a generation script (see http://pypi.python.org/pypi/zope.generations)
and do the migration once, instead of paying the cost of checking for
missing attributes on every single object load.

It's also best to avoid write-on-read semantics, because those tend to
cause database growth and increase the chances of getting ConflictErrors.

Marius Gedminas
-- 
An expert is a person who has made all the mistakes that can be made in a very
narrow field.
-- Niels Bohr


signature.asc
Description: Digital signature
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Vincent Pelletier
Le mardi 19 juin 2012 19:54:21, Claudiu Saftoiu a écrit :
 def calc_it(self, b):
 if not hasattr(self, 'b_cache'): self.b_cache =
 PersistentDict()

If you were to use this, you would probably prefer to do
try:
self.b_cache
except AttributeError:
self.b_cache = PersistentDict()
(assigning the result to a vairable, etc)
as this is faster when b_cache is actually here (which is much more likely 
than not). Also, it avoids hiding too many exceptions, as with hasattr. In the 
times of ReadConflictError it was quite important (at least to stop processing 
earlier on a doomed transaction). I'm not sure how important this is nowadays 
- I just kept the habit of avoiding hasattr on persistent objects.

 The former is OK, but if I have multiple functions that want to use the new
 functionality I'll have to have the code all over, and it won't be obviously
 separated.

Maybe a @property taking care of creating the actual PersistentDict if not 
present, then ?

-- 
Vincent Pelletier
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Vincent Pelletier
Le mardi 19 juin 2012 20:50:55, Marius Gedminas a écrit :
 Yes, you can override __setstate__:
[...]
 It's also best to avoid write-on-read semantics, because those tend to
 cause database growth and increase the chances of getting ConflictErrors.

This is also a problem with __setstate__, and I've been bitten by it when 
moving from Zope 2.8 to 2.12 (maybe 2.11 too ?) as 2.8 PythonScript objects 
were modified on load on the new version (adding a trailing \n IIRC), which 
broke History ZMI tab: one cannot alter a non-current object revision.

-- 
Vincent Pelletier
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Claudiu Saftoiu
On Tue, Jun 19, 2012 at 2:29 PM, Malthe Borch mbo...@gmail.com wrote:

 On 19 June 2012 19:54, Claudiu Saftoiu csaft...@gmail.com wrote:
  That is, a function called whenever the object is loaded, that does all
 the
  necessary backwards-compatibility
  work right there. It separates the backwards-compat code cleanly, and
 also
  only updates the objects
  as-needed... though still a minor performance hit as it does the check
 each
  time the object is loaded.
 
  Is there a way to do that last option? What's the best practice for this
  sort of thing, in general?

 You'll need to override ``__new__``. That's your hook. It's called
 when the database instantiates the object. Note that this is always
 true for Python. The ``__new__`` method is always called before an
 object is instantiated.


Thanks, I wasn't aware. Seems to work - much appreciated!

- Claudiu
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Marius Gedminas
On Tue, Jun 19, 2012 at 10:15:03PM +0200, Vincent Pelletier wrote:
 Le mardi 19 juin 2012 19:54:21, Claudiu Saftoiu a écrit :
  def calc_it(self, b):
  if not hasattr(self, 'b_cache'): self.b_cache =
  PersistentDict()
 
 If you were to use this, you would probably prefer to do
 try:
 self.b_cache
 except AttributeError:
 self.b_cache = PersistentDict()
 (assigning the result to a vairable, etc)
 as this is faster when b_cache is actually here (which is much more likely 
 than not). Also, it avoids hiding too many exceptions, as with hasattr. In 
 the 
 times of ReadConflictError it was quite important (at least to stop 
 processing 
 earlier on a doomed transaction). I'm not sure how important this is nowadays 
 - I just kept the habit of avoiding hasattr on persistent objects.
 
  The former is OK, but if I have multiple functions that want to use the new
  functionality I'll have to have the code all over, and it won't be obviously
  separated.
 
 Maybe a @property taking care of creating the actual PersistentDict if not 
 present, then ?

zope.cachedescriptors.property.Lazy would work even better:


@Lazy
def b_cache(self):
return PersistentDict()


Marius Gedminas
-- 
If you have to run heating in winter, you don't own enough computers.


signature.asc
Description: Digital signature
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Claudiu Saftoiu

 You'll need to override ``__new__``. That's your hook. It's called
 when the database instantiates the object. Note that this is always
 true for Python. The ``__new__`` method is always called before an
 object is instantiated.


Actually, this doesn't seem to be what I want. ``__new__`` is called
*before* any attributes are set on the instance... so it's too early
to tell whether the instance is missing the attribute, as it certainly
will (since it's missing any attributes).

Is there any hook to call *after* the instance attributes get set/loaded
from the database?

Here is my code that didn't work, in case I'm just doing something silly:

class Line(Persistent):
def __new__(cls, *args, **kwargs):
inst = super(Line, cls).__new__(cls, *args, **kwargs)

try:
inst.id #every instance in the DB already has an 'id'
except AttributeError:
return inst
print 'we are here...' #this is never printed

try:
inst.expired  #the thing i actually want to guarantee
except AttributeError:
inst.expired = False

return inst

Thanks,
- Claudiu
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Is any function called when an object is loaded from the database?

2012-06-19 Thread Malthe Borch
On 19 June 2012 23:38, Claudiu Saftoiu csaft...@gmail.com wrote:
 Is there any hook to call *after* the instance attributes get set/loaded
 from the database?

True. I guess you'll have to go with Marius' suggestion – ``__setstate__``.

\malthe
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev