On Wed, Oct 14, 2009 at 2:48 PM, Shane Hathaway <sh...@hathawaymix.org> wrote:
> Jim Fulton wrote:
>> The biggest problem with ZEO performance on the client side is that
>> reads require round trips and that generally a client thread only
>> knows to request one read at a time [1]_.  I plan to add an API for
>> asynchronous reads.  In rare situations in which an application knows
>> it's going to need more than one object, it can prefetch multiple
>> objects at once.  (One can imagine iteration scenarios in which this
>> would be easy to predict.)  An opportunity that this would provide
>> would be to pre-fetch object revisions for objects that were in the
>> ZODB cache and have just been invalidated.
> When ZODB is unpickling the state of an object, it often has to pull in
> several objects, one at a time.

Several object references.

>  I wonder if it would be valuable to
> prefetch the objects that will be pulled in by the unpickling operation.  We
> could use the referencesf() function to get the list of OIDs.

I don't think objects referenced is a very good predictor of objects
used.  When loading a big container, you wouldn't want to load all of
the contained objects, when you might just be traversing to one.
Similarly, an object may have subobjects that contain data that aren't
needed for a current transaction.  Imagine a movie object that you
load to get at meta data and that has a blob containing the movie. You
wouldn't want to load the blob unless necessary.  OTOH, it might be
very easy for an application that is going to serve the blob data to
make a call that gets the retrieval going before it needs the data.
Something like:

   def some_method():
         self.blob._p_preload() # returns right away
         ... do other things
         ... Now do stuff with self.blob. This will block of the blob
data isn't there yet, but it will at least have a
             head start

This gets a lot more interesting if you know you're going to want more
than one object.  For example,
a method that knows it is going to do something will all subobjects in
a container might do something

   def get_it_all():
         for o in self.data:
         for o in self.data:
              ... do some work woth the actual objects

> Alternatively, I wonder if it would be valuable to store a list of
> referenced OIDs in every object.  We might put that list in another pickle,
> placing it before the "class" and "state" pickles that we currently store.

I don't think getting the referenced objects is all that hard. I just
don't think it's all that useful.


Jim Fulton
For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to