On Sat, Mar 9, 2013 at 9:02 AM, Jim Fulton <j...@zope.com> wrote: ... > I think a simple method on a storage that gives a hint that a set of > object ids will be loaded is enough. A network storage could then > issue a pipelined request for those oids. The application can then > proceed as usual. I think I've proposed such an API before, but am > too lazy to look it up. Something like: > > load_hint(*oids) > > I'd like to see this functionality, but I don't have time to do it soon. > > I must say that I think this API is more likely to be abused > than used effectively. Prefetching catalog indexes is a sort of > anti-pattern than only makes sense for small catalogs. It > would likely make more sense to have a dedicated catalog > server that returned oids and possibly object records in > response to queries (or whimper, use solr ).
I forgot to mention an even simpler way to reduce the number of round-trips for cataloging data structures is to increase the bucket and internal node sizes. You can do this now by patching the BTree header files. We've done this in the past to reduce the likelihood of database conflicts when buckets split. I suspect the default sizes are too low. I'd really like BTrees to be subclassible and for sizes to be read from instance data, falling back to class settings. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm _______________________________________________ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev