On Fri, May 6, 2011 at 10:14 PM, Shane Hathaway <sh...@hathawaymix.org> wrote: > From my experience, most people who want ZODB to be faster want Zope > catalogs in particular to be faster. I don't think prefetching can make > catalogs much faster, though.
I've spent a lot of time lately on making ZCatalog faster. The main trick there is to store data in smarter ways, load fewer objects in the first place and trying to minimize data sets as early as possible, so the cost of intersection() and union() gets lower. There's a lot more you can do about optimizing ZCatalog, but prefetching would indeed not help much. The only cases where you could do prefetching are the ones you don't want to do anyways, like loading an entire BTree or TreeSet, because you need to do a len(tree) or actually iterate over the entire thing. All that said, if you hit large datasets, it gets problematic to do catalog operations on each Zope client. At some point a centralized query approach on the server side or via a web API wins in terms of overall resource efficiency. Hanno _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev