Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Lennart Regebro
On Sun, Nov 9, 2008 at 19:58, Roché Compaan [EMAIL PROTECTED] wrote: Since I'm in full agreement that we need to fix indexes that are problematic, I started doing some benchmarks on the large data set that gave us so many headaches. It is probably not surprising that the more complex indexes

Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Lennart Regebro
On Sun, Nov 9, 2008 at 22:29, Matt Hamilton [EMAIL PROTECTED] wrote: Lennart Regebro regebro at gmail.com writes: I would be interested in seeing a bunch of Gurus sit down at some sprint and trying to come up with a catalog engine that is incremental and uses query plans. There is no reason

Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Martin Aspeli
Hi Tres, Index Name |Type |Avg Time |Calls/second == object_implements|KeywordIndex |0.2172234| 4.6 This is clearly not the same issue as the other KeywordIndexes: in fact, I am astonished

Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Roché Compaan
On Mon, 2008-11-10 at 18:38 +0200, Hedley Roos wrote: Kinda pointless for me to continue since this is turning into a Plone-specific discussion on zope-dev. But at least the whole exercise has forced us to look in detail into how all these indexes affect performance with a zodb with many many

Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Roché Compaan
On Mon, 2008-11-10 at 11:08 -0500, Tres Seaver wrote: Index Name |Type |Avg Time |Calls/second == object_implements|KeywordIndex |0.2172234| 4.6 This is clearly not the same issue as the

Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Laurence Rowe
Lennart Regebro wrote: On Sun, Nov 9, 2008 at 19:58, Roché Compaan [EMAIL PROTECTED] wrote: Since I'm in full agreement that we need to fix indexes that are problematic, I started doing some benchmarks on the large data set that gave us so many headaches. It is probably not surprising that the

Re: [Zope-dev] catalog performance: query plan

2008-11-09 Thread Matt Hamilton
Lennart Regebro regebro at gmail.com writes: I would be interested in seeing a bunch of Gurus sit down at some sprint and trying to come up with a catalog engine that is incremental and uses query plans. There is no reason that would not be stupidly fast. :) We can then make a new catalog

Re: [Zope-dev] catalog performance: query plan

2008-11-09 Thread Roché Compaan
On Mon, 2008-10-27 at 11:32 -0500, Alan Runyan wrote: I agree with Tres. A lot more can be done with Indexes and Catalog without caching. The most exiciting development in Catalog optimizations comes out Jarn. Helge Tesdal (iirc) did a buncha work at a RDBMS company when he was in

Re: [Zope-dev] catalog performance: query plan

2008-11-06 Thread Lennart Regebro
On Mon, Oct 27, 2008 at 17:32, Alan Runyan [EMAIL PROTECTED] wrote: I agree with Tres. A lot more can be done with Indexes and Catalog without caching. The most exiciting development in Catalog optimizations comes out Jarn. Helge Tesdal (iirc) did a buncha work at a RDBMS company when he

[Zope-dev] catalog performance: query plan

2008-10-27 Thread Alan Runyan
I agree with Tres. A lot more can be done with Indexes and Catalog without caching. The most exiciting development in Catalog optimizations comes out Jarn. Helge Tesdal (iirc) did a buncha work at a RDBMS company when he was in college. He has a protoype of a query plan for ZCatalog.