Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-29 Thread Dieter Maurer
Lennart Regebro wrote at 2007-3-28 18:25 +0200: On 3/27/07, Dieter Maurer [EMAIL PROTECTED] wrote: However, this approach is only efficient when the sort index size is small compared to the result size. Sure. But with incremental searching, the result size is always one, right? ;-) No. You

[ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-28 Thread Adam Groszer
Hello, Somehow relevant to the subject I just found an article on Wickert's site: http://www.wiggy.net/ , Using a seperate Data.fs for the catalog -- Best regards, Groszer Adam -- Quote of the day: Keep your fears for yourself, but share your courage with others. - Robert Louis Stevenson

[ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-28 Thread Chris Withers
Adam Groszer wrote: Somehow relevant to the subject I just found an article on Wickert's site: http://www.wiggy.net/ , Using a seperate Data.fs for the catalog The win here is actually partitioning the object cache... Similar wins could be achieved without making backup/pack/etc more

Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Dieter Maurer
Jim Fulton wrote at 2007-3-26 15:55 -0400: ... On Mar 26, 2007, at 3:28 PM, Dieter Maurer wrote: Jim Fulton wrote at 2007-3-25 09:53 -0400: On Mar 25, 2007, at 3:01 AM, Adam Groszer wrote: MF I think one of the main limitations of the current catalog (and MF hurry.query) is efficient

Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-26 Thread Jim Fulton
On Mar 26, 2007, at 3:28 PM, Dieter Maurer wrote: Jim Fulton wrote at 2007-3-25 09:53 -0400: On Mar 25, 2007, at 3:01 AM, Adam Groszer wrote: MF I think one of the main limitations of the current catalog (and MF hurry.query) is efficient support for sorting and batching the query MF

Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-25 Thread Jim Fulton
On Mar 25, 2007, at 3:01 AM, Adam Groszer wrote: MF I think one of the main limitations of the current catalog (and MF hurry.query) is efficient support for sorting and batching the query MF results. The Zope 3 catalog returns all matching results, which can then MF be sorted and batched.

Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-25 Thread Jim Fulton
On Mar 25, 2007, at 11:08 AM, Lennart Regebro wrote: ... 2. Use an N-best algorithm. If N is the size of the batch and M is the corpus size, then this is O(M*ln(N)) rather than O(M*ln(M)) which is a significant improvement if N M, but still quite expensive. I don't think relational databases

[ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-25 Thread Martijn Faassen
Hey Jim, Jim Fulton wrote: On Mar 25, 2007, at 12:33 PM, Martijn Faassen wrote: [snip] I have the strong suspicion that modern relational databases are currently better able to scale at queries using LIMIT and ORDER BY than the Zope 3 catalog. I had a similar suspicion. I assigned the