Re: [Zope-dev] ZCatalog scalability

2001-02-18 Thread Michael R. Bernstein
Erik Enge wrote: [Michael Bernstein] | I need to know how far the ZCatalog will scale using this indexing | and search strategy. Does anyone have anectodal or benchmark data to | suggest if (and when) I will hit a 'wall' regarding the number of | objects being indexed and searched?

Re: [Zope-dev] ZCatalog scalability

2001-02-18 Thread Erik Enge
[Michael R. Bernstein] | Erik Enge wrote: | | I'll let you know how it goes. (And please, do poke at me if it takes | too long.) | | Ok, I'm poking :-). Thanks. Keep doing it till you get what you need, I truly don't mind. :-) | How did it go? Thanks to the speed of delivery at Royal

Re: [Zope-dev] ZCatalog scalability

2001-02-18 Thread Michael R. Bernstein
Erik Enge wrote: [Michael R. Bernstein] | Erik Enge wrote: | | I'll let you know how it goes. (And please, do poke at me if it takes | too long.) | | Ok, I'm poking :-). Thanks. Keep doing it till you get what you need, I truly don't mind. :-) | How did it go? Thanks to

Re: [Zope-dev] ZCatalog scalability

2001-02-18 Thread Erik Enge
[Michael R. Bernstein] | What I'm looking for is any indication that object creation time | and/or indexing time goes up with the number of objects already in | the ZODB. Well, one thing I've already learned - which you all probably know - is that you do _not_ want to put index_object() in your

Re: [Zope-dev] ZCatalog scalability

2001-02-18 Thread Michael R. Bernstein
Erik Enge wrote: [Michael R. Bernstein] | What I'm looking for is any indication that object creation time | and/or indexing time goes up with the number of objects already in | the ZODB. Well, one thing I've already learned - which you all probably know - is that you do _not_ want to

Re: [Zope-dev] ZCatalog scalability

2001-01-23 Thread Erik Enge
[Michael Bernstein] | We seem to have disposed of the wildcard issue [snipped out | below], and I'm looking forward to Eric's results, but does | anyone else have any information about whether there is a | practical upper limit on how many objects can be indexed and | searched in a ZCatalog? I

Re: [Zope-dev] ZCatalog scalability

2001-01-23 Thread Erik Enge
[Chris Withers] | ...and is that specifically for BTree folders, or Zope BTree's in general? I don't believe that B-Tree folders have those kinds of limitations by general design. I'm more conserned that somewhere along the lines, doing operations on a huge BTree Folder (Yes, in Zope) will be

Re: [Zope-dev] ZCatalog scalability

2001-01-23 Thread Michael Bernstein
Erik Enge wrote: [Chris Withers] | ...and is that specifically for BTree folders, or Zope BTree's in general? I don't believe that B-Tree folders have those kinds of limitations by general design. I'm more conserned that somewhere along the lines, doing operations on a huge BTree

Re: [Zope-dev] ZCatalog scalability

2001-01-23 Thread Erik Enge
[Michael Bernstein] | Erik Enge wrote: | | I don't believe that B-Tree folders have those kinds of limitations by | general design. I'm more conserned that somewhere along the lines, | doing operations on a huge BTree Folder (Yes, in Zope) will be slow. | | What sort of 'operations' do

Re: [Zope-dev] ZCatalog scalability

2001-01-22 Thread Dieter Maurer
Steve Alexander writes: Michael Bernstein wrote: Also, is there a way to disable wildcards in full text searches? Do not allow direct queries to search the catalog. Instead, make searches go through an external method (or a PythonScript with Proxy permissions) that uses

Re: [Zope-dev] ZCatalog scalability

2001-01-22 Thread Michael Bernstein
We seem to have disposed of the wildcard issue [snipped out below], and I'm looking forward to Eric's results, but does anyone else have any information about whether there is a practical upper limit on how many objects can be indexed and searched in a ZCatalog? Michael Bernstein wrote: After

[Zope-dev] ZCatalog scalability

2001-01-21 Thread Michael Bernstein
After comsidering the feedback I got from the previous 'Massive scalability' thread, I decided to split my queries into two areas: Rack scalability and ZCatalog scalability. This email deals with the latter. Partial match (wildcard) searches have already been identified as a resource hog,

Re: [Zope-dev] ZCatalog scalability

2001-01-21 Thread Steve Alexander
Michael Bernstein wrote: Also, is there a way to disable wildcards in full text searches? Do not allow direct queries to search the catalog. Instead, make searches go through an external method (or a PythonScript with Proxy permissions) that uses string.replace to change '*' and '?' to ''.

Re: [Zope-dev] ZCatalog scalability

2001-01-21 Thread Michael Bernstein
Steve Alexander wrote: Michael Bernstein wrote: Also, is there a way to disable wildcards in full text searches? Do not allow direct queries to search the catalog. Instead, make searches go through an external method (or a PythonScript with Proxy permissions) that uses

Re: [Zope-dev] ZCatalog scalability

2001-01-21 Thread Michael Bernstein
Erik Enge wrote: [Michael Bernstein] | I need to know how far the ZCatalog will scale using this indexing | and search strategy. Does anyone have anectodal or benchmark data to | suggest if (and when) I will hit a 'wall' regarding the number of | objects being indexed and searched?

Re: [Zope-dev] ZCatalog scalability

2001-01-21 Thread Erik Enge
[Michael Bernstein] | I need to know how far the ZCatalog will scale using this indexing | and search strategy. Does anyone have anectodal or benchmark data to | suggest if (and when) I will hit a 'wall' regarding the number of | objects being indexed and searched? I'm going to try to stuff 27

Re: [Zope-dev] ZCatalog scalability

2001-01-21 Thread Chris Withers
Michael Bernstein wrote: Also, is there a way to disable wildcards in full text searches? Do not allow direct queries to search the catalog. Instead, make searches go through an external method (or a PythonScript with Proxy permissions) that uses string.replace to change '*' and '?' to

Re: [Zope-dev] ZCatalog scalability

2001-01-21 Thread Steve Alexander
Chris Withers wrote: Wouldn't using a normal vocabulary as opposed to a globbing vocabulary prevent this as well? That would stop globbing searches for everyone. While I might want to stop users of a site making wildcard searches, I still want to keep that facility for myself :-) --

[Zope-dev] ZCatalog Scalability

2001-01-17 Thread Chris Withers
John Eikenberry wrote: and retrieval. But ZCatalog did not. It was basically useless for partial matching searches (taking many minutes for searches that retrieved more than 100 matches). I was also concerned about the indexing overhead. It doesn't scale well when changing/adding many

Re: [Zope-dev] ZCatalog scalability...

2000-12-06 Thread Chris Withers
John Eikenberry wrote: the potential of up to 50,000 entries. Using a ZCatalog for listings This may cause you real problems, especially if there's a 'bulk data load' at any point. Cheers, Chris PS: How's the catalog revamp coming along? Any published ZSearch interface yet?