[Zope-dev] Re: ZCatalog Scalability

2001-01-17 Thread Christopher Petrilli

Unfortunately, it won't change in b1, it might change before the final
release if I can find a better solution.  The problem is in the Vocabulary,
not in the Catalog itself.  One of the things I'm focusing on is improving
the algorithms that are used for doing globbing.

Chris
- Original Message -
From: "Chris Withers" [EMAIL PROTECTED]
To: "John Eikenberry" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; "Christopher Petrilli" [EMAIL PROTECTED]
Sent: Wednesday, January 17, 2001 5:10 AM
Subject: ZCatalog Scalability


 John Eikenberry wrote:
 
  and retrieval. But ZCatalog did not. It was basically useless for
partial
  matching searches (taking many minutes for searches that retrieved more
  than 100 matches). I was also concerned about the indexing overhead. It
  doesn't scale well when changing/adding many things at a time (we might
  have bulk adds/changes).

 I wonder if this will change with Zope 2.3?

 cheers,

 Chris



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] Re: ZCatalog Scalability

2001-01-17 Thread Michael Bernstein

Christopher Petrilli wrote:
 
 Unfortunately, it won't change in b1, it might change before the final
 release if I can find a better solution.  The problem is in the Vocabulary,
 not in the Catalog itself.  One of the things I'm focusing on is improving
 the algorithms that are used for doing globbing.

Which problem is the algorithm improvement meant to address,
bulk adds or partial search performance?

Thanks,

Michael Bernstein.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] Re: ZCatalog Scalability

2001-01-17 Thread Chris Withers

Michael Bernstein wrote:
 
 Christopher Petrilli wrote:
 
  Unfortunately, it won't change in b1, it might change before the final
  release if I can find a better solution.  The problem is in the Vocabulary,
  not in the Catalog itself.  One of the things I'm focusing on is improving
  the algorithms that are used for doing globbing.
 
 Which problem is the algorithm improvement meant to address,
 bulk adds or partial search performance?

I'm guessing the partial search thing, but the bulk add thing really needs to be
sorted out too.

NIP'd love to move the archives at http://zope.nipltd.com/public/lists.html to a
wholely Zope solution, but ZCatalog consistently exploded when we tried to do
this.
For now, we're working on a Zope front end into MySQL (MySQL's new full text
searching stuff is pretty damn quick, maybe you could nick their algorithms? ;-)

So, are both of these problems being addressed? If so, when are we likely to see
them?

Thanks for the work so far :-)

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] Re: ZCatalog Scalability

2001-01-17 Thread Andy McKay

 Michael Bernstein wrote:
 
  Christopher Petrilli wrote:
  
   Unfortunately, it won't change in b1, it might change before the final
   release if I can find a better solution.  The problem is in the
Vocabulary,
   not in the Catalog itself.  One of the things I'm focusing on is
improving
   the algorithms that are used for doing globbing.
 
  Which problem is the algorithm improvement meant to address,
  bulk adds or partial search performance?

 I'm guessing the partial search thing, but the bulk add thing really needs
to be
 sorted out too.

 NIP'd love to move the archives at
http://zope.nipltd.com/public/lists.html to a
 wholely Zope solution, but ZCatalog consistently exploded when we tried to
do
 this.
 For now, we're working on a Zope front end into MySQL (MySQL's new full
text
 searching stuff is pretty damn quick, maybe you could nick their
algorithms? ;-)

For a similar project (http://mailarchive.activestate.com) we too use a
MSSQL back end, for various reasons similar to Chris. I'm not worried about
it, once upon a time I tried to stick everying in the ZODB. To me one of
Zope's great strengths is it's portability and connectivity to other
databases and storage systems.

--
  Andy McKay.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )




Re: [Zope-dev] Re: ZCatalog Scalability

2001-01-17 Thread Chris Withers

 For a similar project (http://mailarchive.activestate.com) we too use a
 MSSQL back end, for various reasons similar to Chris. I'm not worried
about
 it, once upon a time I tried to stick everying in the ZODB. To me one of
 Zope's great strengths is it's portability and connectivity to other
 databases and storage systems.

That's very true :-)

...however, ODB's seem good for tree-like data, probably better than RDB's,
which are better suited to tables. While the mail messages themselves are
tabular, the threading, which for me is _the_ most important part, is very
tree-like and so, again for me, would sit much more nicely in an ODB.

Now, the Catalog, as well as providing a very easy to use interface, is
great for indexing tree-like data, but sadly its performance ain't that
great for big numbers and/or complex searches right now.

If it was lightning fast, better documented and had much more powerful
searching capabilities, it would be Zope's absolute killer feature.

Well, I can keep dreaming, I suppose :-)

Anyway, all Chris P's work is bringing that closer, so thanks for the
changes so far, I'm waiting with baited breath for those to come ;-)

cheers,

Chris


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )