Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-29 Thread Chris Withers
Dieter Maurer wrote:

But overall, unless you have special (non DC derived) indexes,

That can well be the case...

Anyway, what are we going to do about this crawling tab?

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-22 Thread Dieter Maurer
Chris Withers wrote at 2003-7-21 08:22 +0100:
  Dieter Maurer wrote:
   #objects suggests that it is the number of objects indexed by
   this index. Who is interested in this information?
  
  Well, it's been useful to be on several occasions when I've seen one index has 
  less objects in than another...
  
   Unless one has inhomogeous objects, almost all objects are indexed
   by every index. Thus, #objects is likely to be similar for
   many indexes.
  
  Hmmm... I use ZCatalogs a _lot_ for searching over inhomogenous sets of objects. 
  For example, that's it's primary role in the CMF...

CMF's catalog is highly standardized, thanks to Dublin Core.

  All standard CMF content types define DC attributes.
  Therefore, each CMF content object is indexed under each DC field
  index.
  The Subjects index may lack some objects (because they do not
  define any Subjects).
  A text index may lack a few objects (because some objects
  may have have both an empty Title and an empty Descritpion).

But overall, unless you have special (non DC derived) indexes,
all #objects should be very similar.


Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-21 Thread Chris Withers
Dieter Maurer wrote:
#objects suggests that it is the number of objects indexed by
this index. Who is interested in this information?
Well, it's been useful to be on several occasions when I've seen one index has 
less objects in than another...

Unless one has inhomogeous objects, almost all objects are indexed
by every index. Thus, #objects is likely to be similar for
many indexes.
Hmmm... I use ZCatalogs a _lot_ for searching over inhomogenous sets of objects. 
For example, that's it's primary role in the CMF...

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-18 Thread Chris Withers
Casey Duncan wrote:
Actually I regard the current behavior as a feature. Using a stopwatch and a 
slide-rule I can estimate to within 100 objects, how many values are indexed 
in a catalog by measuring the time it takes to draw the indexes page.

Please do not remove this most valued feature!
I see now winks so am scared ;-)

Seriously though, it is kinda problematic when you want to get to the ZMI of an 
index and have to guess the URL 'cos hitting the indexes page cripples the server...

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-18 Thread Chris Withers
Dieter Maurer wrote:
I suggest to change the title to # index terms and
revert for the indexes to the old behaviour.
If that'll make it quicker, cool :-)

Others pointed out, that also the size determination for an
index may be expensive. However, it is at most linear in the number
(rather than quadratic) and all recently created indexes now
use BTrees.Length to maintain their size (which gives constant time).
Having a feeling how large an index is is valuable information.
Indeed...

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-18 Thread Chris Withers
Anthony Baxter wrote:
if there's no counter present:
   create one, do a count of the docs, initialise the counter
display counter
Sounds good, what needs to happen to make this happen?

Since this is a bug fix, can it go on the 2.6 branch?

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-18 Thread Andreas Jung


--On Freitag, 18. Juli 2003 13:52 Uhr +0100 Chris Withers 
[EMAIL PROTECTED] wrote:

Dieter Maurer wrote:
I suggest to change the title to # index terms and
revert for the indexes to the old behaviour.
If that'll make it quicker, cool :-)
I am usually not interested in the number of index terms but in the number
of documents that are indexed. This is much more meaningful.
-aj

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-18 Thread Andreas Jung


--On Freitag, 18. Juli 2003 13:53 Uhr +0100 Chris Withers 
[EMAIL PROTECTED] wrote:

Anthony Baxter wrote:
if there's no counter present:
   create one, do a count of the docs, initialise the counter
display counter
Sounds good, what needs to happen to make this happen?

Since this is a bug fix, can it go on the 2.6 branch?

First write the fix and then let's see if it might break something :-)

-aj

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-18 Thread Casey Duncan
On Friday 18 July 2003 01:29 pm, Dieter Maurer wrote:
 Anthony Baxter wrote at 2003-7-18 15:14 +1000:
   
Andreas Jung wrote
I agree but the current implementation sux. Switching to a counter 
based
solution would solve the problem. The only problem I see is to keep the
code fully backward compatible.
   
   if there's no counter present:
  create one, do a count of the docs, initialise the counter
 
 We can use the size of the _unindex.
 
 
 However, is it really worth it?
 
 #objects suggests that it is the number of objects indexed by
 this index. Who is interested in this information?
 
 Unless one has inhomogeous objects, almost all objects are indexed
 by every index. Thus, #objects is likely to be similar for
 many indexes.
 
 A much more interesting information would be the size of the index measured
 by the number of index terms.

I agree. and as a plus, its a minor change to the software...

-Casey

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-18 Thread Dieter Maurer
Anthony Baxter wrote at 2003-7-18 15:14 +1000:
  
   Andreas Jung wrote
   I agree but the current implementation sux. Switching to a counter based
   solution would solve the problem. The only problem I see is to keep the
   code fully backward compatible.
  
  if there's no counter present:
 create one, do a count of the docs, initialise the counter

We can use the size of the _unindex.


However, is it really worth it?

#objects suggests that it is the number of objects indexed by
this index. Who is interested in this information?

Unless one has inhomogeous objects, almost all objects are indexed
by every index. Thus, #objects is likely to be similar for
many indexes.

A much more interesting information would be the size of the index measured
by the number of index terms.


Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-17 Thread Dieter Maurer
Chris Withers wrote at 2003-7-17 11:12 +0100:
  Has anyone noticed that the ZCatalog Indexes tab crawls if you have loads of 
  objects indexed.
  
  My guess is that some types of index take way too long to figure out how many 
  objects are indexed. Anyone know which index types those could be?

The one that provide the correct number of indexed objects
(rather than just the number of indexed terms).

Because the same object can be indexed under several terms,
determining the number of indexed objects requires to
build the union of all the index values. This almost surely
has quadratic (worst case) runtime characteristics.

  BTW, would anyone object if I removed that object count, since it's not often 
  very useful...

You probably should replace it with the size of the index (i.e.
the number of index terms).

Formerly, the index overview displayed this information but
under a buggy # objects title. Someone fixed this for most
indexes, they now show the number of objects but at a high
price.

I suggest to change the title to # index terms and
revert for the indexes to the old behaviour.


Others pointed out, that also the size determination for an
index may be expensive. However, it is at most linear in the number
(rather than quadratic) and all recently created indexes now
use BTrees.Length to maintain their size (which gives constant time).

Having a feeling how large an index is is valuable information.


Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-17 Thread Casey Duncan
Actually I regard the current behavior as a feature. Using a stopwatch and a 
slide-rule I can estimate to within 100 objects, how many values are indexed 
in a catalog by measuring the time it takes to draw the indexes page.

Please do not remove this most valued feature!

-Casey

On Thursday 17 July 2003 04:35 pm, Dieter Maurer wrote:
 Chris Withers wrote at 2003-7-17 11:12 +0100:
   Has anyone noticed that the ZCatalog Indexes tab crawls if you have loads 
of 
   objects indexed.
   
   My guess is that some types of index take way too long to figure out how 
many 
   objects are indexed. Anyone know which index types those could be?
 
 The one that provide the correct number of indexed objects
 (rather than just the number of indexed terms).
 
 Because the same object can be indexed under several terms,
 determining the number of indexed objects requires to
 build the union of all the index values. This almost surely
 has quadratic (worst case) runtime characteristics.
 
   BTW, would anyone object if I removed that object count, since it's not 
often 
   very useful...
 
 You probably should replace it with the size of the index (i.e.
 the number of index terms).
 
 Formerly, the index overview displayed this information but
 under a buggy # objects title. Someone fixed this for most
 indexes, they now show the number of objects but at a high
 price.
 
 I suggest to change the title to # index terms and
 revert for the indexes to the old behaviour.
 
 
 Others pointed out, that also the size determination for an
 index may be expensive. However, it is at most linear in the number
 (rather than quadratic) and all recently created indexes now
 use BTrees.Length to maintain their size (which gives constant time).
 
 Having a feeling how large an index is is valuable information.
 
 
 Dieter
 
 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://mail.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists - 
  http://mail.zope.org/mailman/listinfo/zope-announce
  http://mail.zope.org/mailman/listinfo/zope )
 


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-17 Thread Andreas Jung


--On Donnerstag, 17. Juli 2003 18:22 Uhr -0400 Casey Duncan 
[EMAIL PROTECTED] wrote:

Actually I regard the current behavior as a feature. Using a stopwatch
and a  slide-rule I can estimate to within 100 objects, how many values
are indexed  in a catalog by measuring the time it takes to draw the
indexes page.
Please do not remove this most valued feature!
I agree but the current implementation sux. Switching to a counter based
solution would solve the problem. The only problem I see is to keep the
code fully backward compatible.
-aj

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog Indexes tab crawls...

2003-07-17 Thread Anthony Baxter

 Andreas Jung wrote
 I agree but the current implementation sux. Switching to a counter based
 solution would solve the problem. The only problem I see is to keep the
 code fully backward compatible.

if there's no counter present:
   create one, do a count of the docs, initialise the counter

display counter



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )