Re: [Zope-dev] ZCatalog caching with memcached

2008-10-26 Thread Fabio Rizzo Matos
Hi Roché,

I can see your funkload profile?

On Sun, Oct 26, 2008 at 3:43 PM, Roché Compaan
[EMAIL PROTECTED]wrote:

 On Sat, 2008-10-25 at 09:20 +0200, Hedley Roos wrote:
   Have you measures the time needs for some standard ZCatalog queries
   used with a Plone site with the communication overhead with memcached?
   Generally spoken: I think the ZCatalog is in general fast. Queries
 using a
   fulltext index are known to be more expensive or if you have to deal
 with
   large resultsets or complex queries.
  
 
  No I haven't. Roche Compaan has done extensive benchmarking using
  funkload testing plain catalog vs module level cache vs memcached, but
  the tests are more about page serving than catalog query time. I'll
  ask him to comment more on that.

 I actually did some profiling as well and catalog searches were just too
 damn slow. The average execution time for searchResults was 100
 milliseconds and this is why I told Hedley we should do some caching at
 query level in the first place. I experimented with this idea a couple
 of years back but wasn't successful due to inexperience. I was trying to
 cache brains which obviously leads to persistency bugs. This time around
 it was obvious to me that we should cache the IISet result sets.

 I suspect specific indexes are just performing suboptimally and needs to
 be improved. ExtendPathIndex in Plone seems to be one of them.

 The effect on performance is really awesome, now we just need to fine
 tune the implementation.

 --
 Roché Compaan
 Upfront Systems   http://www.upfrontsystems.co.za

 ___
 Zope-Dev maillist  -  Zope-Dev@zope.org
 http://mail.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  http://mail.zope.org/mailman/listinfo/zope-announce
  http://mail.zope.org/mailman/listinfo/zope )




-- 
Fábio Rizzo Matos
ThreePointsWeb
[EMAIL PROTECTED]
http://www.threepointsweb.com
+55 61 3202-6480

Python, Zope e Plone com quem entende do assunto!
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZCatalog caching with memcached

2008-10-26 Thread Fabio Rizzo Matos
Very Nice.

Have a nice holiday :-)

On Sun, Oct 26, 2008 at 3:58 PM, Roché Compaan
[EMAIL PROTECTED]wrote:

 Hi Fabio

 The funkload tests were project specific. I plan to write up my findings
 and to do benchmarks on a standard Plone instance and blog about it.
 This will unfortunately have to wait since I'm on holiday this week :-)

 --
 Roché Compaan
 Upfront Systems   http://www.upfrontsystems.co.za

 On Sun, 2008-10-26 at 15:54 -0200, Fabio Rizzo Matos wrote:
  Hi Roché,
 
  I can see your funkload profile?
 
  On Sun, Oct 26, 2008 at 3:43 PM, Roché Compaan
  [EMAIL PROTECTED] wrote:
  On Sat, 2008-10-25 at 09:20 +0200, Hedley Roos wrote:
Have you measures the time needs for some standard
  ZCatalog queries
used with a Plone site with the communication overhead
  with memcached?
Generally spoken: I think the ZCatalog is in general fast.
  Queries using a
fulltext index are known to be more expensive or if you
  have to deal with
large resultsets or complex queries.
   
  
   No I haven't. Roche Compaan has done extensive benchmarking
  using
   funkload testing plain catalog vs module level cache vs
  memcached, but
   the tests are more about page serving than catalog query
  time. I'll
   ask him to comment more on that.
 
 
  I actually did some profiling as well and catalog searches
  were just too
  damn slow. The average execution time for searchResults was
  100
  milliseconds and this is why I told Hedley we should do some
  caching at
  query level in the first place. I experimented with this idea
  a couple
  of years back but wasn't successful due to inexperience. I was
  trying to
  cache brains which obviously leads to persistency bugs. This
  time around
  it was obvious to me that we should cache the IISet result
  sets.
 
  I suspect specific indexes are just performing suboptimally
  and needs to
  be improved. ExtendPathIndex in Plone seems to be one of them.
 
  The effect on performance is really awesome, now we just need
  to fine
  tune the implementation.
 
  --
  Roché Compaan
  Upfront Systems
  http://www.upfrontsystems.co.za
 
 
  ___
  Zope-Dev maillist  -  Zope-Dev@zope.org
  http://mail.zope.org/mailman/listinfo/zope-dev
  **  No cross posts or HTML encoding!  **
  (Related lists -
   http://mail.zope.org/mailman/listinfo/zope-announce
   http://mail.zope.org/mailman/listinfo/zope )
 
 
 
 
  --
  Fábio Rizzo Matos
  ThreePointsWeb
  [EMAIL PROTECTED]
  http://www.threepointsweb.com
  +55 61 3202-6480
 
  Python, Zope e Plone com quem entende do assunto!
  ___
  Zope-Dev maillist  -  Zope-Dev@zope.org
  http://mail.zope.org/mailman/listinfo/zope-dev
  **  No cross posts or HTML encoding!  **
  (Related lists -
   http://mail.zope.org/mailman/listinfo/zope-announce
   http://mail.zope.org/mailman/listinfo/zope )





-- 
Fábio Rizzo Matos
ThreePointsWeb
[EMAIL PROTECTED]
http://www.threepointsweb.com
+55 61 3202-6480

Python, Zope e Plone com quem entende do assunto!
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )