Re: [Zope-CMF] CMF write performance as poor as Plone?
On Fri, 2008-11-21 at 07:31 +0100, Andreas Jung wrote: > Hi there, > > we are currently doing consultancy work for a bigger Zope-based CMS > project (lotes of users, lots of concurrent editors/write). Plone and > CMF are basically what we are looking into right now. It is well-known > that Plone is poor on concurrent write. It is basically impossible > creating more than 3-4 objects per second on decent hardware. On > observation we made was that Plone causes transaction size of 30k-100k > per each new object (this is also true trivial changes on existing > content objects). So I thought this might be a limiting factor and > looked at CMF. The transaction sizes under CMF are much smaller - > typically between 2k and 4k for new objects or object changes which is > looking good at the first glance. However the transaction size does not > seem to have any impact on the number of simulataneous writes. > I wrote script simple script like this: > > results_folder[randint(1,500)].invokeFactory(Document, some_id) > > where result_folder is a btree based folder containing 500 other empty > btree folders. > > Running 'ab2 -n 100' against the site would create 100 new documents > distributed over the subfolder (avoid conflicts errors here). > The performance was nearly as bad as with Plone. It was hard getting > more 4-6 new objects per second out of a standard CMF site (2.1, > zope.schema-based types). Even variations of the zserver-threads and the > ab2 concurrency level did not help much. > > hmmso why is CMF here nearly as bad a Plone. In Plone we know that > everything is indexed various times (also in CMF I think) but Plone has > much more indexes and metadata compared to CMF. A request in Plone goes > through much more layers than in CMFI am currently clueless > interpreting the results. My current interpretation is: a custom > CMF-based implementation of a CMS will be comparable slow/fast as an > out-of-the-box solution?! I also suspected transaction size a while back but found that it doesn't really impact on write performance. The one thing that stood out clearly was the time it takes to index objects. I managed writes of up to 20 objects per second if I created the object with _setObject while suppressing all events and manually re-indexing the object after creation. Indexing in Plone improved once we cached the expensive Schema lookup that is made for each attribute (see discussion on this in plone-dev) but this shouldn't be an issue in CMF. Maybe there is something else in CMF that slows down attribute access when objects are indexed. I can almost guarantee that there is some application code that slows down writes, since I know now that indexing and object creation is relatively fast. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
[Zope-CMF] CMF write performance as poor as Plone?
Hi there, we are currently doing consultancy work for a bigger Zope-based CMS project (lotes of users, lots of concurrent editors/write). Plone and CMF are basically what we are looking into right now. It is well-known that Plone is poor on concurrent write. It is basically impossible creating more than 3-4 objects per second on decent hardware. On observation we made was that Plone causes transaction size of 30k-100k per each new object (this is also true trivial changes on existing content objects). So I thought this might be a limiting factor and looked at CMF. The transaction sizes under CMF are much smaller - typically between 2k and 4k for new objects or object changes which is looking good at the first glance. However the transaction size does not seem to have any impact on the number of simulataneous writes. I wrote script simple script like this: results_folder[randint(1,500)].invokeFactory(Document, some_id) where result_folder is a btree based folder containing 500 other empty btree folders. Running 'ab2 -n 100' against the site would create 100 new documents distributed over the subfolder (avoid conflicts errors here). The performance was nearly as bad as with Plone. It was hard getting more 4-6 new objects per second out of a standard CMF site (2.1, zope.schema-based types). Even variations of the zserver-threads and the ab2 concurrency level did not help much. hmmso why is CMF here nearly as bad a Plone. In Plone we know that everything is indexed various times (also in CMF I think) but Plone has much more indexes and metadata compared to CMF. A request in Plone goes through much more layers than in CMFI am currently clueless interpreting the results. My current interpretation is: a custom CMF-based implementation of a CMS will be comparable slow/fast as an out-of-the-box solution?! Thoughts? Andreas -- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376 Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535 Geschäftsführer/Gesellschafter: ZOPYX Limited, Birmingham, UK E-Publishing, Python, Zope & Plone development, Consulting begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. & Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:[EMAIL PROTECTED] title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] [dev] Should portal_setup be registered as utility?
Am 20.11.2008 um 12:42 schrieb yuppie: > Several tool methods have to be replaced by view code before we can > register all tools as utilities. It's already on my radar as soon as I get some time to catch my breath (and complete my two other minor tasks first!) >> In view of this, one can understand that Plone has problems with >> the "setup_tool" utility registration. > > The setup tool is responsible for managing configuration data. I can't > see a need to care about the request in that context. Agreed. If third party tools have problems, then they should provide the solutions. Charlie -- Charlie Clark Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-938-5360 GSM: +49-178-782-6226 ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
[Zope-CMF] CMF Tests: 6 OK
Summary of messages to the cmf-tests list. Period Wed Nov 19 12:00:00 2008 UTC to Thu Nov 20 12:00:00 2008 UTC. There were 6 messages: 6 from CMF Tests. Tests passed OK --- Subject: OK : CMF-2.1 Zope-2.10 Python-2.4.5 : Linux From: CMF Tests Date: Wed Nov 19 20:59:35 EST 2008 URL: http://mail.zope.org/pipermail/cmf-tests/2008-November/010411.html Subject: OK : CMF-2.1 Zope-2.11 Python-2.4.5 : Linux From: CMF Tests Date: Wed Nov 19 21:01:05 EST 2008 URL: http://mail.zope.org/pipermail/cmf-tests/2008-November/010412.html Subject: OK : CMF-trunk Zope-2.10 Python-2.4.5 : Linux From: CMF Tests Date: Wed Nov 19 21:02:36 EST 2008 URL: http://mail.zope.org/pipermail/cmf-tests/2008-November/010413.html Subject: OK : CMF-trunk Zope-2.11 Python-2.4.5 : Linux From: CMF Tests Date: Wed Nov 19 21:04:06 EST 2008 URL: http://mail.zope.org/pipermail/cmf-tests/2008-November/010414.html Subject: OK : CMF-trunk Zope-trunk Python-2.4.5 : Linux From: CMF Tests Date: Wed Nov 19 21:05:36 EST 2008 URL: http://mail.zope.org/pipermail/cmf-tests/2008-November/010415.html Subject: OK : CMF-trunk Zope-trunk Python-2.5.2 : Linux From: CMF Tests Date: Wed Nov 19 21:07:06 EST 2008 URL: http://mail.zope.org/pipermail/cmf-tests/2008-November/010416.html ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] [dev] Should portal_setup be registered as utility?
Dieter Maurer wrote: > yuppie wrote at 2008-11-18 12:00 +0100: >> Dieter Maurer wrote: >>> If they would, local utilities were much nearer to tools and >>> the transition would be facilitated. >> They would be nearer to tools, but also more distant from zope 3 >> utilities. I doubt that would really be a win. > > Then, maybe, Zope 3 utilities are inadequate at many places in > to Zope 2 world: e.g. any tool that uses TALES expressions may > want to access the (current, of course) request -- especially > when they are destined for user interaction (as actions are). I agree. Utilities are inadequate for user interaction. Views should be used instead. We don't need the request for managing TALES expressions inside an utility. The request is only needed if we *use* the expressions, and that should be done in view code, not in utilities. Several tool methods have to be replaced by view code before we can register all tools as utilities. > In view of this, one can understand that Plone has problems with > the "setup_tool" utility registration. The setup tool is responsible for managing configuration data. I can't see a need to care about the request in that context. Cheers, Yuppie ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
