Re: [Zope-CMF] CMF write performance as poor as Plone?
ooopss - wrong list :-) On 24.11.2008 10:42 Uhr, Andreas Jung wrote: So weit bei mir nichts dazwischen kommt könnte ich ab 13h. Gruß, Andreas On 22.11.2008 15:39 Uhr, Charlie Clark wrote: zop Am 21.11.2008 um 07:31 schrieb Andreas Jung: hmmso why is CMF here nearly as bad a Plone. In Plone we know that everything is indexed various times (also in CMF I think) but Plone has much more indexes and metadata compared to CMF. A request in Plone goes through much more layers than in CMFI am currently clueless interpreting the results. My current interpretation is: a custom CMF-based implementation of a CMS will be comparable slow/ fast as an out-of-the-box solution?! ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests -- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376 Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535 Geschäftsführer/Gesellschafter: ZOPYX Limited, Birmingham, UK E-Publishing, Python, Zope & Plone development, Consulting begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. & Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:[EMAIL PROTECTED] title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
So weit bei mir nichts dazwischen kommt könnte ich ab 13h. Gruß, Andreas On 22.11.2008 15:39 Uhr, Charlie Clark wrote: zop Am 21.11.2008 um 07:31 schrieb Andreas Jung: hmmso why is CMF here nearly as bad a Plone. In Plone we know that everything is indexed various times (also in CMF I think) but Plone has much more indexes and metadata compared to CMF. A request in Plone goes through much more layers than in CMFI am currently clueless interpreting the results. My current interpretation is: a custom CMF-based implementation of a CMS will be comparable slow/ fast as an out-of-the-box solution?! begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. & Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:[EMAIL PROTECTED] title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ross Patterson wrote: > Tres Seaver <[EMAIL PROTECTED]> writes: > >> Ross Patterson wrote: >>> Tres Seaver <[EMAIL PROTECTED]> writes: >>> Andreas Jung wrote: > On 23.11.2008 11:57 Uhr, Charlie Clark wrote: >> Am 23.11.2008 um 09:24 schrieb Andreas Jung: >> >>> This issue is independent of the client-side. ab2 and cmf/plone were >>> running on the same (fast) machine. >> Is this really content that is suited for the ZODB? > I am talking of the standard content-types that are available in CMF and > Plone like Document, News etc. Are you trying to proceess feed content, one document per request? You should be batching up that content, to get better amortization of the indexing overhead. You should also be checking for conflict errors: they would result in a *large* performance hit. >>> Quick question. How much of the write concurrency problem comes from >>> the fact that the ZPublisher retries the entire *request* on >>> WriteConflict instead of just trying to committing the transaction >>> again? >> It can't retry the commit: the point is that the conflict can't be >> resolved using the values read from the database at the start of the >> transaction. We have to redo the whole request, re-reading any objects >> in the context of a new transaction; otherwise, we would be back to >> "last write wins", which is a lose. > > Not really what I meant, sorry for being unclear. I meant how much of > the write performance problem comes from the overhead of redoing the > whole request, rendering templates and such? I'm just wondering if > anyone has quantified that. I doubt anyone has done so: the results would likely be highly dependent on the specific application. Back in the day, the render-time penalty was one of the reasons I disliked self-posting forms (for updates): I preferred the "update-and-redirect" model, both to avoid rendering during / after the update, and also to avoid screwing up the browser cache with POST requests. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJKbJZ+gerLs4ltQ4RAr3fAKCbL7zElLcQ+ay8J3vl3EvUEwX0wQCfaEPY y12fwCwskKLPZy/a0hVrbvU= =gBMJ -END PGP SIGNATURE- ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
Tres Seaver <[EMAIL PROTECTED]> writes: > Ross Patterson wrote: >> Tres Seaver <[EMAIL PROTECTED]> writes: >> >>> Andreas Jung wrote: On 23.11.2008 11:57 Uhr, Charlie Clark wrote: > Am 23.11.2008 um 09:24 schrieb Andreas Jung: > >> This issue is independent of the client-side. ab2 and cmf/plone were >> running on the same (fast) machine. > Is this really content that is suited for the ZODB? I am talking of the standard content-types that are available in CMF and Plone like Document, News etc. >>> Are you trying to proceess feed content, one document per request? You >>> should be batching up that content, to get better amortization of the >>> indexing overhead. >>> >>> You should also be checking for conflict errors: they would result in a >>> *large* performance hit. >> >> Quick question. How much of the write concurrency problem comes from >> the fact that the ZPublisher retries the entire *request* on >> WriteConflict instead of just trying to committing the transaction >> again? > > It can't retry the commit: the point is that the conflict can't be > resolved using the values read from the database at the start of the > transaction. We have to redo the whole request, re-reading any objects > in the context of a new transaction; otherwise, we would be back to > "last write wins", which is a lose. Not really what I meant, sorry for being unclear. I meant how much of the write performance problem comes from the overhead of redoing the whole request, rendering templates and such? I'm just wondering if anyone has quantified that. Ross ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ross Patterson wrote: > Tres Seaver <[EMAIL PROTECTED]> writes: > >> Andreas Jung wrote: >>> On 23.11.2008 11:57 Uhr, Charlie Clark wrote: Am 23.11.2008 um 09:24 schrieb Andreas Jung: > This issue is independent of the client-side. ab2 and cmf/plone were > running on the same (fast) machine. Is this really content that is suited for the ZODB? >>> >>> I am talking of the standard content-types that are available in CMF and >>> Plone like Document, News etc. >> Are you trying to proceess feed content, one document per request? You >> should be batching up that content, to get better amortization of the >> indexing overhead. >> >> You should also be checking for conflict errors: they would result in a >> *large* performance hit. > > Quick question. How much of the write concurrency problem comes from > the fact that the ZPublisher retries the entire *request* on > WriteConflict instead of just trying to committing the transaction > again? It can't retry the commit: the point is that the conflict can't be resolved using the values read from the database at the start of the transaction. We have to redo the whole request, re-reading any objects in the context of a new transaction; otherwise, we would be back to "last write wins", which is a lose. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJKaxt+gerLs4ltQ4RApTVAKDHRJBiEBOqedIh8bh27d7VJMMozQCfacFs jhNWzVm+L5XWHuK2XxNYlR4= =Xgr3 -END PGP SIGNATURE- ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
Tres Seaver <[EMAIL PROTECTED]> writes: > Andreas Jung wrote: >> On 23.11.2008 11:57 Uhr, Charlie Clark wrote: >>> Am 23.11.2008 um 09:24 schrieb Andreas Jung: >>> This issue is independent of the client-side. ab2 and cmf/plone were running on the same (fast) machine. >>> >>> Is this really content that is suited for the ZODB? >> >> >> I am talking of the standard content-types that are available in CMF and >> Plone like Document, News etc. > > Are you trying to proceess feed content, one document per request? You > should be batching up that content, to get better amortization of the > indexing overhead. > > You should also be checking for conflict errors: they would result in a > *large* performance hit. Quick question. How much of the write concurrency problem comes from the fact that the ZPublisher retries the entire *request* on WriteConflict instead of just trying to committing the transaction again? Ross ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andreas Jung wrote: > On 23.11.2008 11:57 Uhr, Charlie Clark wrote: >> Am 23.11.2008 um 09:24 schrieb Andreas Jung: >> >>> This issue is independent of the client-side. ab2 and cmf/plone were >>> running on the same (fast) machine. >> >> Is this really content that is suited for the ZODB? > > > I am talking of the standard content-types that are available in CMF and > Plone like Document, News etc. Are you trying to proceess feed content, one document per request? You should be batching up that content, to get better amortization of the indexing overhead. You should also be checking for conflict errors: they would result in a *large* performance hit. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJKVZL+gerLs4ltQ4RAtfYAJ4y11dcMlEFLJYKxcRCvI+qoWM3zgCg0Sh2 0CvciY+vn6F+XLC8eedAirs= =bmyq -END PGP SIGNATURE- ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
On 23.11.2008 11:57 Uhr, Charlie Clark wrote: Am 23.11.2008 um 09:24 schrieb Andreas Jung: This issue is independent of the client-side. ab2 and cmf/plone were running on the same (fast) machine. Is this really content that is suited for the ZODB? I am talking of the standard content-types that are available in CMF and Plone like Document, News etc. I'm just thinking of an environment with lots of concurrent writes and content management doesn't spring directly to mind. Write performance specially in Plone in always a topic in large installations. Read performance is doable on large installations with caching etc. But we know of several Plone projects that failed at some point because you can't get a reasonable performance with lots of editors working the same time with the system. If that isn't possible then you will probably have to look at using an RDBMS and even then you might need server-side optimisations for performance? I just benchmarked that. I just wrote a a simple RDBMS-based Document implementation (using SQLAlchemy and table inheritance for sharing the dublin core table structure among different content-types). Against a Postgres 7.4 database I could reach a performance of roughly 350 new documents per second with a pure python implementation. I am now going to test the performance of the implementation with a CMF integration. Andreas begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. & Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:[EMAIL PROTECTED] title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
Am 23.11.2008 um 09:24 schrieb Andreas Jung: > This issue is independent of the client-side. ab2 and cmf/plone were > running on the same (fast) machine. Is this really content that is suited for the ZODB? I'm just thinking of an environment with lots of concurrent writes and content management doesn't spring directly to mind. Have you looked at performance with a standalone ZODB? If that isn't possible then you will probably have to look at using an RDBMS and even then you might need server-side optimisations for performance? Charlie -- Charlie Clark Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-938-5360 GSM: +49-178-782-6226 ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
Andreas Jung wrote at 2008-11-23 09:24 +0100: > ... >> Have you profiled an individual request to learn what the time is spent for? >> >> If the time is dominated by client side activity, you need client side >> skaling to increase the throughput significantly. > >> >> If, on the other hand, the time is heavily dominated by server side activity, >> you would need backend skaling (which is currently quite difficult >> to get) or get the backend significantly faster (I assume that >> the bandwidth is not the limiting factor). >> > >This issue is independent of the client-side. ab2 and cmf/plone were >running on the same (fast) machine. The other responses you have gotten indicate that client side activity (especially indexing) can significantly influence the observed "write performance" (going down by one order of magnitude). That means: it might be worth to check explicitely that this is not the case in your setup. If indexing is suppressed/delayed (as indicated in the other responses), the number of "store" operations also decrease. Each "store" operation may account for a few ms (I have observed a general ZEO overhead of about 3 ms -- however some years ago and therefore on an older hardware geneation; things may be faster nowadays). Looking at the number of your "store"s (and the time they take), you may check whether this may account for the pure write performance you observe. In this case, optimizations would be possible. In principle, the "store"s could be batched rather than transmitted individually (of course, this would require ZODB modifications). -- Dieter ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
On 23.11.2008 9:17 Uhr, Dieter Maurer wrote: Andreas Jung wrote at 2008-11-21 07:31 +0100: ... write performance ... However the transaction size does not seem to have any impact on the number of simulataneous writes. Have you profiled an individual request to learn what the time is spent for? If the time is dominated by client side activity, you need client side skaling to increase the throughput significantly. If, on the other hand, the time is heavily dominated by server side activity, you would need backend skaling (which is currently quite difficult to get) or get the backend significantly faster (I assume that the bandwidth is not the limiting factor). This issue is independent of the client-side. ab2 and cmf/plone were running on the same (fast) machine. Andreas begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. & Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:[EMAIL PROTECTED] title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
Andreas Jung wrote at 2008-11-21 07:31 +0100: > ... write performance ... >However the transaction size does not >seem to have any impact on the number of simulataneous writes. Have you profiled an individual request to learn what the time is spent for? If the time is dominated by client side activity, you need client side skaling to increase the throughput significantly. If, on the other hand, the time is heavily dominated by server side activity, you would need backend skaling (which is currently quite difficult to get) or get the backend significantly faster (I assume that the bandwidth is not the limiting factor). -- Dieter ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
On 22.11.2008 15:39 Uhr, Charlie Clark wrote: Hi Andreas, a very interesting situation. I've never thought of object creation when choosing "pure" CMF over Plone as this is largely a ZODB issue. As Roché points out it is less likely to be the transactions and more likely to be the cataloguing and any other event subscribers that are limiting factors here. Speed comparions between CMF and Plone only make sense for serving content where I find pure CMF to be at least 10 times as fast as Plone - I think the speed difference is largely down to the sheer size of Archetypes and the overloading of getattr(). The catalog is of course a hotspot. There is collective.indexing and the catalog queue that ease the pain a bit and bring some improvements to the overall performance (for both Plone and CMF) - however not as satisfying as I was thinking of. Going with a RDBMS as backend is likely the only option when it comes scalablity on a system with lots of concurrent write. Andreas begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. & Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:[EMAIL PROTECTED] title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
zop
Am 21.11.2008 um 07:31 schrieb Andreas Jung:
> hmmso why is CMF here nearly as bad a Plone. In Plone we know
> that everything is indexed various times (also in CMF I think) but
> Plone has much more indexes and metadata compared to CMF. A request
> in Plone goes through much more layers than in CMFI am currently
> clueless interpreting the results. My current interpretation is: a
> custom CMF-based implementation of a CMS will be comparable slow/
> fast as an out-of-the-box solution?!
Hi Andreas,
a very interesting situation. I've never thought of object creation
when choosing "pure" CMF over Plone as this is largely a ZODB issue.
As Roché points out it is less likely to be the transactions and more
likely to be the cataloguing and any other event subscribers that are
limiting factors here. Speed comparions between CMF and Plone only
make sense for serving content where I find pure CMF to be at least 10
times as fast as Plone - I think the speed difference is largely down
to the sheer size of Archetypes and the overloading of getattr().
For tests you might want to see if you can defer indexing and I'd
suggest explicit transaction management. Any chance you can run the
tests outside the ZMI?
I currently have a migration from Plone to CMF and this is a bit of
the code I use.
import transaction
from zope.component import createObject
from zope.app.container.interfaces import INameChooser
from zope.event import notify
from zope.lifecycleevent import ObjectCreatedEvent
for i in range(randint(1,500)):
d = createObject("cmf.document", "")
d.id = INameChooser(results_folder).chooseName(d.getId(), d) # this
will be a bottleneck
results_folder._setObject(d.id, d)
transaction.commit()
for obj in results_folder.contentValues(filter='Document'):
notify(ObjectCreatedEvent(d))
You can run this to see the time it takes to create the invidiual
objects. You could run the transaction within the loop to see what
effect that has and you can add notification of the object creation to
the loop or outside of it to see how the event subscribers (usually
the portal catalogue) affects things.
Charlie
--
Charlie Clark
Helmholtzstr. 20
Düsseldorf
D- 40215
Tel: +49-211-938-5360
GSM: +49-178-782-6226
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMF write performance as poor as Plone?
On Fri, 2008-11-21 at 07:31 +0100, Andreas Jung wrote: > Hi there, > > we are currently doing consultancy work for a bigger Zope-based CMS > project (lotes of users, lots of concurrent editors/write). Plone and > CMF are basically what we are looking into right now. It is well-known > that Plone is poor on concurrent write. It is basically impossible > creating more than 3-4 objects per second on decent hardware. On > observation we made was that Plone causes transaction size of 30k-100k > per each new object (this is also true trivial changes on existing > content objects). So I thought this might be a limiting factor and > looked at CMF. The transaction sizes under CMF are much smaller - > typically between 2k and 4k for new objects or object changes which is > looking good at the first glance. However the transaction size does not > seem to have any impact on the number of simulataneous writes. > I wrote script simple script like this: > > results_folder[randint(1,500)].invokeFactory(Document, some_id) > > where result_folder is a btree based folder containing 500 other empty > btree folders. > > Running 'ab2 -n 100' against the site would create 100 new documents > distributed over the subfolder (avoid conflicts errors here). > The performance was nearly as bad as with Plone. It was hard getting > more 4-6 new objects per second out of a standard CMF site (2.1, > zope.schema-based types). Even variations of the zserver-threads and the > ab2 concurrency level did not help much. > > hmmso why is CMF here nearly as bad a Plone. In Plone we know that > everything is indexed various times (also in CMF I think) but Plone has > much more indexes and metadata compared to CMF. A request in Plone goes > through much more layers than in CMFI am currently clueless > interpreting the results. My current interpretation is: a custom > CMF-based implementation of a CMS will be comparable slow/fast as an > out-of-the-box solution?! I also suspected transaction size a while back but found that it doesn't really impact on write performance. The one thing that stood out clearly was the time it takes to index objects. I managed writes of up to 20 objects per second if I created the object with _setObject while suppressing all events and manually re-indexing the object after creation. Indexing in Plone improved once we cached the expensive Schema lookup that is made for each attribute (see discussion on this in plone-dev) but this shouldn't be an issue in CMF. Maybe there is something else in CMF that slows down attribute access when objects are indexed. I can almost guarantee that there is some application code that slows down writes, since I know now that indexing and object creation is relatively fast. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
[Zope-CMF] CMF write performance as poor as Plone?
Hi there, we are currently doing consultancy work for a bigger Zope-based CMS project (lotes of users, lots of concurrent editors/write). Plone and CMF are basically what we are looking into right now. It is well-known that Plone is poor on concurrent write. It is basically impossible creating more than 3-4 objects per second on decent hardware. On observation we made was that Plone causes transaction size of 30k-100k per each new object (this is also true trivial changes on existing content objects). So I thought this might be a limiting factor and looked at CMF. The transaction sizes under CMF are much smaller - typically between 2k and 4k for new objects or object changes which is looking good at the first glance. However the transaction size does not seem to have any impact on the number of simulataneous writes. I wrote script simple script like this: results_folder[randint(1,500)].invokeFactory(Document, some_id) where result_folder is a btree based folder containing 500 other empty btree folders. Running 'ab2 -n 100' against the site would create 100 new documents distributed over the subfolder (avoid conflicts errors here). The performance was nearly as bad as with Plone. It was hard getting more 4-6 new objects per second out of a standard CMF site (2.1, zope.schema-based types). Even variations of the zserver-threads and the ab2 concurrency level did not help much. hmmso why is CMF here nearly as bad a Plone. In Plone we know that everything is indexed various times (also in CMF I think) but Plone has much more indexes and metadata compared to CMF. A request in Plone goes through much more layers than in CMFI am currently clueless interpreting the results. My current interpretation is: a custom CMF-based implementation of a CMS will be comparable slow/fast as an out-of-the-box solution?! Thoughts? Andreas -- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376 Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535 Geschäftsführer/Gesellschafter: ZOPYX Limited, Birmingham, UK E-Publishing, Python, Zope & Plone development, Consulting begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. & Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:[EMAIL PROTECTED] title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
