-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jonathan wrote: > > ----- Original Message ----- From: "Tres Seaver" <[EMAIL PROTECTED]> >> >> Jonathan wrote: >>> During recent load testing of a new application 3.1% to 7.6% of all >>> http requests resulted in conflict errors >>> (3.1% with 10 simultaneous users; 7.6% with 50 simultaneous users). >>> >>> The conflict error occurs when the application attempts to write a >>> small image object into a temporary folder, and each conflict error >>> generates the same traceback: >>> >>> ConflictError: database conflict error (oid 0x07, class >>> Products.TemporaryFolder.TemporaryFolder.SimpleTemporaryContainer) >>> >>> >>> >>> I am running Zope 2.9.2 on CentOS 4.3 (linux). >>> >>> Does anyone have any ideas as to what I could do to reduce/eliminate >>> these conflict errors? >> >> First, make sure that your application is not trying to overwrite the >> *same* image in each request. If it needs to do that, then the >> conflicts are unavoidable. >> >> Next, making simultaneous writes into any naive container is going to >> cause conflicts. You need to think carefully about how you want those >> writes to work, and plan to minimize conflicts: >> >> - Use a BTreeFolder, rather than a normal OFS.Folder (or derivative). >> >> - Arrange for the IDs of your images to be substantially different, >> typically by mangling in a random number (or, for instance, the >> microseconds value of the current timestamp). > > > The ids are assigned sequentially, so they never collide, but should id > be 'substantially different' to improve BTreeFolder2 performance?
Sequential IDs can be problematic: they lead to many more bucket splits (and therefore conflicts) in the BTree. Adding some "entropy" (like a random number or the milliseconds of the time) *earlier* in the key than any sequence number, will help keep splits down to a minimum. > As per your suggestion I added a BTreeFolder2 folder inside of the > temporary folder and used that for image storage and ran some more load > tests. There was a significant improvement: with 25 simultaneous users > the error conflict error rate dropped from 4.4% to 0.6% and there was an > unexpected side effect - a 20% improvement in server performance!!! > > Thanks for the most excellent suggestion Tres! You're welcome. Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 [EMAIL PROTECTED] Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v126.96.36.199 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEo/9H+gerLs4ltQ4RAqgRAJ9zWJMtCLWT1rkyoOM3GIPpovlccwCeOlnl 5yioYSdimjt7FyXWRODDA6A= =tNlC -----END PGP SIGNATURE----- _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )