Hash: SHA1

Jonathan wrote:
> ----- Original Message ----- From: "Tres Seaver" <[EMAIL PROTECTED]>
>> Jonathan wrote:
>>> During recent load testing of a new application 3.1% to 7.6% of all
>>> http requests resulted in conflict errors
>>> (3.1% with 10 simultaneous users; 7.6% with 50 simultaneous users).
>>> The conflict error occurs when the application attempts to write a
>>> small image object into a temporary folder, and each conflict error
>>> generates the same traceback:
>>> ConflictError: database conflict error (oid 0x07, class
>>> Products.TemporaryFolder.TemporaryFolder.SimpleTemporaryContainer)
>>> I am running Zope 2.9.2 on CentOS 4.3 (linux).
>>> Does anyone have any ideas as to what I could do to reduce/eliminate
>>> these conflict errors?
>> First, make sure that your application is not trying to overwrite the
>> *same* image in each request.  If it needs to do that, then the
>> conflicts are unavoidable.
>> Next, making simultaneous writes into any naive container is going to
>> cause conflicts.  You need to think carefully about how you want those
>> writes to work, and plan to minimize conflicts:
>>  - Use a BTreeFolder, rather than a normal OFS.Folder (or derivative).
>>  - Arrange for the IDs of your images to be substantially different,
>>    typically by mangling in a random number (or, for instance, the
>>    microseconds value of the current timestamp).
> The ids are assigned sequentially, so they never collide, but should id
> be 'substantially different' to improve BTreeFolder2 performance?

Sequential IDs can be problematic:  they lead to many more bucket splits
(and therefore conflicts) in the BTree.  Adding some "entropy" (like a
random number or the milliseconds of the time) *earlier* in the key than
any sequence number, will help keep splits down to a minimum.

> As per your suggestion I added a BTreeFolder2 folder inside of the
> temporary folder and used that for image storage and ran some more load
> tests.  There was a significant improvement:  with 25 simultaneous users
> the error conflict error rate dropped from 4.4% to 0.6% and there was an
> unexpected side effect - a 20% improvement in server performance!!!
> Thanks for the most excellent suggestion Tres!

You're welcome.

- --
Tres Seaver          +1 202-558-7113          [EMAIL PROTECTED]
Palladion Software   "Excellence by Design"    http://palladion.com
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

Zope maillist  -  Zope@zope.org
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to