Re: [ZODB-Dev] ZEO and time.sleep
On Mar 18, 2009, at 8:09 PM, Laurence Rowe wrote: For Plone, the standard remedy to this problem is to separate out portal_catalog into it's own storage (zeo has support for serving multiple storages). Minor note. The ability to host multiple storages in ZEO is unrelated to the use of multiple storages for the application. An application can as easily use multiple ZEO servers and multiple storages hosted by a single server. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZEO and time.sleep
Hi all... I'm profiling a big plone 2.5 instance with huge performance problems and I was wondering if this bug is still present in zope 2.9.9-final, http://mail.zope.org/pipermail/zodb-dev/2007-March/010855.html We can't increment the zodb-cache-size because we're running out of memory... so a lot of times we read objects from zeo/zodb... Any help will be really appreciated... Saludos... signature.asc Description: Esta parte del mensaje está firmada digitalmente ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO and time.sleep
On Mar 18, 2009, at 9:04 AM, Juan Pablo Gimenez wrote: Hi all... I'm profiling a big plone 2.5 instance with huge performance problems and I was wondering if this bug is still present in zope 2.9.9-final, http://mail.zope.org/pipermail/zodb-dev/2007-March/010855.html No, it was fixed a long time ago. We can't increment the zodb-cache-size because we're running out of memory... so a lot of times we read objects from zeo/zodb... You can also increment the zeo cache size. Most likely, your application is using more data than it actually needs to. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO and time.sleep
Jim Fulton wrote: http://mail.zope.org/pipermail/zodb-dev/2007-March/010855.html No, it was fixed a long time ago. For those who are stuck with an older version, I wrote a nanosleep implementation to replace time.sleep with: http://www.upfrontsystems.co.za/Members/izak/sysadman/nanosleep-for-python-2.4 regards, Izak ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO and time.sleep
For Plone, the standard remedy to this problem is to separate out portal_catalog into it's own storage (zeo has support for serving multiple storages). You may then control the object cache size per storage, setting the one for the portal_catalog storage large enough to keep all it's objects in the cache. As navigation is driven from the catalog this can significantly help performance and reduce the number of zeo loads to only those objects required to traverse to the published object. Other things that might help: * Reduce the number of zserver-threads from the default 4, object caches are per thread so this allows you to have fewer, larger caches. * Use FileSystemStorage for Archetypes, this can help if you serve many files. Files are stored in 64k pdata chunks, serving a large file can clear your cache. With newer versions of Plone you can use ZODB 3.8 and blobs. * Put Varnish or some other proxy cache in front and cache agressively. * Buy more memory, memory is cheap. Hope that helps, Laurence Juan Pablo Gimenez wrote: Hi all... I'm profiling a big plone 2.5 instance with huge performance problems and I was wondering if this bug is still present in zope 2.9.9-final, http://mail.zope.org/pipermail/zodb-dev/2007-March/010855.html We can't increment the zodb-cache-size because we're running out of memory... so a lot of times we read objects from zeo/zodb... Any help will be really appreciated... Saludos... ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO and time.sleep
On Mar 28, 2007, at 6:28 PM, Benji York wrote: Last week I spent a very enjoyable day (no kidding) debugging a very, very slow cold-start situation (more than 15 minutes to return from the first request). When making the first request to the app (Zope 3 based), the app server and storage server would show virtually no CPU utilization, and there would be about a megabit of network traffic (on a gigabit link). There was no obvious bottleneck. After liberal application of strace, tcpdump, wireshark (aka ethereal), and the Python profiler we discovered that while waiting for an outstanding request for an object to load, ZEO calls a threading.Connection threading.Condition :) instance's wait method with a timeout. When given a timeout that method enters a wait loop with a time.sleep to sleep for a while and then see if the condition has been met. We found that time.sleep on that box had a minimum granularity of 10ms (when passed a non-zero value), thus causing each object load to take approximately that long. As you can imagine, that somewhat slowed down the retrieval of the several thousand objects required to satisfy the initial request(s) (until the ZEO cache was sufficiently warm). The fix? Short-term: bump the operating system's timer interrupt on that box to 1000Hz from 100Hz, increasing time.sleep's granularity to 1ms (this was on Linux, Window's time.sleep appears have a much higher resolution). Long-term: Jim has found that the timeout call in the wait-for- result code can be avoided, side-stepping the call to time.sleep altogether. What was the time to return the first request after the fix? Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZEO and time.sleep
On Mar 28, 2007, at 6:37 PM, Benji York wrote: Jim Fulton wrote: On Mar 28, 2007, at 6:28 PM, Benji York wrote: threading.Connection threading.Condition :) Yeah, that. :) What was the time to return the first request after the fix? I didn't stick around for the post-Linux-kernel-recompile testing, but when we hacked the Python standard library to use sleep(0) all the time (emulating an infinite granularity timer), it went down to about 10 seconds. A pretty nice improvement. ;) And the original time was 15 minutes! :) Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
