Hello, On top of that, be aware that those numbers 'cache-size' (or cache-size-bytes) are for each connection. By default a Z3 ZODB can have up to 7 connections in the pool. That means you might easily end with 7 times the cache size set.
Most people tend to limit the number of connections too. Monday, March 22, 2010, 8:01:30 PM, you wrote: HS> On Mon, Mar 22, 2010 at 7:05 PM, Jeff Shell <j...@bottlerocket.net> wrote: >> Are there any metrics about how to set the ZODB 'cache-size' (or >> cache-size-bytes) option? >> We've been using '5000' (arbitrarily chosen) for our Zope 3.4-ish app >> servers. We have access >> to zc.z3monitor which can output the "number of objects in the object caches >> (combined)" and >> "number of non-ghost objects in the object caches (combined)". >> >> But I don't know understand how to interpret those numbers and use them to >> make better >> settings. HS> ZODB cache size optimization is a typical case of black art. There's HS> no real good way to find the perfect number. I would advise against HS> using the "cache-size-bytes" option. There's a known critical problem HS> with it, that is currently only fixed on the 3.9 SVN branch. So stick HS> to the old object count cache-size for now. HS> But generally database caching is a trade-off between performance and HS> available RAM. As the upper limit, you could have your entire data set HS> in each ZODB cache. So you could look at the number of persistent HS> objects in the database and match your cache-size to that number. HS> That's usually not want you want. HS> As a real strategy, you should set up detailed monitoring of the HS> server. Monitor and graph overall RAM usage, RAM usage per Zope HS> process, number of DB loads and writes over time. Preferably include HS> some way of measuring application request performance and track CPU HS> and I/O usage on the server hosting the database. HS> If you have those numbers, you can play around with the cache setting HS> and increase it. See what impact it has on your application and data HS> set. At some point you run out of memory and need to decrease the HS> number or the increased cache size doesn't actually buy you any HS> application performance anymore. HS> For general Zope 3 application there are no rules of thumb that I HS> know. The dataset and load patterns of the applications are too HS> different to have any of those. It's affected a lot by the dataset and HS> if you use ZODB blobs or another mechanism to store large binary HS> content outside the DB. 5000 persistent objects including 1000 images HS> of 5mb each are obviously very different, than 1000 BTree buckets HS> containing only integers. One main advantage of blobs is that they HS> aren't loaded into the ZODB connection caches, so they lower the HS> memory requirements for applications with binary content a lot. HS> In the Plone context I generally use something like "number of content HS> objects in the catalog" + 5000 objects for the general application as HS> a starting point. But that has a lot of assumptions on the type of HS> content and the application in it. HS> Hanno HS> _______________________________________________ HS> For more information about ZODB, see the ZODB Wiki: HS> http://www.zope.org/Wikis/ZODB/ HS> ZODB-Dev mailing list - ZODB-Dev@zope.org HS> https://mail.zope.org/mailman/listinfo/zodb-dev -- Best regards, Adam GROSZER mailto:agros...@gmail.com -- Quote of the day: You can't cross a chasm in two small jumps. _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev