I've been working with zope for over 12 years and something that keeps coming 
up is sacling IO bound operations in Zope. The typical example is where you 
build an app that calls external apis. While this is happening a zope thread 
isn't doing any other processing and because there is a 1 thread 1 zodb cache 
limit.  You can run into scalability problems as you can only have as many 
threads your RAM / average cache size. The end result is low throughput while 
still having low CPU. I've consulted on some $$$ sites where others have made 
this mistake. It's an easy mistake to make as SQL/PHP systems don't tend to 
have this limitation so new developers to zope often don't to think of it. The 
possible workarounds aren't pretty. You can segregate your api calling requests 
to zeo clients with large numbers of threads with small caches using some fancy 
load balancing rules. You can rework that  part of your application to not use 
zope, perhaps using edge side includes to make it seem p
 art of the same app.

Feel free to shoot down the following makes no sense.
What if two or more threads could share a zodb cache up until the point at 
which one wants to write. This is the point at which you can't share a cache in 
a consistent manner in my understanding. At that point the transaction could be 
blocked until other readonly transactions had finished and continue by itself? 
or perhaps the write transaction could be aborted and restarted with a special 
flag to ensure it was processed with the cache to itself. As long as requests 
which involve external access are readonly with regard to zope then this would 
improve throughput. This might seem an edge case but consider where you want to 
integrate an external app into a zope or Plone app. Often the external api is 
doing the writing not the zope part. For example clicking a button on a plone 
site to make plone send a tweet. It might also improve throughput on zope 
requests which involve zodb cache misses as they are also IO bound.
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to