Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
On Wed, Oct 13, 2010 at 8:28 PM, Chris Withers ch...@simplistix.co.uk wrote: On 12/10/2010 17:56, Jim Fulton wrote: Currently the only configuration docs are in the .xml Alan provided a link to. The best size will depend on the app. Is there a downside to too big, provided there's plenty of disk space free on the app servers? Yes. On startup, the cache has to be scanned, and large caches can cause excessive IO on startup. ... In ZODB 3.10, the cache tracing analysis tools actually work and produce meaningful numbers if you start tracing with an empty cache. Okay, does that mean the stuff that's documented should be ignored for ZODB 3.9? Cache tracing is busted for 3.9, so yeah. :) The documentation may be wrong for 3.10 too. I don't know. I'm working (slowly) on new docs. This will be executable and thus more likely to be accurate. I went for the following in the end: ... cache-size 200Mb client app wait on /zeoclient mount-point / cache-size 5 cache-size-bytes 500Mb Any obvious issues with that? No. One problem I did hit, though, was if I restarted an app server, I got: File /var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/config.py, line 210, in open **options) File /var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZEO/ClientStorage.py, line 395, in __init__ self._cache = self.ClientCacheClass(cache_path, size=cache_size) File /var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZEO/cache.py, line 194, in __init__ self._lock_file = zc.lockfile.LockFile(path + '.lock') File /var/buildout-eggs/zc.lockfile-1.0.0-py2.6.egg/zc/lockfile/__init__.py, line 76, in __init__ _lock_file(fp) File /var/buildout-eggs/zc.lockfile-1.0.0-py2.6.egg/zc/lockfile/__init__.py, line 59, in _lock_file raise LockError(Couldn't lock %r % file.name) LockError: Couldn't lock 'zec.lock' ZEO caches can't be shared between processes. This is catching that. I think the lock was added in 3.9 to protect you from cache corruption due to multiple processes accessing the cache at once. ...and yet the app server came up fine. This is rather disturbing. What does this error message imply? That two processes were trying to access the cache. The above did not, however, occur if I stopped and then started the app server... Race condition? Hopefully, you screwed up somehow and started the process while it was already running. :) Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
On 14/10/2010 17:09, Jim Fulton wrote: What does this error message imply? That two processes were trying to access the cache. The above did not, however, occur if I stopped and then started the app server... Race condition? Hopefully, you screwed up somehow and started the process while it was already running. :) I don't think so. This is using zc.recipe.rhrc to drive zope2instance Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
On 14/10/2010 17:13, Alan Runyan wrote: I went for the following in the end: ... cache-size 200Mb client app wait on /zeoclient mount-point / cache-size 5 cache-size-bytes 500Mb Any obvious issues with that? No. each zeoclient needs a name in your conf. That's what the 'client' parameter provides, and there's only one client per machine. These clients do open two storages, but the storage name forms part of the cache file name. so the .zec files dont collide. this is what is happening to you. look at component.xml again for details on assigning zeo client a name or identifier. See above ;-) please open issue at launchpad; if this fails zope should NOT start. Sure, which tracker do you recommend? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
I went for the following in the end: ... cache-size 200Mb client app wait on /zeoclient mount-point / cache-size 5 cache-size-bytes 500Mb Any obvious issues with that? No. each zeoclient needs a name in your conf. required so the .zec files dont collide. this is what is happening to you. look at component.xml again for details on assigning zeo client a name or identifier. that will give the filename a suffix like something-$identifier.zec please open issue at launchpad; if this fails zope should NOT start. alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
On 12/10/2010 17:56, Jim Fulton wrote: Currently the only configuration docs are in the .xml Alan provided a link to. The best size will depend on the app. Is there a downside to too big, provided there's plenty of disk space free on the app servers? There are some docs in http://svn.zope.org/ZODB/trunk/doc/, although I don't know if they're accurate. ...yeah, those look a lot like the original docs that Guido(?) wrote. In ZODB 3.10, the cache tracing analysis tools actually work and produce meaningful numbers if you start tracing with an empty cache. Okay, does that mean the stuff that's documented should be ignored for ZODB 3.9? I went for the following in the end: ... cache-size 200Mb client app wait on /zeoclient mount-point / cache-size 5 cache-size-bytes 500Mb Any obvious issues with that? One problem I did hit, though, was if I restarted an app server, I got: File /var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/config.py, line 210, in open **options) File /var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZEO/ClientStorage.py, line 395, in __init__ self._cache = self.ClientCacheClass(cache_path, size=cache_size) File /var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZEO/cache.py, line 194, in __init__ self._lock_file = zc.lockfile.LockFile(path + '.lock') File /var/buildout-eggs/zc.lockfile-1.0.0-py2.6.egg/zc/lockfile/__init__.py, line 76, in __init__ _lock_file(fp) File /var/buildout-eggs/zc.lockfile-1.0.0-py2.6.egg/zc/lockfile/__init__.py, line 59, in _lock_file raise LockError(Couldn't lock %r % file.name) LockError: Couldn't lock 'zec.lock' ...and yet the app server came up fine. What does this error message imply? The above did not, however, occur if I stopped and then started the app server... Race condition? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
I'm wondering if some of my issues might be alleviated by persistent client caches. anything is possible. Now, I have the impression that these were unreliable and buggy and had been so for some time. they appear to be more stable now. we use them on several customer's in very specific cases. mainly with infrequently modified databases. Is this still the case? If not, where are the docs on how to configure them nowadays? http://svn.zope.org/ZODB/tags/3.9.7/src/ZODB/component.xml?rev=117035view=auto key name=client description Enables persistent cache files. The string passed here is used to construct the cache filenames. If it is not specified, the client creates a temporary cache that will only be used by the current object. /description /key -alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
On Tue, Oct 12, 2010 at 11:33 AM, Chris Withers ch...@simplistix.co.uk wrote: Hi All, I'm wondering if some of my issues might be alleviated by persistent client caches. Now, I have the impression that these were unreliable and buggy and had been so for some time. They became much better in later 3.8 releases and still much better lately. We've been using them in production for quite a while. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
On 12/10/2010 17:11, Jim Fulton wrote: Now, I have the impression that these were unreliable and buggy and had been so for some time. They became much better in later 3.8 releases and still much better lately. We've been using them in production for quite a while. Any docs on recommended settings around? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?
On Tue, Oct 12, 2010 at 12:25 PM, Chris Withers ch...@simplistix.co.uk wrote: On 12/10/2010 17:11, Jim Fulton wrote: Now, I have the impression that these were unreliable and buggy and had been so for some time. They became much better in later 3.8 releases and still much better lately. We've been using them in production for quite a while. Any docs on recommended settings around? Currently the only configuration docs are in the .xml Alan provided a link to. The best size will depend on the app. There are some docs in http://svn.zope.org/ZODB/trunk/doc/, although I don't know if they're accurate. In ZODB 3.10, the cache tracing analysis tools actually work and produce meaningful numbers if you start tracing with an empty cache. Like much of ZODB, the documentation is wanting. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
