Re: [Zope-dev] ZEO TempStorage: Odd behavior on ZEO restart
On Wed, Jul 14, 2010 at 6:59 PM, Leonardo Rochael Almeida leoroch...@gmail.com wrote: On Wed, Jul 14, 2010 at 19:22, Benji York be...@benjiyork.com wrote: [...] Not surprising really. What could be surprising is that, since the objects are not in the object cache or the ZEO cache, how can the clients 'know' about them to request them? And the answer is probably that there are other objects which ARE in the object cache, or the ZEO cache and that hold references (ghost objects in the case of the object-cache) to the objects in the zeo-distributed temporary storage. Indeed. My jet lag ate that part of the story. Thanks. So, perhaps, Sebastian can avoid a Zope restart if he finds a way to flush all caches. Flushing the object caches is easy, it's in the Control_Panel. Flushing the ZEO cache is something else. Perhaps he can run with a 0-sized ZEO cache for the TempStorages? If this were me, I'd use a non-temporary storage and implement an expiry policy for the results cache. The bforest package would be a good start (http://pypi.python.org/pypi/zope.bforest). In fact, I would expect they already have an expiry policy in place, so they might be able to just start using a non-temporary storage right away. -- Benji York ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.12.9 broke ZMySQLDA in the middle of a stable release series
On Wed, Jul 14, 2010 at 12:22 AM, Leonardo Rochael Almeida leoroch...@gmail.com wrote: The attached patch adds such a warning, and also reveals where Zope2 commits the same sins. If there are no objections I'll commit it to 2.12 and trunk. Thanks for committing it. Finally, I'd like to ask that, when major changes land in a stable series (like the spinning off of a whole package), that we allow at least a week or two to pass before making a release, so people who have test runners for their apps running against a stable repository branch have time to adapt to the changes That sounds reasonable. I just wasn't aware that anyone except CMF actually tracks Zope 2 SVN and there would be any point in waiting at all. Unless the nightly CMF builds reported problems, I considered the code to be stable and ready for release. But to avoid any such issues, I'll restrain myself from doing any further feature changes to Zope 2.12. 2.13 is closing in fast enough and there's always a 2.14. Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.12.9 broke ZMySQLDA in the middle of a stable release series
Am 15.07.2010, 10:35 Uhr, schrieb Hanno Schlichting ha...@hannosch.eu: But to avoid any such issues, I'll restrain myself from doing any further feature changes to Zope 2.12. 2.13 is closing in fast enough and there's always a 2.14. I'll check ZPsycopgDA and make sure we look at the mxODBCZope DA as well. I'm not sure what other Products are likely to be affected by these kind of changes. Charlie -- Charlie Clark Managing Director Clark Consulting Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 51 OK, 17 Failed, 2 Unknown
Summary of messages to the zope-tests list. Period Wed Jul 14 12:00:00 2010 UTC to Thu Jul 15 12:00:00 2010 UTC. There were 70 messages: 6 from Zope Tests, 2 from buildbot at enfoldsystems.com, 4 from buildbot at pov.lt, 17 from buildbot at winbot.zope.org, 13 from ccomb at free.fr, 28 from jdriessen at thehealthagency.com. Test failures - Subject: FAILED : Bluebream / Python2.4.6 32bit linux From: ccomb at free.fr Date: Wed Jul 14 22:04:06 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016701.html Subject: FAILED : Bluebream / Python2.5.2 32bit linux From: ccomb at free.fr Date: Wed Jul 14 22:05:46 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016702.html Subject: FAILED : Bluebream / Python2.6.4 32bit linux From: ccomb at free.fr Date: Wed Jul 14 22:05:47 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016703.html Subject: FAILED : winbot / ztk_dev py_244_win32 From: buildbot at winbot.zope.org Date: Wed Jul 14 22:13:17 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016705.html Subject: FAILED : winbot / ztk_dev py_270_win32 From: buildbot at winbot.zope.org Date: Wed Jul 14 22:58:45 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016710.html Subject: FAILED : winbot / ztk_dev py_270_win64 From: buildbot at winbot.zope.org Date: Wed Jul 14 23:10:08 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016711.html Subject: FAILED : winbot / ztk_10 py_244_win32 From: buildbot at winbot.zope.org Date: Wed Jul 14 23:18:20 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016712.html Subject: FAILED : winbot / ztk_10 py_270_win32 From: buildbot at winbot.zope.org Date: Wed Jul 14 23:46:31 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016716.html Subject: FAILED : winbot / ztk_10 py_270_win64 From: buildbot at winbot.zope.org Date: Wed Jul 14 23:53:21 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016717.html Subject: FAILED : ZTK 1.0dev / Python2.7.0 Linux 32bit From: ccomb at free.fr Date: Thu Jul 15 00:17:19 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016718.html Subject: FAILED : winbot / ZODB_dev py_254_win32 From: buildbot at winbot.zope.org Date: Thu Jul 15 00:56:27 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016723.html Subject: FAILED : winbot / ZODB_dev py_265_win32 From: buildbot at winbot.zope.org Date: Thu Jul 15 01:51:03 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016725.html Subject: FAILED : ZTK 1.0 / Python2.7.0 Linux 32bit From: ccomb at free.fr Date: Thu Jul 15 01:53:13 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016727.html Subject: FAILED : winbot / ZODB_dev py_265_win64 From: buildbot at winbot.zope.org Date: Thu Jul 15 02:45:43 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016730.html Subject: FAILED : winbot / ZODB_dev py_270_win32 From: buildbot at winbot.zope.org Date: Thu Jul 15 03:41:27 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016732.html Subject: FAILED : winbot / ZODB_dev py_270_win64 From: buildbot at winbot.zope.org Date: Thu Jul 15 04:37:15 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016734.html Subject: FAILED : Zope Buildbot / ztk_win slave-win From: jdriessen at thehealthagency.com Date: Thu Jul 15 07:09:15 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016735.html Unknown --- Subject: [zodb-tests] buildbot failure in Enfold Systems on zodb-trunk-python-2.6-maestro From: buildbot at enfoldsystems.com Date: Thu Jul 15 03:01:34 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016731.html Subject: [zodb-tests] buildbot failure in Enfold Systems on zodb-trunk-python-2.5-maestro From: buildbot at enfoldsystems.com Date: Thu Jul 15 03:57:38 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016733.html Tests passed OK --- Subject: OK : Zope Buildbot / zope2.12 slave-ubuntu64 From: jdriessen at thehealthagency.com Date: Wed Jul 14 08:55:13 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016675.html Subject: OK : Zope Buildbot / zope2.12 slave-osx From: jdriessen at thehealthagency.com Date: Wed Jul 14 08:55:25 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016676.html Subject: OK : Zope Buildbot / zope2.12 slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Wed Jul 14 08:55:27 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016677.html Subject: OK : Zope Buildbot / zope2 slave-ubuntu64 From: jdriessen at thehealthagency.com Date: Wed Jul 14 08:57:03 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016678.html Subject: OK : Zope Buildbot / zope2 slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Wed Jul 14 08:57:23 EDT 2010 URL:
Re: [Zope-dev] ZEO TempStorage: Odd behavior on ZEO restart
Hi Sebastian,
Why don’t you consider to use memcached for caching the results instead of
using sessions? By using this, Zeo server/Zeo clients restart wont affect
cached data.
HTH,
Dragos
On Sun, Jul 11, 2010 at 8:41 AM, Sebastian Tiedtke s...@mdvtts.com wrote:
Hi all,
We're working on a large scale Plone deployment and recently came across
some odd behavior with tempstorage at our ZEO server. A limited number, yet
highly frequented, of pages on our Plone site query backend web services
that provide location based services such as geocoding and routing (a to b).
Since those calls to the services are rather computation intense we decided
to store intermediate and resulting data in sessions vs. re-querying the
service for no reason.
The system setup consists of a central ZEO server that provides the storage
for content as well as a temp storage for sessioning. There are multiple
other servers that run multiple Plone instances with ZEO clients. The ZEO
server setup is straight forward but I'm happy to provide the configuration
files. Generally the concept works great.
However, whenever we restart the ZEO server and keep the client instances
running we get odd behavior with the client instances' sessioning. Every
client appears to reconnect and, if required, revalidates its cache.
2010-07-11T14:46:03 INFO ZEO.ClientStorage (6132) Disconnected from
storage: ('ITIS.mentzdv.de', 8100)
2010-07-11T14:46:03 INFO ZEO.ClientStorage (6132) Disconnected from
storage: ('ITIS.mentzdv.de', 8100)
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Testing connection
ManagedClientConnection ('127.0.0.1', 8100)
2010-07-11T14:46:09 INFO ZEO.zrpc.Connection(C) (127.0.0.1:8100) received
handshake 'Z303'
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Server authentication
protocol None
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Connected to storage: ('
ITIS.mentzdv.de', 8100)
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Testing connection
ManagedClientConnection ('127.0.0.1', 8100)
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) last inval tid:
'\x03\x87n\x9d|\x1a\xc2\xaa' 2010-07-11 04:45:29.087000
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) last transaction:
'\x00\x00\x00\x00\x00\x00\x00\x00' 1900-01-01 00:00:00.00
2010-07-11T14:46:09 INFO ZEO.zrpc.Connection(C) (127.0.0.1:8100) received
handshake 'Z303'
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) Verifying cache
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) Server authentication
protocol None
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) Connected to storage: ('
ITIS.mentzdv.de', 8100)
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) No verification necessary
(last_inval_tid up-to-date)
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) endVerify finishing
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) endVerify finished
The problem is that whenever we hit a session bound page we get following
error and stack trace:
2010-07-11T14:58:12 ERROR ZODB.Connection Couldn't load state for 0x82
Traceback (most recent call last):
File
g:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZODB\Connection.py, line
761, in setstate
self._setstate(obj)
File
g:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZODB\Connection.py, line
801, in _setstate
p, serial = self._storage.load(obj._p_oid, self._version)
File
G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\ClientStorage.py,
line 727, in load
return self.loadEx(oid, version)[:2]
File
G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\ClientStorage.py,
line 750, in loadEx
data, tid, ver = self._server.loadEx(oid, version)
File
G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\ServerStub.py, line
196, in loadEx
return self.rpc.call(loadEx, oid, version)
File
G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\zrpc\connection.py,
line 650, in call
raise inst # error raised by server
KeyError: '\x00\x00\x00\x00\x00\x00\x00\x82'
In the ZMI under the Control-Panel/Database Tab the temporary database
seems to be fine. The /temp_folder returns a similar stack trace though:
Site Error
An error was encountered while publishing this resource.
*KeyError*
Sorry, a site error occurred.
Traceback (innermost last):
- Module ZPublisher.Publish, line 202, in publish_module_standard
- Module ZPublisher.Publish, line 150, in publish
- Module plone.app.linkintegrity.monkey, line 21, in
zpublisher_exception_hook_wrapper
- Module Zope2.App.startup, line 221, in zpublisher_exception_hook
- Module ZPublisher.Publish, line 119, in publish
- Module ZPublisher.mapply, line 88, in mapply
- Module ZPublisher.Publish, line 42, in call_object
- Module App.special_dtml, line 65, in __call__
- Module DocumentTemplate.DT_String, line 476, in __call__
- Module Products.Transience.Transience, line 926, in housekeep
- Module
Re: [Zope-dev] ZEO TempStorage: Odd behavior on ZEO restart
On 7/15/10 17:03 , Dragos Chirila wrote: Hi Sebastian, Why don’t you consider to use memcached for caching the results instead of using sessions? By using this, Zeo server/Zeo clients restart wont affect cached data. or plone.memoize, using either a RAMCache or memcached as backend. Wichert. ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
