[Zope-dev] ZEO TempStorage: Odd behavior on ZEO restart
Hi all,
We're working on a large scale Plone deployment and recently came across
some odd behavior with tempstorage at our ZEO server. A limited number, yet
highly frequented, of pages on our Plone site query backend web services
that provide location based services such as geocoding and routing (a to b).
Since those calls to the services are rather computation intense we decided
to store intermediate and resulting data in sessions vs. re-querying the
service for no reason.
The system setup consists of a central ZEO server that provides the storage
for content as well as a temp storage for sessioning. There are multiple
other servers that run multiple Plone instances with ZEO clients. The ZEO
server setup is straight forward but I'm happy to provide the configuration
files. Generally the concept works great.
However, whenever we restart the ZEO server and keep the client instances
running we get odd behavior with the client instances' sessioning. Every
client appears to reconnect and, if required, revalidates its cache.
2010-07-11T14:46:03 INFO ZEO.ClientStorage (6132) Disconnected from storage:
"('ITIS.mentzdv.de', 8100)"
2010-07-11T14:46:03 INFO ZEO.ClientStorage (6132) Disconnected from storage:
"('ITIS.mentzdv.de', 8100)"
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Testing connection
2010-07-11T14:46:09 INFO ZEO.zrpc.Connection(C) (127.0.0.1:8100) received
handshake 'Z303'
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Server authentication
protocol None
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Connected to storage: ('
ITIS.mentzdv.de', 8100)
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) Testing connection
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) last inval tid:
'\x03\x87n\x9d|\x1a\xc2\xaa' 2010-07-11 04:45:29.087000
2010-07-11T14:46:09 INFO ZEO.ClientStorage (6132) last transaction:
'\x00\x00\x00\x00\x00\x00\x00\x00' 1900-01-01 00:00:00.00
2010-07-11T14:46:09 INFO ZEO.zrpc.Connection(C) (127.0.0.1:8100) received
handshake 'Z303'
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) Verifying cache
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) Server authentication
protocol None
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) Connected to storage: ('
ITIS.mentzdv.de', 8100)
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) No verification necessary
(last_inval_tid up-to-date)
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) endVerify finishing
2010-07-11T14:46:10 INFO ZEO.ClientStorage (6132) endVerify finished
The problem is that whenever we hit a session bound page we get following
error and stack trace:
2010-07-11T14:58:12 ERROR ZODB.Connection Couldn't load state for 0x82
Traceback (most recent call last):
File
"g:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZODB\Connection.py", line
761, in setstate
self._setstate(obj)
File
"g:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZODB\Connection.py", line
801, in _setstate
p, serial = self._storage.load(obj._p_oid, self._version)
File
"G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\ClientStorage.py",
line 727, in load
return self.loadEx(oid, version)[:2]
File
"G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\ClientStorage.py",
line 750, in loadEx
data, tid, ver = self._server.loadEx(oid, version)
File
"G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\ServerStub.py", line
196, in loadEx
return self.rpc.call("loadEx", oid, version)
File
"G:\mentzdv\plone\instances\zeo04\zope2\lib\python\ZEO\zrpc\connection.py",
line 650, in call
raise inst # error raised by server
KeyError: '\x00\x00\x00\x00\x00\x00\x00\x82'
In the ZMI under the Control-Panel/Database Tab the temporary database seems
to be fine. The /temp_folder returns a similar stack trace though:
Site Error
An error was encountered while publishing this resource.
*KeyError*
Sorry, a site error occurred.
Traceback (innermost last):
- Module ZPublisher.Publish, line 202, in publish_module_standard
- Module ZPublisher.Publish, line 150, in publish
- Module plone.app.linkintegrity.monkey, line 21, in
zpublisher_exception_hook_wrapper
- Module Zope2.App.startup, line 221, in zpublisher_exception_hook
- Module ZPublisher.Publish, line 119, in publish
- Module ZPublisher.mapply, line 88, in mapply
- Module ZPublisher.Publish, line 42, in call_object
- Module App.special_dtml, line 65, in __call__
- Module DocumentTemplate.DT_String, line 476, in __call__
- Module Products.Transience.Transience, line 926, in housekeep
- Module Products.Transience.Transience, line 929, in _housekeep
- Module Products.Transience.Transience, line 569, in _finalize
- Module Products.Transience.Transience, line 593, in _do_finalize_work
- Module ZODB.Connection, line 761, in setstate
- Module ZODB.Connection, line 801, in _setstate
- Module ZEO.ClientStorage, line 727, in load
- Module ZEO.ClientStorage, line 750, in loadEx
- Module ZEO.ServerStub, line 196, in loadEx
[Zope-dev] access to download.zope.org (for bluebream)
Hi, I would like to release Bluebream 1.0b3. May I have scp access to download.zope.org in order to upload the new version file? regards, Christophe ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] z3c.form: Problem with DictionaryField datamanger
On 11 July 2010 01:56, Michael Howitz wrote: > Hi Wichert, > > In Revision r112225 (see [1]) you make sure that the DictionaryField data > manager conforms to the interface by raising an exception in > datamanager.get() when a requested key is not found in the underlying > dictionary. > > Although it is nice to conform to the interface, this changes breaks code in > a project of mine: z3c.form.form.applyChanges uses datamanager.get() to check > whether the value has changed. Before r112225 this call returned the > field.missing_value when the requested key does not exist. Now it raises an > AttributeError, so applyChanges cannot continue. > > I'm not sure whether it was intended that now all keys must exist in the > dictionary. I think this should be consistent with AttributeField. This does: return getattr(self.adapted_context, self.field.__name__) which would indeed raise an AttributeError if the field doesn't exist. > In r114527 I prepared a change on a branch which uses datamanager.query() get > the old value for comparison which allows that keys can be missing again in > the dictionary. This seems like the use case query() was designed for... > I'd like to merge this change (it breaks no tests) into the trunk. Any > objections? +1 for this change Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] z3c.form: Problem with DictionaryField datamanger
Hi Wichert, In Revision r112225 (see [1]) you make sure that the DictionaryField data manager conforms to the interface by raising an exception in datamanager.get() when a requested key is not found in the underlying dictionary. Although it is nice to conform to the interface, this changes breaks code in a project of mine: z3c.form.form.applyChanges uses datamanager.get() to check whether the value has changed. Before r112225 this call returned the field.missing_value when the requested key does not exist. Now it raises an AttributeError, so applyChanges cannot continue. I'm not sure whether it was intended that now all keys must exist in the dictionary. In r114527 I prepared a change on a branch which uses datamanager.query() get the old value for comparison which allows that keys can be missing again in the dictionary. I'd like to merge this change (it breaks no tests) into the trunk. Any objections? [1] .. http://svn.zope.org/?rev=112225&view=rev [2] .. http://svn.zope.org/?view=rev&rev=114527 Yours sincerely, -- Michael Howitz · m...@gocept.com · software developer gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 67 OK, 16 Failed
Summary of messages to the zope-tests list. Period Fri Jul 9 12:00:00 2010 UTC to Sat Jul 10 12:00:00 2010 UTC. There were 83 messages: 6 from Zope Tests, 4 from buildbot at pov.lt, 17 from buildbot at winbot.zope.org, 11 from ccomb at free.fr, 1 from ct at gocept.com, 44 from jdriessen at thehealthagency.com. Test failures - Subject: FAILED : Zope Buildbot / ztk_win slave-win From: jdriessen at thehealthagency.com Date: Fri Jul 9 08:53:03 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016319.html Subject: FAILED : Zope Buildbot / ztk_win slave-win From: jdriessen at thehealthagency.com Date: Fri Jul 9 09:53:02 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016322.html Subject: FAILED : Zope Buildbot / ztk_win slave-win From: jdriessen at thehealthagency.com Date: Fri Jul 9 11:53:04 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016325.html Subject: FAILED : Zope Buildbot / ztk_win slave-win From: jdriessen at thehealthagency.com Date: Fri Jul 9 13:53:02 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016328.html Subject: FAILED : Zope Buildbot / ztk_win slave-win From: jdriessen at thehealthagency.com Date: Fri Jul 9 14:53:02 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016333.html Subject: FAILED : Zope Buildbot / zope2.12 slave-ubuntu64 From: jdriessen at thehealthagency.com Date: Fri Jul 9 16:57:00 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016342.html Subject: FAILED : Zope Buildbot / zope2.12 slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Fri Jul 9 16:57:18 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016343.html Subject: FAILED: Repository policy check found errors in 415 projects From: ct at gocept.com Date: Fri Jul 9 21:13:40 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016349.html Subject: FAILED : winbot / ztk_dev py_270_win32 From: buildbot at winbot.zope.org Date: Fri Jul 9 22:59:53 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016366.html Subject: FAILED : winbot / ztk_dev py_270_win64 From: buildbot at winbot.zope.org Date: Fri Jul 9 23:11:05 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016367.html Subject: FAILED : winbot / ztk_10 py_244_win32 From: buildbot at winbot.zope.org Date: Fri Jul 9 23:19:16 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016368.html Subject: FAILED : winbot / ztk_10 py_270_win32 From: buildbot at winbot.zope.org Date: Fri Jul 9 23:48:28 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016372.html Subject: FAILED : winbot / ztk_10 py_270_win64 From: buildbot at winbot.zope.org Date: Fri Jul 9 23:55:23 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016373.html Subject: FAILED : winbot / ZODB_dev py_270_win32 From: buildbot at winbot.zope.org Date: Sat Jul 10 03:36:50 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016385.html Subject: FAILED : winbot / ZODB_dev py_270_win64 From: buildbot at winbot.zope.org Date: Sat Jul 10 04:32:31 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016386.html Subject: FAILED : Zope Buildbot / ztk_win slave-win From: jdriessen at thehealthagency.com Date: Sat Jul 10 07:17:36 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016391.html Tests passed OK --- Subject: OK : Zope Buildbot / ztk slave-ubuntu64 From: jdriessen at thehealthagency.com Date: Fri Jul 9 08:58:01 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016320.html Subject: OK : Zope Buildbot / ztk slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Fri Jul 9 08:59:19 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016321.html Subject: OK : Zope Buildbot / ztk slave-ubuntu64 From: jdriessen at thehealthagency.com Date: Fri Jul 9 09:57:59 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016323.html Subject: OK : Zope Buildbot / ztk slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Fri Jul 9 09:59:18 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016324.html Subject: OK : Zope Buildbot / ztk slave-ubuntu64 From: jdriessen at thehealthagency.com Date: Fri Jul 9 11:57:59 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016326.html Subject: OK : Zope Buildbot / ztk slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Fri Jul 9 11:59:19 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016327.html Subject: OK : Zope Buildbot / ztk slave-ubuntu64 From: jdriessen at thehealthagency.com Date: Fri Jul 9 13:58:39 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/016329.html Subject: OK : Zope Buildbot / ztk slave-ubuntu32 From: jdriessen at thehealthagency.com Date: Fri Jul 9 14:00:00 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-July/01
Re: [Zope-dev] Zope Toolkit - 1.0a1 release
Le 06/07/2010 16:38, Jan-Wijbrand Kolman a écrit : > On 6/30/10 21:47 PM, Hanno Schlichting wrote: >> To use the ZTK release, you can use: >> >> [buildout] >> extends = >> http://download.zope.org/zopetoolkit/index/1.0a1/ztk-versions.cfg >> http://download.zope.org/zopetoolkit/index/1.0a1/zopeapp-versions.cfg >> >> This first release focuses on getting something out and defining the >> actual technical outcome of a release. Currently the release outcome >> is tailored to the needs of the three frameworks. > > Grok's groktoolkit now extends from this 1.0a1 release. After today's > zc.catlog bugfix release, Grok is using the ZTK now without any issues. Same thing for BlueBream: the next version 1.0b3 will be using the ZTK 1.0a2. And There is no bbkit anymore. Eveything is in the bluebream package. Christophe > > Thanks Hanno! > > regards, jw > > ___ > Zope-Dev maillist - Zope-Dev@zope.org > https://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > https://mail.zope.org/mailman/listinfo/zope-announce > https://mail.zope.org/mailman/listinfo/zope ) > > ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 10 July 2010 18:16, Hanno Schlichting wrote: > On Sat, Jul 10, 2010 at 12:14 PM, Martin Aspeli > wrote: >> Fixed in r114488 (2.12 branch) and r114490 (trunk). I don't think I'm >> allowed to close the issue on Launchpad, but it should be fine now. > > Awesome! You truly rock! My powers of copy and paste and sharp, it must be said. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On Sat, Jul 10, 2010 at 12:14 PM, Martin Aspeli wrote: > Fixed in r114488 (2.12 branch) and r114490 (trunk). I don't think I'm > allowed to close the issue on Launchpad, but it should be fine now. Awesome! You truly rock! Much appreciated, Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [zope2] Help needed with security checks and add views
On 27 June 2010 00:24, Hanno Schlichting wrote:
> Hi there,
>
> recently MJ opened a security related bug and disclosed it to the
> public at https://bugs.launchpad.net/zope2/+bug/578326.
>
> In short Zope 2 never supported the permission attribute on ZCML
> browser:view declarations. It seems some people might have specified
> this attribute and assumed it would do something.
>
> I have added a warning message to Zope 2 (trunk + 2.12 branch) which
> warns about those cases. This is similar to how we handle other such
> cases like the unsupported and set_attributes="..." /> on class directives.
>
> But it turns out that Zope 2 itself is using this in one place, that
> looks like it ought to have a security declaration. The
> Products.Five.adding.ContentAdding class registered as an add view
> ("+") has no working security declarations I can see, and only has
> such a non-functioning permission="zope2.ViewManagementScreens" set.
> I'm not familiar enough with the add view concept to understand what
> this is doing. It also looks like both CMF and Plone use similar
> registrations for their add views.
>
> Ideally I'd love to add support for the permission attribute, as
> clearly people have been using it. But if there's nobody who can
> figure out how to do that, I'd at least like to clarify the add view
> case.
Fixed in r114488 (2.12 branch) and r114490 (trunk). I don't think I'm
allowed to close the issue on Launchpad, but it should be fine now.
Cheers,
Martin
___
Zope-Dev maillist - Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )
