I'm hunting down an arcane problem in one of our Zope sites and find
myself hitting new and wildly diverse problems every day. I hope
somebody can point me in the direction of some solution strategies.
Main suspect is a corrupted Data.fs.in file. How can I audit and
repair it?
problem A: access denied vs. KeyError
For starters, we built a website with a lot of custom programming. I
created the NFGnav product
http://www.zope.org/Members/gyst/NFGnav
and my partner built a collection of ZClasses.
All this was Tested And Working Fine (TM).
Since some weeks, part of the site gives an 'access denied' error. We
had to shut down that subtree. Strange thing is, it also gives access
denied for the manager/owner logins of the site - that's me and I
really have access to everything. What is even stranger, is that when
I logon as superuser, I get a KeyError instead of an access denied
error.
=> Question: which parts of zope would be prone to denying access to a
privileged user, while generating KeyErrors for the superuser?
problem B: hanging zope thread zombies all zope sites
While debugging this problem, I found a strange correlation with
another problem we're having: zope threads consuming 99.9% CPU. Now
and again, a zope thread hangs. All zope sites are dead. I have to not
only restart zope; I have to stop zope, restart apache, and start zope
again to get rid of the hanging thread and get the zope sites alive
again.
As soon as I start digging around in the buggy site, the number of
thread crashes jumps up. In combination with the problems we're having
with that site itself, this suggests two possible causes:
1) ZODB curruption. We've had a complete server crash 3 weeks ago and
it is quite possible the ZODB isn't 100% healthy. In my zope
installation I found a utility 'bbb.py' to audit data storage
files. Alas, it is of no use:
| zeno:/usr/lib/zope/utilities# python bbb.py
|/var/lib/zope/import/www.intratuin.nl.zexp
|
| The input file is not a ZODB 2 database file.
| This script only works with ZODB 2 (BoboPOS) data or export files.
=> Question: is there another utility for checking and repairing
object stores?
2) Another possible cause would be an infinite loop in one of our
custom scripts. To try and debug this I wanted to import a copy of the
offending site into one of our testlab machines. However, on importing
the zexp file I got:
Error Type: TypeError
Error Value: ('len() of unsized object', , None)
