I am forwarding this message to ZODB-dev, as it is more appropriate there.
----- Original Message ----- From: "Chris Kratz" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Wednesday, December 05, 2001 6:17 PM Subject: [Zope-dev] corrupted data.fs > Hello all, > > My apologies if this is not appropriate on this list and my apologies for > this "book". But, we have been having more and more problems with Zope and > it is getting quite frustrating. I thought since I had posted on the zope > list and no one had any suggestions that I would try this list next. > > The behavior we are seeing is as follows. We are using versions heavily. > There are two of us primary developers, soon to be three working heavily > within the same data.fs. We use versions, often running several at the same > time. When the code is ready, we save the version incorporating the changes > to the live site. > > Ever since we upgraded to Zope 2.4.1, saving the version rarely actually does > that. It will save some objects, but usually only a few. We have to then go > to each object and save it again (sometimes making changes) and continue > saving the version until we get all the locked objects unlocked and > committed. This is very annoying, but not a show stopper. What is starting > to scare me is that I wonder if this problem with the version commits is > somehow causing the corruption of the database. > > Now, I say all this because it may or may not be related to the other > problems we continue to have. We periodically, get folders that are no > longer visible within the ZMI and produce a keyerror in filestorage.py > (traceback at end of msg). But, the same folder is still happily being > served externally (ie it generates the web pages based within that folder > without error to our customers). This tells me that it is still in the ZODB, > it is just unreachable via the ZODB. Since we cannot access it from > within zope any longer, we have to load the data.fs from backup and restore > the offending folder losing any work done since the previous night. It never > seems to effect any other parts of the website and it has happened in several > different areas. This is becoming more and more of a showstopper especially > when we lose code. > > 1. I have packed the database. > 2. I have run fsrecover.py over the data.fs. - Nothing to recover. So, if we > aren't corrupted, why can we no longer work with that folder? > 3. I have exported and imported the offending folder without effect (XML and > regular). > 4. We were packing the zope database nightly via a cron job to control ZODB > bloat. I have turned that off for now on the theory that somehow mixing > overnight packs with versions is causing problems. > > Help!!! This continues to happen and it is really scary when your database > continues to get corrupted underneath you on a live site when you are in > heavy development. Can anyone give any ideas on how to track down this > problem? Could it be with the versioning code? Is there a better way to > work with a production site and still get the benefits of the versioning? Is > versioning just dangerous and shouldn't be used? Pardon my tone, but I am a > little frustrated. > > I will say that the last time this happened, thankfully, that particular > folder was being worked on in a version and even though the folder was > inaccessible outside of the version, when working within the version, it was > still there. So, when that version was cut in, we were again able to work > with that folder. This time however, the folder in question wasn't begin > worked on in a version. > > Thank you for your time, > > Sincerely > > -Chris > > ------------- > > **Traceback** > > > Error Type: KeyError > Error Value: ?f > > Traceback > Traceback (innermost last): File > /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZPublisher/Publish .py, > line 171, in publish File > /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZPublisher/mapply. py, > line 160, in mapply (Object: manage_main) File > /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZPublisher/Publish .py, > line 112, in call_object (Object: manage_main) File > /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/Shared/DC/Scripts/ Bindings.py, > line 324, in __call__ (Object: manage_main) File > /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/Shared/DC/Scripts/ Bindings.py, > line 354, in _bindAndExec (Object: manage_main) File > /usr/local/zope/lib/python/App/special_dtml.py, line 244, in _exec (Object: > manage_main) File /usr/local/zope/lib/python/DocumentTemplate/DT_In.py, line > 711, in renderwob (Object: objectItems) File > /usr/local/zope/lib/python/DocumentTemplate/DT_In.py, line 839, in > sort_sequence (Object: objectItems) File > /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZODB/Connection.py , line > 519, in setstate File lib/python/ZODB/FileStorage.py, line 588, in load > (Object: /usr/local/zope/var/Data.fs) File lib/python/ZODB/FileStorage.py, > line 564, in _load (Object: /usr/local/zope/var/Data.fs) KeyError: ?f > > > ---------------- > **Instalation Info** > > Platform: RH 7.2 > Zope Version: (Zope 2.4.1 (binary release, python 2.1, linux2-x86), python > 2.1.1, linux2) > Python Version: 2.1.1 (#1, Aug 8 2001, 11:57:42) [GCC 2.96 20000731 (Red Hat > Linux 7.1 2.96-81)] > System Platform: linux2 > > Products installed: > > eCrumbler (Installed product CookieCrumbler (CookieCrumbler-0.3)) > ExternalMethod (Installed product ExternalMethod (External Method-1-0-0)) > MIMETools (Installed product MIMETools) > MailHost (Installed product MailHost (MailHost-1-2-0)) > OFSP (Installed product OFSP (OFSP-1-0-0)) > PlugIns (Installed product PlugIns (PlugIns-0-4-3p1)) > PluginIndexes (Installed product PluginIndexes) > PythonScripts (Installed product PythonScripts (PythonScripts-2-0-0)) > Renderable (Installed product Renderable) > SiteAccess (Installed product SiteAccess (SiteAccess-2-0-0)) > Squishdot (Installed product Squishdot (Squishdot-1-2-1)) > StandardCacheManagers (Installed product StandardCacheManagers > (StandardCacheManagers-1-1-0)) > ThreadSafeCounter (Installed product ThreadSafeCounter (0.0.3)) > TinyTablePlus (Installed product TinyTablePlus (TinyTablePlus-0-9)) > UpdateSupport (Installed product UpdateSupport (UpdateSupport-1_0_0)) > WarpFramework (Installed product WarpFramework (2001-05-02-1222)) > ZCatalog (Installed product ZCatalog (ZCatalog-2-2-0)) > ZGDChart (Installed product ZGDChart (ZGDChart 0.5.1)) > ZGadflyDA (Installed product ZGadflyDA) > ZPatterns (Installed product ZPatterns (ZPatterns-0-3-0)) > ZPoPyDA (Installed product ZPoPyDA) > ZPsycopgDA (Installed product ZPsycopgDA) > ZSQLMethods (Installed product ZSQLMethods) > ZSyncer (Installed product ZSyncer) > ZWiki (Installed product ZWiki (ZWiki-0-9-6)) > ZopeTutorial (Installed product ZopeTutorial (Zope Tutorial 1.0)) > iTrack (Installed product iTrack (0.2.0)) > > -- > Systems Analyst/Programmer > VistaShare LLC > chris.kratz-no-spam-please-vistashare.com > (replace -no-spam-please- by @) > www.vistashare.com > > _______________________________________________ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > _______________________________________________ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )