Re: [ZODB-Dev] Recovering from BTree corruption

[Jim Fulton]

On Sep 10, 2007, at 10:34 AM, Alan Runyan wrote:

Hi guys.

It seems that one of our customers has a corrupted BTree. I would love
for someone to provide some insight on how we can recover the data.

we have two databases: 1 for resources and 1 for 'content'.  resources
contain lots of very big files.

The system is configured to have a mount point at /plone/resources is
a subclass of BTreeFolder, using internal data struct of OOBTree.

And, as you said in another node, the BTree folder actually loves in  
the resources database.

anytime I iterate over the keys I get POSKeyError.  anytime I iterate
over the values the same.  if I run BTree.check() on the data
structure's tree attribute (the OOBTree itself) I get a POSKeyError.

Running the utils.checkbtrees doesnt say this btree has a problem.

While debugging this I had a conversation with sidnei about mounted
databases.  He recalled that if your using a mounted database you
should not pack.  If for some reason your mounted database had a cross
reference to another database and somehow you had a dangling reference
to the other database it would cause POSKeyError.

Cross database references are inherently weak.  A reference from a  
foreign database doesn't prevent an object from being treated as  
garbage.  So, if the only reference to an object is from a foreign  
database, then the object is considered garbage.  It doesn't sound  
like this is what's affecting you.  The cross-database reference is  
to the BTree.  It sounds like the internal references are within  
database.



Is there any other ways of "testing consistency" of FileStorage  
other than:
  - checkbtrees.py
  - fstest.py

There's an fsrefs script that checks internal references I believe.

And any ideas how I can salvage the data? This BTree, of course, had
the most valuable data.

Possibly, there's a backup that has data records for the missing OIDs.

Jim

--
Jim Fulton                      [EMAIL PROTECTED]                Python 
Powered!
CTO                             (540) 361-1714                  
http://www.python.org
Zope Corporation        http://www.zope.com             http://www.zope.org



_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev