Re: [ZODB-Dev] Repair corrupted RelStorage database

2010-02-01 Thread Rudá Porto Filgueiras 
On Sat, Jan 30, 2010 at 2:51 PM, Rudá Porto Filgueiras wrote:

> On Sat, Jan 30, 2010 at 6:56 AM, Shane Hathaway wrote:
>
>
>> - Run the MySQL database verification and repair tools.  It's possible
>> that the object you need is there, but MySQL can't find it due to corrupted
>> indexes or similar.
>>
>>
> I will do that, but it must afert 02:00 am. :-)
>

Only to update:

After check and recover all MYISAM tables:  "myisamchk -c *.MYI"  and
"myisamchk -r *.MYI", zodbconvert.py runs fine and read all objects and
trasactions from MySQL and export to FileStorage file with 22GB.

I think it solve these integrity problems, but I will run the migration
process to see what will happen.
Shane, thanks again by your support.

Cheers,

-- 
Rudá Porto Filgueiras
http://python-blog.blogspot.com
http://twitter.com/rudaporto
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Repair corrupted RelStorage database

2010-01-30 Thread Rudá Porto Filgueiras 
On Sat, Jan 30, 2010 at 6:56 AM, Shane Hathaway wrote:

> Rudá Porto Filgueiras wrote:
>
>> How can I repair or verify a RelStorage database?
>> I´m using MySQL Adapter, there is any script like fsrecover.py
>>
>> Follow the failure when RelStorage try to load some objects:
>>
>> 2010-01-30T04:25:14 ERROR ZODB.Connection Couldn't load state for
>> 0x04ef0f59
>> Traceback (most recent call last):
>>  File
>> "/usr/local/zope-agecom/plone/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
>> line 847, in setstate
>>self._setstate(obj)
>>  File
>> "/usr/local/zope-agecom/plone/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
>> line 888, in _setstate
>>p, serial = self._storage.load(obj._p_oid, self._version)
>>  File
>> "/usr/local/zope-agecom/plone/eggs/RelStorage-1.4.0b1-py2.4.egg/relstorage/storage.py",
>> line 432, in load
>>raise POSKeyError(oid)
>> POSKeyError: 0x04ef0f59
>>
>
> To diagnose this, I would:
>
> - Examine the logs in detail.  There should be a log entry giving details
> about that specific POSKeyError.
>


Follow error from november 2009 (there are some repeated (five) errors like
that on the date/time).

550308-2009-11-28T05:31:51 ERROR ZODB.Connection Couldn't load state for
0x03ebd4
550309-Traceback (most recent call last):
550310-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 797, in setstate
550311-self._setstate(obj)
550312-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 855, in _setstate
550313-self._reader.setGhostState(obj, p)
550314-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/serialize.py",
line 604, in setGhostState
550315-state = self.getState(pickle)
550316-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/serialize.py",
line 597, in getState
550317-return unpickler.load()
550318-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/serialize.py",
line 471, in _persistent_load
550319-return self.load_oid(reference)
550320-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/serialize.py",
line 537, in load_oid
550321-return self._conn.get(oid)
550322-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 225, in get
550323-p, serial = self._storage.load(oid, self._version)
550324-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 1196, in load
550325-return self._storage.load(oid, self._base_version)
550326-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZEO/ClientStorage.py",
line 727, in load
550327-return self.loadEx(oid, version)[:2]
550328-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZEO/ClientStorage.py",
line 750, in loadEx
550329-data, tid, ver = self._server.loadEx(oid, version)
550330-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZEO/ServerStub.py",
line 196, in loadEx
550331-return self.rpc.call("loadEx", oid, version)
550332-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.7.3_polling-py2.4-linux-x86_64.egg/ZEO/zrpc/connection.py",
line 650, in call
550333-raise inst # error raised by server
550334:POSKeyError: 0x06294e

This errros is today errors, there are repeated (10) errors like that:

835744:2010-01-30T09:26:47 WARNING relstorage POSKeyError on oid 1793877: no
tid found; Current transaction is 253262639700997905; Recent object tids: []
835745---
835746-2010-01-30T09:26:47 ERROR ZODB.Connection Couldn't load state for
0x1b5f55
835747-Traceback (most recent call last):
835748-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 847, in setstate
835749-self._setstate(obj)
835750-  File
"/usr/local/zope/agecom-virtual/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 888, in _setstate
835751-p, serial = self._storage.load(obj._p_oid, self._version)
835752-  File
"/usr/local/zope/agecom-virtual/eggs/RelStorage-1.4.0b1-py2.4.egg/relstorage/storage.py",
line 432, in load
835753:raise POSKeyError(oid)
835754:POSKeyError: 0x1b5f55
835756-2010-01-30T09:26:48 INFO Archetypes
ESC[00mESC[01;32m/usr/local/zope/agecom-virtual/parts/plone/Archetypes/UIDCatalog.py[106]:getObject
835757-ESC[00mUIDCatalogBrains getObject raised an error:
835758- Traceback (most recent call last):
835759-
835760-  File
"/usr/local/zope/agecom-virtual/parts/plone/Archetypes/UIDCatalog.py", line
87, in getObject
835761-path = self.getPath()
835762-
835763-  File
"/usr/local/zope/agecom-virtual/parts/zope2/lib/python/Products/ZCatalog/CatalogBrains.py",
line 33, in

Re: [ZODB-Dev] Repair corrupted RelStorage database

2010-01-30 Thread Shane Hathaway 
Rudá Porto Filgueiras wrote:
> How can I repair or verify a RelStorage database?
> I´m using MySQL Adapter, there is any script like fsrecover.py
> 
> Follow the failure when RelStorage try to load some objects:
> 
> 2010-01-30T04:25:14 ERROR ZODB.Connection Couldn't load state for 0x04ef0f59
> Traceback (most recent call last):
>   File 
> "/usr/local/zope-agecom/plone/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
>  
> line 847, in setstate
> self._setstate(obj)
>   File 
> "/usr/local/zope-agecom/plone/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
>  
> line 888, in _setstate
> p, serial = self._storage.load(obj._p_oid, self._version)
>   File 
> "/usr/local/zope-agecom/plone/eggs/RelStorage-1.4.0b1-py2.4.egg/relstorage/storage.py",
>  
> line 432, in load
> raise POSKeyError(oid)
> POSKeyError: 0x04ef0f59

To diagnose this, I would:

- Examine the logs in detail.  There should be a log entry giving 
details about that specific POSKeyError.

- Shut down MySQL and back up the database as-is.

- Run the MySQL database verification and repair tools.  It's possible 
that the object you need is there, but MySQL can't find it due to 
corrupted indexes or similar.

- Run a pack in dry-run mode, which will fill the object_refs table so I 
can find out exactly what object(s) refer to that OID (note that 
0x04ef0f59 is 82775897 in decimal).

- Use both SQL and "zopectl debug" to examine the breakage.

Running a pack in dry-run mode performs verification without actually 
deleting anything.  There is no fsrecover-like script, but I could be 
convinced to create one if the need arises.

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

[ZODB-Dev] Repair corrupted RelStorage database

2010-01-29 Thread Rudá Porto Filgueiras 
How can I repair or verify a RelStorage database?
I´m using MySQL Adapter, there is any script like fsrecover.py

Follow the failure when RelStorage try to load some objects:

2010-01-30T04:25:14 ERROR ZODB.Connection Couldn't load state for 0x04ef0f59
Traceback (most recent call last):
  File
"/usr/local/zope-agecom/plone/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 847, in setstate
self._setstate(obj)
  File
"/usr/local/zope-agecom/plone/eggs/ZODB3-3.8.4_polling-py2.4-linux-x86_64.egg/ZODB/Connection.py",
line 888, in _setstate
p, serial = self._storage.load(obj._p_oid, self._version)
  File
"/usr/local/zope-agecom/plone/eggs/RelStorage-1.4.0b1-py2.4.egg/relstorage/storage.py",
line 432, in load
raise POSKeyError(oid)
POSKeyError: 0x04ef0f59

-- 
Rudá Porto Filgueiras
http://python-blog.blogspot.com
http://twitter.com/rudaporto
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev