[Zope] unpickle error on Data.fs pack

2006-01-25 Thread Gerhard Schmidt 
Hi, 

since three days we have problems when packing the Data.fs. 

2006-01-25T03:40:42 ERROR(200) zrpc:7266 Error raised in delayed method
Traceback (most recent call last):
  File /usr/local/www/Zope/lib/python/ZEO/StorageServer.py, line 991, in run
result = self._method(*self._args)
  File /usr/local/www/Zope/lib/python/ZEO/StorageServer.py, line 315, in 
_pack_impl
self.storage.pack(time, referencesf)
  File /usr/local/www/Zope/lib/python/ZODB/FileStorage.py, line 1582, in pack
opos = p.pack()
  File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 700, in pack
self.gc.findReachable()
  File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 456, in 
findReachable
self.findReachableAtPacktime([z64])
  File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 531, in 
findReachableAtP
acktime
todo.extend(self.findrefs(pos))
  File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 604, in findrefs
return referencesf(self._file.read(dh.plen))
  File /usr/local/www/Zope/lib/python/ZODB/referencesf.py, line 38, in 
referencesf
raise ValueError, 'Error unpickling %r' % p
ValueError: Error unpickling 
'((U\x0eBTrees.OIBTreeq\x01U\x08OIBucketq\x02tq\x03Nt.((
U\x05nchenq\x04J\xc6{a\xfeU\x0fnchen/ottobrunnq\x05J\xbd\xeby\xcfU\x04ndigq\x06J\n\xf
0}QU\x05ndnisq\x07J\xd9\xdc\xbfIU\x02neq\x08J1!\x15\xe9U\x05nebenq\tJT]4\xc0U\x03netq
\nJ\xf3cU\xb6U\x04net/q\x0bJ\nM\xe5\xd6U\x07networkq\x0cJ\xf5\x85!\xe5U\tnetzartigq\r
J\xd4\xf9\x906U\x03neuq\x0eJv\xd7\xe9U\x04neueq\x0fJW\xedD\xd8U\x05neuenq\x10J0!\x0
7U\x05neuesq\x11J\xb9\xa5\xb4sU\x08neuestenq\x12JW2\xcc-U\x07nftigenq\x13J\xd5iU\x0
3ngeq\x14J%\xa9X\x10U\x06ngerenq\x15J\x1d\x14YU\x04ngigq\x16J\xc4\xe6\xe5\xd4U\x06ng
igenq\x17J}\xbd\xffpU\nngigkeitenq\x18J)]\x06IU\x05nichtq\x19J\x0bgyU\x07nkungenq\x1
aJC4\xf7\x10U\x04nnenq\x1bJU\xc4bFU\x04nochq\x1cJ\xb4\xf6\xcdUU\x07norbertq\x1dJ-\xf3
\xd7\x8fU\x06normenq\x1eJ[\x84\xd4\xaeU\x07normungq\x1fJ\xf4\xe9\xfc\xfcU\x08notebook
q 
J\xf7\xf2\x9e\xf9U\x0fnotebookeinsatzq!J`\x8fRiU\tnotebooksqJ\xba\xecvU\x12notebo
okverwendungq#J_R\x10\x9aU\x02nrq$J\xfcSg\xddU\x05nscheq%J-\x88\xf8\xccU\x06nstigeqJ
\xa35\x0e\xcdU\x04nterq\'J\xb1\x94\x9b\xeeU\nnumerischeq(Jf\\n\xfeQ\x03nurq)J\xd2\x95
C\xf4U\x0cnutzbringendq*J\xd3\x84\x84\xeaU\x06nutzenq+J\xa7^\x86IU\tnutzungsmq,J\xc1\
xca\xb9LU\x02obq-J\xbe\xb7e\x94U\x06objectq.J\xaaP\x14\xf9U\x04oderq/J\\0\xc2(U\x05od
imaq0J\xcd\xf9\x0f:t(U\x08\x00\x00\x00\x00\x00%\xa5\xdaq1(U\x0eBTrees.OIBTreeq2U\x08O
IBucketq3ttq4Qtq5.'
--

I've tried to recover the data.fs with fsrecover but it returns 
without error and the error remains. fsrefs.py terminates with 
an error. Any idea how to fix the Data.fs. 

The System is still up an running and no error shown so far. 

Bye
Estartu


Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | EMail: [EMAIL PROTECTED]  |  on request 
Germany||  



pgpCXh1QIuAmp.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )

Re: [Zope] unpickle error on Data.fs pack

2006-01-25 Thread Gerhard Schmidt 
On Wed, Jan 25, 2006 at 12:52:10PM +0100, Gerhard Schmidt wrote:
 I've tried to recover the data.fs with fsrecover but it returns 
 without error and the error remains. fsrefs.py terminates with 
 an error. Any idea how to fix the Data.fs. 

fsrefs.py returns the following.

Traceback (most recent call last):
  File /usr/local/www/Zope/bin/fsrefs.py, line 189, in ?
main(path)
  File /usr/local/www/Zope/bin/fsrefs.py, line 161, in main
refs = get_refs(data)
  File /usr/local/www/Zope/bin/fsrefs.py, line 105, in get_refs
u.noload() # instance state info
cPickle.UnpicklingError: invalid load key, ''.

Thats all. 

The Data.fs is about 3Gig. 

Bye
Estartu

-
Gerhard Schmidt   | E-Mail: [EMAIL PROTECTED]
TU-München|
WWW  Online Services |
Tel: 089/289-25270|
Fax: 089/289-25257| PGP-Publickey auf Anfrage 



pgphjjYxUYG04.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )

Re: [Zope] unpickle error on Data.fs pack

2006-01-25 Thread Dieter Maurer 
Gerhard Schmidt wrote at 2006-1-25 12:52 +0100:
since three days we have problems when packing the Data.fs. 

2006-01-25T03:40:42 ERROR(200) zrpc:7266 Error raised in delayed method
Traceback (most recent call last):
 ...
  File /usr/local/www/Zope/lib/python/ZODB/referencesf.py, line 38, in 
 referencesf
raise ValueError, 'Error unpickling %r' % p
ValueError: Error unpickling 
'((U\x0eBTrees.OIBTreeq\x01U\x08OIBucketq\x02tq\x03Nt.((
U\x05nchenq\x04J\xc6{a\xfeU\x0fnchen/ottobrunnq\x05J\xbd\xeby\xcfU\x04ndigq\x06J\n\xf
0}QU\x05ndnisq\x07J\xd9\xdc\xbfIU\x02neq\x08J1!\x15\xe9U\x05nebenq\tJT]4\xc0U\x03netq
\nJ\xf3cU\xb6U\x04net/q\x0bJ\nM\xe5\xd6U\x07networkq\x0cJ\xf5\x85!\xe5U\tnetzartigq\r
J\xd4\xf9\x906U\x03neuq\x0eJv\xd7\xe9U\x04neueq\x0fJW\xedD\xd8U\x05neuenq\x10J0!\x0
7U\x05neuesq\x11J\xb9\xa5\xb4sU\x08neuestenq\x12JW2\xcc-U\x07nftigenq\x13J\xd5iU\x0
3ngeq\x14J%\xa9X\x10U\x06ngerenq\x15J\x1d\x14YU\x04ngigq\x16J\xc4\xe6\xe5\xd4U\x06ng
igenq\x17J}\xbd\xffpU\nngigkeitenq\x18J)]\x06IU\x05nichtq\x19J\x0bgyU\x07nkungenq\x1
aJC4\xf7\x10U\x04nnenq\x1bJU\xc4bFU\x04nochq\x1cJ\xb4\xf6\xcdUU\x07norbertq\x1dJ-\xf3
\xd7\x8fU\x06normenq\x1eJ[\x84\xd4\xaeU\x07normungq\x1fJ\xf4\xe9\xfc\xfcU\x08notebook
q 
J\xf7\xf2\x9e\xf9U\x0fnotebookeinsatzq!J`\x8fRiU\tnotebooksqJ\xba\xecvU\x12notebo
okverwendungq#J_R\x10\x9aU\x02nrq$J\xfcSg\xddU\x05nscheq%J-\x88\xf8\xccU\x06nstigeqJ
\xa35\x0e\xcdU\x04nterq\'J\xb1\x94\x9b\xeeU\nnumerischeq(Jf\\n\xfeQ\x03nurq)J\xd2\x95
C\xf4U\x0cnutzbringendq*J\xd3\x84\x84\xeaU\x06nutzenq+J\xa7^\x86IU\tnutzungsmq,J\xc1\
xca\xb9LU\x02obq-J\xbe\xb7e\x94U\x06objectq.J\xaaP\x14\xf9U\x04oderq/J\\0\xc2(U\x05od
imaq0J\xcd\xf9\x0f:t(U\x08\x00\x00\x00\x00\x00%\xa5\xdaq1(U\x0eBTrees.OIBTreeq2U\x08O
IBucketq3ttq4Qtq5.'
--

I've tried to recover the data.fs with fsrecover but it returns 
without error and the error remains. fsrefs.py terminates with 
an error. Any idea how to fix the Data.fs. 

Looks like something corrupted your file.

Python 2.3 and above contain a module pickleutils (or similar).
You can use this module to analyse you pickle and (maybe) find
how you can fix the object.

-- 
Dieter
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )

Re: [Zope] unpickle error on Data.fs pack

2006-01-25 Thread Tim Peters 
[Gerhard Schmidt]
 since three days we have problems when packing the Data.fs.

 2006-01-25T03:40:42 ERROR(200) zrpc:7266 Error raised in delayed method
 Traceback (most recent call last):
   File /usr/local/www/Zope/lib/python/ZEO/StorageServer.py, line 991, in run
 result = self._method(*self._args)
   File /usr/local/www/Zope/lib/python/ZEO/StorageServer.py, line 315, in 
 _pack_impl
 self.storage.pack(time, referencesf)
   File /usr/local/www/Zope/lib/python/ZODB/FileStorage.py, line 1582, in 
 pack
 opos = p.pack()
   File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 700, in pack
 self.gc.findReachable()
   File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 456, in 
 findReachable
 self.findReachableAtPacktime([z64])
   File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 531, in 
 findReachableAtP
 acktime
 todo.extend(self.findrefs(pos))
   File /usr/local/www/Zope/lib/python/ZODB/fspack.py, line 604, in findrefs
 return referencesf(self._file.read(dh.plen))
   File /usr/local/www/Zope/lib/python/ZODB/referencesf.py, line 38, in 
 referencesf
 raise ValueError, 'Error unpickling %r' % p
 ValueError: Error unpickling 
 '((U\x0eBTrees.OIBTreeq\x01U\x08OIBucketq\x02tq\x03Nt.((
 U\x05nchenq\x04J\xc6{a\xfeU\x0fnchen/ottobrunnq\x05J\xbd\xeby\xcfU\x04ndigq\x06J\n\xf
 0}QU\x05ndnisq\x07J\xd9\xdc\xbfIU\x02neq\x08J1!\x15\xe9U\x05nebenq\tJT]4\xc0U\x03netq
 \nJ\xf3cU\xb6U\x04net/q\x0bJ\nM\xe5\xd6U\x07networkq\x0cJ\xf5\x85!\xe5U\tnetzartigq\r
 J\xd4\xf9\x906U\x03neuq\x0eJv\xd7\xe9U\x04neueq\x0fJW\xedD\xd8U\x05neuenq\x10J0!\x0
 7U\x05neuesq\x11J\xb9\xa5\xb4sU\x08neuestenq\x12JW2\xcc-U\x07nftigenq\x13J\xd5iU\x0
 3ngeq\x14J%\xa9X\x10U\x06ngerenq\x15J\x1d\x14YU\x04ngigq\x16J\xc4\xe6\xe5\xd4U\x06ng
 igenq\x17J}\xbd\xffpU\nngigkeitenq\x18J)]\x06IU\x05nichtq\x19J\x0bgyU\x07nkungenq\x1
 aJC4\xf7\x10U\x04nnenq\x1bJU\xc4bFU\x04nochq\x1cJ\xb4\xf6\xcdUU\x07norbertq\x1dJ-\xf3
 \xd7\x8fU\x06normenq\x1eJ[\x84\xd4\xaeU\x07normungq\x1fJ\xf4\xe9\xfc\xfcU\x08notebook
 q 
 J\xf7\xf2\x9e\xf9U\x0fnotebookeinsatzq!J`\x8fRiU\tnotebooksqJ\xba\xecvU\x12notebo
 okverwendungq#J_R\x10\x9aU\x02nrq$J\xfcSg\xddU\x05nscheq%J-\x88\xf8\xccU\x06nstigeqJ
 \xa35\x0e\xcdU\x04nterq\'J\xb1\x94\x9b\xeeU\nnumerischeq(Jf\\n\xfeQ

Not that it will help much, but the \x03 on the start of the next line
is the immediate cause of the error (it's in a _position_ where a
pickle opcode is expected, but 0x03 isn't a legitimate pickle
opcode).

 \x03nurq)J\xd2\x95
 C\xf4U\x0cnutzbringendq*J\xd3\x84\x84\xeaU\x06nutzenq+J\xa7^\x86IU\tnutzungsmq,J\xc1\
 xca\xb9LU\x02obq-J\xbe\xb7e\x94U\x06objectq.J\xaaP\x14\xf9U\x04oderq/J\\0\xc2(U\x05od
 imaq0J\xcd\xf9\x0f:t(U\x08\x00\x00\x00\x00\x00%\xa5\xdaq1(U\x0eBTrees.OIBTreeq2U\x08O
 IBucketq3ttq4Qtq5.'
 --

 I've tried to recover the data.fs with fsrecover but it returns
 without error and the error remains.

This is corruption _inside_ an object pickle.  fsrecover can't do
anything about that.  See

http://zope.org/Wikis/ZODB/FileStorageBackup

for background info.

 fsrefs.py terminates with an error.

Yes, fsrefs can't load the pickle either.

 Any idea how to fix the Data.fs.

Sorry, not easily, no.  You need to exploit your application-specific
knowledge about the importance of this specific OIBucket, and couple
that with fiddly knowledge about how pickle works.

 The System is still up an running and no error shown so far.

Maybe you could locate the object in question from an fsdump (see the
link above), and delete the whole object.
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )