Tres Seaver wrote:
> Pedro Ferreira wrote:
> > Thanks a lot for your help. In fact, it was a matter of increasing the
> > maximum recursion limit.
> > There's still an unsolved issue, though. Each time we try to recover a
> > backup using repozo, we get a CRC error. Is this normal? Has it happened
> > to anyone?
> I don't recall anything like that. Can you provide a traceback?
pferr...@pcituds01 ~/projects/indico/db $ repozo -Rv -r
/afs/cern.ch/project/indico/db_backups/indico/ -D 2009-05-23-21-31-33 -o
looking for files between last full backup and 2009-05-23-21-31-33...
files needed to recover state as of 2009-05-23-21-31-33:
Recovering file to data.fs
Traceback (most recent call last):
File "/usr/bin/repozo", line 8, in <module>
load_entry_point('ZODB3==3.8.1', 'console_scripts', 'repozo')()
line 513, in main
line 501, in do_recover
reposz, reposum = concat(repofiles, outfp)
line 263, in concat
bytesread += dofile(func, ifp)
line 200, in dofile
data = fp.read(todo)
File "/usr/lib/python2.5/gzip.py", line 227, in read
File "/usr/lib/python2.5/gzip.py", line 292, in _read
File "/usr/lib/python2.5/gzip.py", line 311, in _read_eof
raise IOError, "CRC check failed"
IOError: CRC check failed
> > I guess we have a very large database, for what is normal in ZODB
> > applications.
> Not really: I know of clients whose database routinely grow much larger
> than yours (15 Gb, packing down to 6 Gb, right?)
> > We were wondering if there's any way to optimize the size
> > (and performance) of such a large database, through the removal of
> > unused objects and useless data. We perform packs in a weekly basis, but
> > we're not sure if this is enough, or if there are other ways of
> > lightening up the DB. Any recommendations regarding this point?
> Without knowing anything about the application:
> - Check that it is not holding onto "old" data inappropriately
> (e.g., maintaining lots of "archival" versions of content).
Yes, I think we can do some improvements there. We actually store some
deleted content as a a safeguard, we're considering a major cleanup
> - Check into the catalog / index usage: you may be able to slow
> the growth by batching updates, especially to text indexes.
I'm not sure I understand this one... we're not using ZCatalog, nor
full-text indexes, though...
José Pedro Ferreira
(Software Developer, Indico Project)
Tel. +41 22 76 77159
For more information about ZODB, see the ZODB Wiki:
ZODB-Dev mailing list - ZODB-Dev@zope.org