Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Pedro Ferreira
Tres Seaver wrote: > Pedro Ferreira wrote: > > > Thanks a lot for your help. In fact, it was a matter of increasing the > > maximum recursion limit. > > There's still an unsolved issue, though. Each time we try to recover a > > backup using repozo, we get a CRC error. Is this normal? Has it happene

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Pedro Ferreira
Jim Fulton wrote: > > On May 25, 2009, at 9:23 AM, Pedro Ferreira wrote: > >> Dear Andreas, Marius, >> >>> This means that you're using ZEO, right? Have you tried to use strace >>> to see what it's doing? Is it using any CPU time? >>> >>> >> Yes, we're using ZEO. >> It's doing a lot of lseek() an

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Marius Gedminas
On Tue, May 26, 2009 at 12:00:51PM +0200, Pedro Ferreira wrote: > Jim Fulton wrote: > > What was in the ZEO server log when this happened? > > > 2009-05-24T12:22:54 (28965) new connection ('137.138.128.213', 45138): > > 2009-05-24T12:22:54 (28965) new connection ('137.138.128.213', 45139): > > 20

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Jim Fulton
On May 26, 2009, at 6:00 AM, Pedro Ferreira wrote: > Jim Fulton wrote: >> >> What was in the ZEO server log when this happened? >> > 2009-05-24T12:22:54 (28965) new connection ('137.138.128.213', 45138): > > 2009-05-24T12:22:54 (28965) new connection ('137.138.128.213', 45139): > > 2009-05-24T1

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Pedro Ferreira
> > That's what I was afraid of. > > FileStorage indexes can't be saved after they reach a certain size, > where size roughly based on the number of objects. > > I need to find a way to fix this. So, from this I infer that our database has grown in such a proportion that we're reaching some of th

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Adam GROSZER
Hello Jim, Where's that certain size on the scale? Tuesday, May 26, 2009, 3:35:56 PM, you wrote: JF> FileStorage indexes can't be saved after they reach a certain size, JF> where size roughly based on the number of objects. -- Best regards, Adam GROSZERmailto:agr

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Jim Fulton
On May 26, 2009, at 10:16 AM, Pedro Ferreira wrote: > >> >> That's what I was afraid of. >> >> FileStorage indexes can't be saved after they reach a certain size, >> where size roughly based on the number of objects. >> >> I need to find a way to fix this. > So, from this I infer that our databas

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Kazuhiko Shiozaki
Hi, Pedro Ferreira wrote: I've also tried to run the "analyze.py" script, but it returns me a stream of '''type' object is unsubscriptable" errors, due to: classinfo = pickle.loads(record.data)[0] any suggestion? I personally apply the attached patch to analyze.py, that does not load pickle

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Laurence Rowe
Jim Fulton wrote: > On May 26, 2009, at 10:16 AM, Pedro Ferreira wrote: >> In any case, it's not such a surprising number, since we have ~73141 >> event objects and ~344484 contribution objects, plus ~492016 resource >> objects, and then each one of these may contain authors, and fore sure >> som

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Jim Fulton
On May 26, 2009, at 11:17 AM, Laurence Rowe wrote: > Jim Fulton wrote: >> On May 26, 2009, at 10:16 AM, Pedro Ferreira wrote: > >>> In any case, it's not such a surprising number, since we have ~73141 >>> event objects and ~344484 contribution objects, plus ~492016 >>> resource >>> objects, an

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Chris Bainbridge
2009/5/26 Pedro Ferreira : > In any case, it's not such a surprising number, since we have ~73141 > event objects and ~344484 contribution objects, plus ~492016  resource > objects, and then each one of these may contain authors, and fore sure > some associated objects that store different bits of

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Jim Fulton
On May 26, 2009, at 10:21 AM, Adam GROSZER wrote: > Hello Jim, > > Where's that certain size on the scale? > > Tuesday, May 26, 2009, 3:35:56 PM, you wrote: > > JF> FileStorage indexes can't be saved after they reach a certain > size, > JF> where size roughly based on the number of objects. I

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Jim Fulton
Well said. A feature I'd like to add is the ability to have persistent objects that don't get their own database records, so that you can get the benefit of having them track their changes without incuring the expense of a separate database object. Jim On May 26, 2009, at 11:48 AM, Chris B

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Laurence Rowe
Jim Fulton wrote: > Well said. A feature I'd like to add is the ability to have persistent > objects that don't get their own database records, so that you can get > the benefit of having them track their changes without incuring the > expense of a separate database object. +lots Hanno Schl

[ZODB-Dev] ZODB Documentation

2009-05-26 Thread Laurence Rowe
A few weeks ago I converted the "ZODB/ZEO Programming Guide" and a few more articles into structured text and added them to the zope2docs buildout. I've now moved them to their own buildout in svn+ssh://svn.zope.org/repos/main/zodbdocs/trunk and they will soon appear at http://docs.zope.org/zod

Re: [ZODB-Dev] ZODB Documentation

2009-05-26 Thread Jim Fulton
On May 26, 2009, at 12:54 PM, Laurence Rowe wrote: > A few weeks ago I converted the "ZODB/ZEO Programming Guide" and a few > more articles into structured text and added them to the zope2docs > buildout. I've now moved them to their own buildout in > svn+ssh://svn.zope.org/repos/main/zodbdocs/tr

Re: [ZODB-Dev] ZODB Documentation

2009-05-26 Thread Andreas Jung
On 26.05.09 18:54, Laurence Rowe wrote: > A few weeks ago I converted the "ZODB/ZEO Programming Guide" and a few > more articles into structured text and added them to the zope2docs > buildout. I've now moved them to their own buildout in > svn+ssh://svn.zope.org/repos/main/zodbdocs/trunk and th

Re: [ZODB-Dev] ZODB Documentation

2009-05-26 Thread Andreas Jung
On 26.05.09 19:08, Andreas Jung wrote: > On 26.05.09 18:54, Laurence Rowe wrote: > >> A few weeks ago I converted the "ZODB/ZEO Programming Guide" and a few >> more articles into structured text and added them to the zope2docs >> buildout. I've now moved them to their own buildout in >> svn+s

Re: [ZODB-Dev] ZODB Documentation

2009-05-26 Thread Laurence Rowe
Andreas Jung wrote: > On 26.05.09 19:08, Andreas Jung wrote: >> On 26.05.09 18:54, Laurence Rowe wrote: >> >>> A few weeks ago I converted the "ZODB/ZEO Programming Guide" and a few >>> more articles into structured text and added them to the zope2docs >>> buildout. I've now moved them to thei

Re: [ZODB-Dev] ZODB Documentation

2009-05-26 Thread Laurence Rowe
Jim Fulton wrote: > On May 26, 2009, at 12:54 PM, Laurence Rowe wrote: > >> A few weeks ago I converted the "ZODB/ZEO Programming Guide" and a few >> more articles into structured text and added them to the zope2docs >> buildout. I've now moved them to their own buildout in >> svn+ssh://svn.zope.o

Re: [ZODB-Dev] ZODB Documentation

2009-05-26 Thread Alan Runyan
Would it be useful to have a document that starts detailing sections that are missing in ZODB/ZEO programming guide? i.e. - Blobs are not mentioned. - ZEO/Persistent disk cache not mentioned - xdb reference safetybelt - Documentation of various backend storages - Relstorage, Demostorag

Re: [ZODB-Dev] ZODB Documentation

2009-05-26 Thread Jim Fulton
On May 26, 2009, at 1:45 PM, Alan Runyan wrote: > Would it be useful to have a document that starts detailing sections > that are > missing in ZODB/ZEO programming guide? I'd honestly rather start from scratch with something that is more oriented to users. Of course, I'd want it to be compl

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Shane Hathaway
Jim Fulton wrote: > FileStorage indexes can't be saved after they reach a certain size, > where size roughly based on the number of objects. > > I need to find a way to fix this. It might be interesting to use SQLite for FileStorage indexes. With SQLite, we wouldn't have to store the whole in

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Marius Gedminas
On Tue, May 26, 2009 at 04:16:29PM +0200, Pedro Ferreira wrote: > Also, is there any documentation about the basic structures of the > database available? We found some information spread through different > sites, but we couldn't find exhaustive documentation for the API > (information about the d

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Alan Runyan
On Tue, May 26, 2009 at 1:58 PM, Shane Hathaway wrote: > Jim Fulton wrote: >> FileStorage indexes can't be saved after they reach a certain size, >> where size roughly based on the number of objects. >> >> I need to find a way to fix this. > > It might be interesting to use SQLite for FileStorage

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Chris Withers
Laurence Rowe wrote: > Jim Fulton wrote: >> Well said. A feature I'd like to add is the ability to have persistent >> objects that don't get their own database records, so that you can get >> the benefit of having them track their changes without incuring the >> expense of a separate database

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Hanno Schlichting
Chris Withers wrote: > Laurence Rowe wrote: >> Jim Fulton wrote: >>> Well said. A feature I'd like to add is the ability to have persistent >>> objects that don't get their own database records, so that you can get >>> the benefit of having them track their changes without incuring the >>> ex