Re: [ZODB-Dev] RFC: External Garbage Collection
Jim Fulton wrote: > I've posted a new proposal: > >http://wiki.zope.org/ZODB/ExternalGC > > That addresses multi-database garbage collection and can also be > useful in other situations. > > Comments are welcome. I assume this would fix the following bug: http://bugs.launchpad.net/zodb/+bug/192960 ? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RFC: External Garbage Collection
On Nov 6, 2008, at 7:24 PM, Shane Hathaway wrote: > Jim Fulton wrote: >> I've posted a new proposal: >> >> http://wiki.zope.org/ZODB/ExternalGC >> >> That addresses multi-database garbage collection and can also be >> useful in other situations. >> >> Comments are welcome. Absent objections, I may start working on this >> fairly soon. > > I see where you're going with this. The "Sample (naive)" > implementation > would be very expensive with large databases; do you have ideas on how > it might be done more efficiently? Sure. First, you don't need a good set. You can just remove good oids from the starting set, which becomes the bad set. I'd store the oids on disk as a oid->flag mapping, or maybe even as a set. An advantage of making this external is that we can innovate on the external gc independent of the zodb release, although, eventually, we'd include a built-in gc tool. Another bonus is that, in the presence of replication, the analysis phase can be performed against a secondary storage, keeping load off the primary until the final deletion step. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RFC: External Garbage Collection
On Nov 7, 2008, at 1:40 AM, Christian Theune wrote: > On Thu, 2008-11-06 at 17:46 -0500, Jim Fulton wrote: >> I've posted a new proposal: >> >> http://wiki.zope.org/ZODB/ExternalGC >> >> That addresses multi-database garbage collection and can also be >> useful in other situations. >> >> Comments are welcome. Absent objections, I may start working on this >> fairly soon. > > +1 as far as I can see. > > One little thing: I find the interface a bit of a misnomer as all > functions are IMHO useful on their own. The fact that you're going to > instrument them to implement GC might not need to go into that > interface > name. I'm seriously contemplating a ZODB 4. That might be an opportunity collapse a number of the storage interfaces into a new IStorage. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RFC: External Garbage Collection
On Nov 7, 2008, at 3:24 AM, Chris Withers wrote: > Jim Fulton wrote: >> I've posted a new proposal: >> http://wiki.zope.org/ZODB/ExternalGC >> That addresses multi-database garbage collection and can also be >> useful in other situations. >> Comments are welcome. > > I assume this would fix the following bug: > > http://bugs.launchpad.net/zodb/+bug/192960 > > ? Yes, although I don't quite consider this a bug. Note that this is also addresses by the ability to disable GC in the first place as is now provided in ZODB 3.9 and in 3.8 via zc.FileStorage. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RFC: External Garbage Collection
On Fri, 2008-11-07 at 09:15 -0500, Jim Fulton wrote: > On Nov 7, 2008, at 1:40 AM, Christian Theune wrote: > > > On Thu, 2008-11-06 at 17:46 -0500, Jim Fulton wrote: > >> I've posted a new proposal: > >> > >> http://wiki.zope.org/ZODB/ExternalGC > >> > >> That addresses multi-database garbage collection and can also be > >> useful in other situations. > >> > >> Comments are welcome. Absent objections, I may start working on this > >> fairly soon. > > > > +1 as far as I can see. > > > > One little thing: I find the interface a bit of a misnomer as all > > functions are IMHO useful on their own. The fact that you're going to > > instrument them to implement GC might not need to go into that > > interface > > name. > > > Thought so. :) > I'm seriously contemplating a ZODB 4. That might be an opportunity > collapse a number of the storage interfaces into a new IStorage. W00t. -- Christian Theune · [EMAIL PROTECTED] gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development signature.asc Description: This is a digitally signed message part ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
