Op 22-02-11 17:25, Martijn Pieters schreef:
> I was already investigating the possibility to split the RelStorage
> packing process up into smaller chunks.
> Due to the expected load on the Oracle cluster during a pack, we'll
> have to run the pack at night and want to be absolutely certain that
> database is ready for normal site operations again the next day. With
> a 40+GB database (hasn't been packed for it's entire run, more than 2
> years now) we are not confident packing will be done in one night.
I wonder it it may help to set pack-gc to false during the first pack.
According to the docs this is faster, though it of course leaves more
unused objects behind. Set pack-gc to the default true value for
subsequent packs. Theoretically this should make sure that the first
pack will finish on time and leave an already smaller database; after
the second pack the database is at its smallest.
> To at least get a handle on how much work the packing is going to be,
> and to have a nice stopping point, I looked at splitting pre-pack and
> pack operations out into two separate steps. To my delight I saw that
> the 1.5.0 beta already implements basically running only the pre-pack
> phase (the --dry-run option). From there I created the attached patch,
> one that renames the dry-run op into a 'prepack only' option, and adds
> another option to skip the pre-pack and just use whatever is present
> in the pack tables.
> I haven't yet actually run this code, but the change isn't big. I
> didn't find any relevant tests to update. Anyone want to venture some
Makes sense to me. The patch looks good too, though I have not tried it.
Maurits van Rees
Web App Programmer at Zest Software: http://zestsoftware.nl
Personal website: http://maurits.vanrees.org/
For more information about ZODB, see the ZODB Wiki:
ZODB-Dev mailing list - ZODB-Dev@zope.org