Op 22-02-11 17:25, Martijn Pieters schreef: > Hi, > > I was already investigating the possibility to split the RelStorage > packing process up into smaller chunks. > > Due to the expected load on the Oracle cluster during a pack, we'll > have to run the pack at night and want to be absolutely certain that > database is ready for normal site operations again the next day. With > a 40+GB database (hasn't been packed for it's entire run, more than 2 > years now) we are not confident packing will be done in one night.
I wonder it it may help to set pack-gc to false during the first pack. According to the docs this is faster, though it of course leaves more unused objects behind. Set pack-gc to the default true value for subsequent packs. Theoretically this should make sure that the first pack will finish on time and leave an already smaller database; after the second pack the database is at its smallest. > To at least get a handle on how much work the packing is going to be, > and to have a nice stopping point, I looked at splitting pre-pack and > pack operations out into two separate steps. To my delight I saw that > the 1.5.0 beta already implements basically running only the pre-pack > phase (the --dry-run option). From there I created the attached patch, > one that renames the dry-run op into a 'prepack only' option, and adds > another option to skip the pre-pack and just use whatever is present > in the pack tables. > > I haven't yet actually run this code, but the change isn't big. I > didn't find any relevant tests to update. Anyone want to venture some > feedback? Makes sense to me. The patch looks good too, though I have not tried it. -- Maurits van Rees Web App Programmer at Zest Software: http://zestsoftware.nl Personal website: http://maurits.vanrees.org/ _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev