On Jan 23, 2008, at 9:59 AM, Benji York wrote:

Flavio Coelho wrote:
Actually what I am trying to run away from is the "packing monster" ;-)

Jim has done a great deal of work on packing (that will go into 3.9 I presume)

and is available now in zc.FileStorage.

that should make your pack 3 to 6 times faster (depending on if you do garbage collection at pack time or not).

And consumes twice as much memory, depending on your settings.

For testing, I used a 20G database containing catalog data whos size was cut in half by packing. It used around 900MB of memory for packing with garbage collection. :( Without GC, it used much less.

Another major benefit of my new packing code is that it does most of the work in a separate process, which allows it to take advantage of multiple processors.

I want to be able to use an OO database without the inconvenience of having
it growing out of control and then having to spend hours packing the
database every once in a while. (I do a lot of writes in my DBs). Do this
Holy grail of databases exist? :-)

Why not put the pack in cron?

My new packing code helps a lot, but packing is still very disruptive.

IMO, something that packed incrementally, with disk being freed along the way, would be a big improvement. This isn't possible with FileStorage.


Jim Fulton
Zope Corporation

For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to