On Jan 23, 2008, at 9:59 AM, Benji York wrote:
Flavio Coelho wrote:
Actually what I am trying to run away from is the "packing
monster" ;-)
Jim has done a great deal of work on packing (that will go into 3.9
I presume)
and is available now in zc.FileStorage.
that should make your pack 3 to 6 times faster (depending on if you
do garbage collection at pack time or not).
And consumes twice as much memory, depending on your settings.
For testing, I used a 20G database containing catalog data whos size
was cut in half by packing. It used around 900MB of memory for
packing with garbage collection. :( Without GC, it used much less.
Another major benefit of my new packing code is that it does most of
the work in a separate process, which allows it to take advantage of
multiple processors.
I want to be able to use an OO database without the inconvenience
of having
it growing out of control and then having to spend hours packing the
database every once in a while. (I do a lot of writes in my DBs).
Do this
Holy grail of databases exist? :-)
Why not put the pack in cron?
My new packing code helps a lot, but packing is still very disruptive.
IMO, something that packed incrementally, with disk being freed along
the way, would be a big improvement. This isn't possible with
FileStorage.
Jim
--
Jim Fulton
Zope Corporation
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
