On Dec 29, 2007, at 3:22 PM, Dieter Maurer wrote:

Jim Fulton wrote at 2007-12-28 10:20 -0500:
...
There Berkely Database Storage supported automatic incremental packing without garbage collection. If someone were to revitalize that effort
and if one was willing to do without cyclic garbage collection, then
that storage would remove the need for the sort of disruptive pack we
have with FileStorage now.

Why do you consider "pack" disruptive?

Disruptive isn't exactly the word I was looking for, although the current pack approach is disruptive. It causes a concentrated I/O and CPU load. For our databases, database performance drops significantly, especially during the later stages of packing.



Note that I'm working on a new FileStorage packer that is 2-3 times
faster and, I believe, much less disruptive than the current packing
algorithm.

If you are at it: I think the lock which protects the "finish" test
is hold too long. Currently, it is just release for a very short time
and then immeadiately reacquired. It should be safe to release it
immediately after the "finish" test has failed.


I don't know what you are referring to. Could you be more specific?

Note that the packing code in zc.FileStorage releases the commit lock far more frequently than the old pack code and doesn't begin acquiring it until after copying data written up to the time packing started, rather than after writing data up to the pack time. I expect that the packing code in 3.9 will acquire the commit lock much later.

Jim

--
Jim Fulton
Zope Corporation


_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev

Reply via email to