My branch merge is now complete. I rebuilt the binaries that changed for
release and debug on all 3 OS. If you don't do your own build, you need to run
'make install' in chandler after your next 'svn update' there.
The repository format changed, you need to also re-create your repository
after this update.
For those who do their own builds, you need to rebuild the binaries that
changed in external before building chandlerdb in internal.
Here is what I changed while on my branch and is now included on the trunk:
Berkeley DB
-----------
- integrated new version of Berkeley DB, 4.4.16
- added patch to Berkeley DB to implement better memory use by avoiding
malloc/memcpy/free cycles by adding the capability to register a callback
with a DBT (the structure by which memory is returned from Berkeley DB).
That callback allows much finer control in memory allocation and and in
memory copies.
- on Mac OS X, 64 Mb of cache is now used since without it, performance is
4 times slower than it used to be. Cache now has a positive effect on
performance (in earlier versions of Berkeley DB, using cache on Mac OS X
would actually slow the system down). Closing chandler for the first time
after a repository create may take a few seconds (it didn't used to, I
have a question open with Sleepycat about that).
Python
------
- Python is still at version 2.4.2 but the bsddb extension is no longer
included nor built since it is superceded by the latest chandlerdb code.
PyLucene
--------
- PyLucene is now at version 1.9rc1 and is built from the latest in the Java
Lucene SVN tree. A lot of new features and bugs were fixed in the Lucene
1.9 software tree.
- PyLucene is also built against Berkeley DB 4.4.16 now
PyICU
-----
- minor bug fixes
chandlerdb
----------
- chandlerdb now implements the portions of the Python to C interface to
Berkeley DB it needs, bypassing Python's bsddb extension.
Bypassing bsddb makes it safer (better type checking) and simpler for me
to access Berkeley DB from C and allows me to extract better performance
from a number of operations by using better memory control (using the
callback mechanism added to Berkeley DB) and less back and forth between C
and Python
- the repository version is now implemented by a Berkeley DB sequence which
natively handles all the previously self-implemented locking and
incrementing needed to persistently maintain it
- the new interface to C Berkeley DB got me to revisit a number of threading
and concurrency issues that could lead to hard deadlocks. I hope they are
all fixed.
- better use of the transaction log file APIs were made and the repository
now, overall, uses a *lot* less disk space as it used to. Unused log
files are removed much quicker.
Overall, this branch's focus was performance. Commit performance, in
particular, is considerably faster. On my Mac, I used to get around 70 items
pers second committed after importing the good'ol 3000 event calendar, I now
get up to 127 items per second.
First chandler commit from scratch used to take around 14.5 s on my mac, it
now takes around 8 s.
Thank your for patience.
Andi..
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
