Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Fred Drake wrote:
I have a need for 64-bit BTrees (at least for IOBTree and OIBTree),
and I'm not the first. I've created a feature development branch for
this, and checked in my initial implementation.
I've modified the existing code to use PY_LONG_LONG instead of int for
the key and/or value type; there's no longer a 32-bit version in the
modified code. Any Python int or long that can fit in 64 bits is
accepted; ValueError is raised for values that require 65 bits (or
more). Keys and values that can be reported as Python ints are, and
longs are only returned when the value cannot be converted to a Python
int.
This can have a substantial effect on memory consumption, since keys
and/or values now take twice the space. There may be performance
issues as well, but those have not been tested.
There are new unit tests, but more are likely needed.
If you're interested in getting the code from Subversion, it's available at:
svn://svn.zope.org/repos/main/ZODB/branches/fdrake-64bits/
Ideally, this or some variation on this could be folded back into the
main development for ZODB. If this is objectionable, making 64-bit
btrees available would require introducing new versions of the btrees
(possibly named LLBTree, LOBTree, and OLBTree).
I think coming up with new types is the only reasonable thing to do,
given the prevalence of persistent BTrees out in the wild. Changing the
runtime behavior (footprint, performance) of those objects is probably
not something which most users are going to want, at least not without
carefully considering the implications.
It really depends on what the impact is. It would be nice to get a feel
for whether this really impacts memory or performance for real applications.
This adds 4-bytes per key or value. That isn't much, especially in a typical
Zope application. Similarly, it's hard to say what the difference in C integer
operations will be. I can easily imagine it being negligible (or being
significant :).
OTOH, adding a new type could be a huge PITA. We'd like to use these with
existing
catalog and index code, all of which uses IIBTrees. If the performance impacts
are
modest, I'd much rather declare IIBTrees to use 64-bit rather than 32-bit
integers.
I suppose an alternative would be to add a mechanism to configure IIBTrees to
use
either 32-bit or 64-bit integers at run-time.
Jim
--
Jim Fulton mailto:[EMAIL PROTECTED] Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
