We have a large dataset of 650,000+ records that I'd like to examine easily in Python. I have figured out how to put this into a ZODB file that totals 4 GB in size. But I'm new to ZODB and very large databases, and have a few questions.

1. The data is in a IOBTree so I can access each item once I know the key, but to get the list of keys I tried:

scores = root['scores']
ids = [id for id in scores.iterkeys()]

This seems to require the entire tree to be loaded into memory which takes more RAM than I have.

What kind of objects are the values? They should probably extend Persistent. Just my $0.02.


For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to