I posted the message below on the numpy discussion list. Few if any people there seem to use sqlite.
I am interesting in using sqlite to store data for scientific research. I wrote the attached test program to save and load a simulated 11x500,000 recarray. Average save and load times are given below (timeit with 20 repetitions). The save time for sqlite is not really fair because I have to delete the data table each time before I create the new one. It is still pretty slow in comparison. Loading the recarray from sqlite is significantly slower than cPickle. I am hoping there may be more efficient ways to save and load recarray¹s from/to sqlite than what I am now doing. Note that I infer the variable names and types from the data rather than specifying them manually. saving recarray with cPickle: 1.448568 sec/pass saving recarray with sqlite: 193.286204 sec/pass loading recarray using cPickle: 0.471365 sec/pass loading recarray with sqlite: 15.977018 sec/pass To run the test you do need numpy (http://www.scipy.org/Download) Best, Vincent
load_sqlite_test.py
Description: Binary data
_______________________________________________ DB-SIG maillist - DB-SIG@python.org http://mail.python.org/mailman/listinfo/db-sig
