On Mon, Jan 10, 2005 at 09:51:50AM -0800, [EMAIL PROTECTED] wrote: > sqlite3_step() is the function that is making all the excessive calls, one > call per row fetched.
> about 3000 of these tables. One complete scan of all 3000 companies will > make roughly 5.5 million malloc() calls! That seems highly non-optimal. If there is no other way around that already present in SQLite, and you don't want to hack one in, then the obvious step might be to link in one of the several high-performance malloc replacement libraries out there, e.g. Hoard: http://www.cs.umass.edu/~emery/hoard/ http://developers.sun.com/solaris/articles/multiproc/multiproc.html Tcl 8.4.x includes a threaded allocator (which is used by AOLserver), which maintains its own thread-local pools of memory and calls the system malloc() underneath only infrequently. It is not intended as a drop in replacement for malloc(), but I THINK the various malloc replacement libraries operate in similar (though more complex) fashions. I'd expect (but have not tested) that these memory pooling techniques would offer you a speed benefit even when you're using only 1 thread, as malloc'ing millions of times is usually not a good idea. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/
