On Thu, Aug 21, 2008 at 10:32:23AM -0400, Jeffrey Becker wrote: > Just out of curiosity what happens if you call > "PRAGMA page_size=4096" > before running the import?
I wonder too, what does the page cache do when doing full table scans? If the cache has an LRU/LFU page eviction algorithm then full table scans should not be a big deal. Ideally it should not allow pages read during a full table scan to push out other pages, but if the cache is cold then a full table scan just might fill the cache. In this case we have full table scans in the process of copying on-disk tables to a memory DB. And it looks like the cache is cold in this case. The default cache size is 2000 pages (there's a way to persist a cache size). The default page size is 1024 bytes, but it's hard to tell exactly what it is in the OP's case. So it looks like the cache size should be ~20MB. And the DB size is ~9MB. The cache is plenty large enough to hold a copy of the on-disk DB. So we have: 9MB will be consumed in the page cache, and 9MB will be consumed by the memory DB (I assume memory DB pages aren't cached). Add in the overhead per-page, which seems to be .5KB, and you have 18MB + 9MB / 2 = 22.5MB That seems close to what the OP claimed. Perhaps to keep the memory footprint of SQLite3 low the OP should set the cache size way down during the copy-the-DB-into-memory part of the program using the cache_size pragma. I could be way off-track, but, try it and see. Nico -- _______________________________________________ sqlite-users mailing list email@example.com http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users