On Thu, Aug 21, 2008 at 10:32:23AM -0400, Jeffrey Becker wrote:
> Just out of curiosity what happens if you call
> "PRAGMA page_size=4096"
> before running the import?
I wonder too, what does the page cache do when doing full table scans?
If the cache has an LRU/LFU page eviction algorithm then full table
scans should not be a big deal. Ideally it should not allow pages read
during a full table scan to push out other pages, but if the cache is
cold then a full table scan just might fill the cache.
In this case we have full table scans in the process of copying on-disk
tables to a memory DB. And it looks like the cache is cold in this
case.
The default cache size is 2000 pages (there's a way to persist a cache
size). The default page size is 1024 bytes, but it's hard to tell
exactly what it is in the OP's case.
So it looks like the cache size should be ~20MB. And the DB size is
~9MB. The cache is plenty large enough to hold a copy of the on-disk
DB.
So we have: 9MB will be consumed in the page cache, and 9MB will be
consumed by the memory DB (I assume memory DB pages aren't cached). Add
in the overhead per-page, which seems to be .5KB, and you have
18MB + 9MB / 2 = 22.5MB
That seems close to what the OP claimed.
Perhaps to keep the memory footprint of SQLite3 low the OP should set
the cache size way down during the copy-the-DB-into-memory part of the
program using the cache_size pragma.
I could be way off-track, but, try it and see.
Nico
--
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
