On Sat, Dec 19, 2009 at 04:10:09PM -0500, Robert Citek scratched on the wall:
> Building an index works well for 1 MM records. The build+query time
> (~40s=27+10) is about 6x faster than querying without an index
> (~240s), but still 6x slower than querying and piping to the sort
> command (~6s).
>
> However, times take noticeably longer when the dataset size is doubled
> to 2 MM records.
Since building the index essentially involves a sort, it is extremely
sensitive to cache size. Temporarily increasing the cache size to
something extremely large will help make the index build go much
faster.
Once it is built, it is built. You don't typically need a monster
cache to use the index, or for manipulating individual records.
One thing you need to remember is that SQLite is going to use a very
fixed and bound amount of memory, while something like the UNIX sort
command is just going gobble up as much as it needs until it hits VM
and just keeps going. Unless you make the SQLite cache several gig,
you're not really getting an apples-to-apples comparison.
-j
--
Jay A. Kreibich < J A Y @ K R E I B I.C H >
"Our opponent is an alien starship packed with atomic bombs. We have
a protractor." "I'll go home and see if I can scrounge up a ruler
and a piece of string." --from Anathem by Neal Stephenson
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
