On Sat, Dec 19, 2009 at 12:46 PM, Roger Binns <rog...@rogerbinns.com> wrote: > Robert Citek wrote: >> and much longer than >> piping the list of random integers into the sort command. > > A considerable amount of time in your test script is actually spent in print > calls to pipes.
I don't follow. An unordered query takes 3 seconds while an ordered query takes 99 seconds. Are you saying the sorting process is printing to pipes behind the scenes? > Here are some timings I got. Note that used tmpfs so that > disk speeds are not a factor. What I suspect is that I am hitting some limit which indeed causes disk access to be a factor. While I can avoid disk access for smaller datasets, I will not be able to for larger ones. >> Any pointers in the right direction are greatly appreciated, >> especially given that this dataset only needs to be queried and sorted >> once. > > If this is really what your data looks like the just use the sort command. > If you have to use SQLite (which will involve import, sorting, output) then > you need to play with page and cache_sizes and indices. Thanks for the pointers. I'll try out various values for page and cache_size. > However this overhead is unlikely to be as fast as sort. If I could get sqlite3 to be within a factor of 2 of sort, that would work fine. Regards, - Robert _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
