You have exposed two relatively advanced programming topics: code profiling and database performance tuning.
Because I am a relative noobie to both Python and Sqlite, I cannot unfortunately give you specific directions. But I can offer an approach you might try. Maybe you should first learn where the bottleneck lies through code profiling. Generally a code profiler will trace the code as it runs and timestamp the steps. It should be relatively easy to spot long waits after database calls, for example. Python does have built in code profiling as described here: http://docs.python.org/library/profile.html As a short cut, you might try indexing your table(s). In general, you want an index on any column that appears in a where clause or an order by clause. Is the data normalized at all, or is it all in one huge table like a spreadsheet would produce? If the data is not normalized, you will need to find a way to normalize it.
