count(*) is pretty slow in sqlite because it basically does select * and then counts the results. This means it's looking through your whole big file.
You can come up with some tricks like keeping a separate count up-to-date with triggers. There have been some old threads on optimizing count that you may want to look for. Brett
