A scientific application I work with has clumsy data retrieval options. I dumped the application's output--integer and floating point numbers--into an SQLite database and soon after began to enjoy the power of SQL to pull out interesting results.
The main complaint for making the transfer to SQLite a permanent part of our solution is the time it takes to insert the numbers into a database. It takes about a minute to insert 24 million numbers into three tables. Most database people (including me) would be thrilled to see this kind of insert performance but my colleagues are asking if it can be sped up.
Try setting PRAGMA synchronous=OFF and seeing how much improvement you get. If it's significant, you'll have to decide for yourself if you can live with the increased risk of corruption if there's a power failure during a batch load.
