On 2014/07/16 14:55, Rob Willett wrote:
I’ll second what Simon says, I use the very same technique for a table with 4M+ records in and its so fast I thought I had an error and looked for bugs in my code. I >assumed (incorrectly) that it would be very slow, It isn’t.
In a similar fashion I had made this system for basically loading CSV files into an SQLite DB, then running all kinds of rules on it (which I could make with a designer) such as search-replace, substitute column values, do checks, delete rows with empty values in a certain column, etc. etc. and then finally export it to a CSV again, all basically streamlining a datafeed alteration process into a one-click thing. The viewer I used was based on a method discussed in another thread where the virtual view would get actual data only for items in the visible field by primary key. I tested it with some CSV tables over 250MB big resulting in 10mil+ rows, and here was my surprise, whether I looked at the top of the list, or the bottom, or anywhere else, the data retrieval was instantaneous - retrieving a page worth of records at whatever speed I can scroll the vertical scrollbar - not a single slow-down as I got further down or indeed any other slowness. I have come to expect great performance from SQLite as a standard, but I am still often surprised at just how quick it can be.
(btw: This app is freely shared if anyone needs something of the kind or fancy testing the above, just mail me)
...// I have come to realise that the people who answer here are real experts, I will not embarrass them by naming names//...
Oh don't worry, we know exactly who you mean... ;) It's Igor right? We all want to be like Igor when we grow up... *sighs dreamily* _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
