On Sun, Mar 11, 2012 at 11:26 AM, Barbie <[email protected]> wrote: > Andreas has already alerted me to this, but even recreating the SQLite > DB from scratch has the same problem. The only thing I can think of is > that the DB is so big now that SQLite has problems storing the data. > > This is why I want to look at an alternative method of getting at the > data. I'm going to look at creating an API that returns the records in > CSV or JSON format for a given range of IDs. This will allow you to > maintain your own DB as you wish, rather than have to continually > download several Gig repeatedly.
Have you considered splitting up the database so one will have all the data up till date X then another from X-Y etc. This way only the newest database will be constantly updated. This would of course make queries a bit harder but maybe this would be an easier solution. At least in the short run. Gabor
