Can you send ore details about your data and the updates and indexes you are using?
On 5/17/17, Joseph L. Casale <[email protected]> wrote: >> From: sqlite-users [mailto:[email protected]] >> On >> Behalf Of Clemens Ladisch >> Sent: Wednesday, May 17, 2017 6:04 AM >> To: [email protected] >> Subject: Re: [sqlite] Bulk load strategy >> >> Without an index, searching for a previous entry is likely to involve >> a scan through the entire table. It might be a better idea to have the >> index available here, even with the additional cost of updating it. > > While that showed true, both approaches are still too slow. Maintaining the > data in memory in order to facilitate the potential manipulation before > persisting it far exceeds the workstations memory capacity of 12Gb so > I need to come up with a new strategy. > > I tried adding ANALYZE statements periodically to update the indexes > however it seemed not to matter, I also tried committing transactions > before the ANALYZE at the same interval without any success. > > Anyone have any other suggestions? > > Thanks guys, > jlc > _______________________________________________ > sqlite-users mailing list > [email protected] > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > -- D. Richard Hipp [email protected] _______________________________________________ sqlite-users mailing list [email protected] http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
