On 5 Nov 2018, at 3:28pm, Wasilios Goutas <was...@goutas.de> wrote: > To speed up pre-processing and importing of data I would like to grand > several threads access to the same in memory database
I did not look at your source code, but you may not get the results you want. Using many threads/processes to import data into the same database does not speed things up very much, because each thread has to lock the database while it writes to it. If all your data is available at once then, for speed, do multi-threaded pre-processing and write the resulting INSERT commands (or CSV lines) to a text file. Once the text file is complete use a single thread to import this text file into your SQLite database as a single transaction (surround it with BEGIN ... END). Obviously, I have to tested relative speeds with your data and your hardware. Simon. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
