On 3 May 2014, at 1:59pm, Hayden Livingston <halivings...@gmail.com> wrote:
> My workload is quite straightforward, I have a multi-threaded application > that logs to this file but from within any single thread at any given time. > So from SQLite's perspective only one person will be writing to this > database and nobody will ever read from it. > > The reading happens after the entire database is written to. > > Is this a use case for SQLite? It seems like the benefit I get from SQLite > is that I can do SQL on my previously textual data and optionally add > indexes. The benefit you'd get from SQL is the ability to look up a specific entry easily rather than having to look through an entire file to find the entry you want. > HOWEVER, this is my catch -- I sometimes have the need to query two > databases at the same time. In fact, sometimes I might need to query 100's > of these small databases. Would you just recommend querying the 100's in > parallel and then joining them in memory or something? Had you considered merging them all into one big database ? You currently have some sort of variable that you use to devise a filename. Turn that into a column name which is part of a UNIQUE key. Then, when you want to run a query all your data is in one big database and you can query every 'database' at the same time and return the answers in a single list. You should be able to get enough data together to test relatively easily. Write an importer that takes your existing text files and adds their data into one big SQLite database. Then you can try numerous SELECT queries using the SQLite shell tool (without having to write different software each time) and see what indexes would be useful and how fast they'd give you results. Simon. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
