Hi Adam, Thank you very much for your response. I had not considered using copies of the database. In my method there are cases where writes to the database by one process may be relevant to another process, so I'm not sure that that would be a good option.
I don't explicitly set sqlite3_busy_timeout(sqlite3*, int ms) so I will try adding that after the database connection is opened to see if it limits those errors. Thanks Again. On Thu, Jul 2, 2015 at 10:28 AM, Adam Devita <adevita at verifeye.com> wrote: > Good day, > > I'm sure others on the list will add better insight, but is your task > parallel enough that your nodes can work with a copy of the database > and submit changes the one the others copy from when 'done' their > calculation? > > Are you using https://www.sqlite.org/c3ref/busy_timeout.html ? > > regards, > Adam > > This may be beside the point in terms of optimization, but your query > looks rather character based on int like information. > > On Thu, Jul 2, 2015 at 10:09 AM, Kathleen Alexander <katcalex at mit.edu> > wrote: > > Hi, > > > > I apologize if this is an incorrect forum for this question, but I am > > pretty new to SQLite and have been unable to resolve this issue through > > other searches. Feel free to direct me to a more appropriate forum. > > > > Essentially, I have written an application in C++ that interfaces (reads > > and writes) with a SQLite database, and I am getting lots of 'database is > > locked' errors. Right now, I am trying to establish whether those errors > > are due to my improper use of SQLite itself, or if the real problem is > that > > SQLite is not a good fit for my application. > > > > My application runs on Linux (ubuntu 13.10), and is driven by a bash > script > > that spawns many (~60 on a 64 core workstation) instances of a serial, > C++ > > program, each of which opens its own connection to the database and > > performs reads and writes. > > > > *An example SELECT query from my program looks like:* > > //open db connection > > sqlite3 *db; > > char *zErrMsg = 0; > > SQLITE3 sql(dbase.c_str()); > > > > statement = "SELECT * from configs_table WHERE id='31'"; > > sql.exe(statement.c_str()); > > if( sql.vcol_head.size() > 0 ){ > > //do things with sql.vdata[] > > }//end query returned results > > > > *An example of a write statement looks like:* > > statement = "UPDATE configs_table SET searched='2' WHERE id='31'"; > > sql.exe(statement.c_str()); > > > > About 97% of the time, the select statement works fine, but in the other > 3% > > of cases, I see a 'database is locked' error in the log file of my > program. > > About 50% of the time, the write statement returns 'database is locked'. > > > > Additionally, if this application is running and I try to query the > > database from the terminal, I almost always get a 'database is locked' > > error. > > > > Thus, I am wondering if I am doing something wrong in my implementation > of > > the C++ --> SQLite interaction, or if the real problem is that this > > application is not well suited to use with SQLite (I went through the > > checklist before implementing it and thought my application passed the > > suitability requirements). > > > > Lastly: > > A. if it seems like this is an implementation issue, rather than a > > capability issue, if I were to scale up my method to spawn say 500-1000 > > processes at a time (on a supercomputing cluster), would there be any > > concern about SQLite scaling to that level? > > B. If SQLite is not a good fit for my program, do you have any > suggestions > > of an alternative database engine that is free or might be free or > > inexpensive for academic use? > > > > Thanks in advance, > > Kathleen > > _______________________________________________ > > sqlite-users mailing list > > sqlite-users at mailinglists.sqlite.org > > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > > > > -- > -------------- > VerifEye Technologies Inc. > 151 Whitehall Dr. Unit 2 > Markham, ON > L3R 9T1 > _______________________________________________ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
