What is the system memory utilization? If the in-memory database overloads the real memory the frantic os virtual memory paging (thrashing) could slow everything down.
>From your use scenario it sounds like your database shouldn't be that large, but how large is the database? and how large is the system ram? What percent of the ram is being used? Jim Callahan On Wed, Aug 6, 2014 at 3:39 PM, Fabian Giesen <fabi...@radgametools.com> wrote: > On 8/6/2014 11:51 AM, Jensen, Vern wrote: > >> *bump* >> >> Anyone? >> > > Not sure if this is your problem, but this kind of load puts a lot of > stress on SQLite's page cache, which might well perform worse than your > OS'es page cache (used for "real" disk IO) does, especially under high > contention. > > Also, SQLite's defaults wrt. threading are set up to be safe, which means > that when in doubt about what your code might be doing, SQLite will > serialize. > > I did a bit of SQLite perf tuning recently and here's two things you might > want to try: > > 1. What's your value for "flags"? SQLite by default (pessimistically) > assumes that you might be sharing a DB connection between multiple threads, > and thus wraps essentially all API functions in locks to make sure this is > safe. In a heavily multi-threaded scenario, I would recommend that you > manually ensure that each connection is only used by one thread at a time, > and then add SQLITE_OPEN_NOMUTEX to your flags. This reduces lock traffic > substantially. (If your connection was per-thread already, doing so will > not reduce contention or have any impact on your asymptotic perf or > scalability, but locks aren't free.) > > 2. Before sqlite3_initialize, try > > "sqlite3_config(SQLITE_CONFIG_MEMSTATUS, 0);" > > This disables some API functions that provide access to memory stats, but > avoids a global lock on every memory allocation/free, and SQLite does a lot > of them. If you don't need these memory statistics, turning this off can > make a noticeable difference in scalability. > > I'm not sure if this will affect page cache throughput specifically, but > these two things are both fairly easy to try. > > -Fabian > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
