On 8/6/2014 11:51 AM, Jensen, Vern wrote:
*bump*
Anyone?
Not sure if this is your problem, but this kind of load puts a lot of
stress on SQLite's page cache, which might well perform worse than your
OS'es page cache (used for "real" disk IO) does, especially under high
contention.
Also, SQLite's defaults wrt. threading are set up to be safe, which
means that when in doubt about what your code might be doing, SQLite
will serialize.
I did a bit of SQLite perf tuning recently and here's two things you
might want to try:
1. What's your value for "flags"? SQLite by default (pessimistically)
assumes that you might be sharing a DB connection between multiple
threads, and thus wraps essentially all API functions in locks to make
sure this is safe. In a heavily multi-threaded scenario, I would
recommend that you manually ensure that each connection is only used by
one thread at a time, and then add SQLITE_OPEN_NOMUTEX to your flags.
This reduces lock traffic substantially. (If your connection was
per-thread already, doing so will not reduce contention or have any
impact on your asymptotic perf or scalability, but locks aren't free.)
2. Before sqlite3_initialize, try
"sqlite3_config(SQLITE_CONFIG_MEMSTATUS, 0);"
This disables some API functions that provide access to memory stats,
but avoids a global lock on every memory allocation/free, and SQLite
does a lot of them. If you don't need these memory statistics, turning
this off can make a noticeable difference in scalability.
I'm not sure if this will affect page cache throughput specifically, but
these two things are both fairly easy to try.
-Fabian
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
