I've managed to reproduce a scenario where one database, database1.db is rebuilding a large wal-index. Meanwhile, all other threads attempting to open completely unrelated databases (e.g. database2.db) are blocked until the wal-index on database1.db is finished rebuilding. This is obviously undesirable.
Here is how I reproduced it. There are at least 3 threads involved here. Shared cache mode must be enabled and database1.db must have a large wal file that needs its wal-index rebuilt. Thread #1 succeeds in opening database1.db. It proceeds to call, in my case "PRAGMA cache_size = ...". I believe this thread acquires a btree lock of database1.db and starts rebuilding the wal-index. Thread #2 attempts to open database1.db, and inside of the function sqlite3BtreeOpen, proceeds to acquire the SQLITE_MUTEX_STATIC_OPEN lock, and then call sqlite3BtreeSchema, which calls sqlite3BtreeEnter and waits on the same btree lock that Thread #1 has. Thread #3 can attempt to open any database and it will block attempting to acquire the SQLITE_MUTEX_STATIC_OPEN lock. Does this make sense? I'm not 100% sure what the SQLITE_MUTEX_STATIC_OPEN is needed for, but since it's a global lock, it seems that it should do a little as possible while holding this lock, and everything goes wrong when it attempts to acquire a btree lock while holding this lock. Here are some actual stack traces reproduced from sqlite 3.8.7.2. From looking at the source, it appears the latest version has the same problem. Let me know if you need any additional information from me. Thread #1 #0 0x00007f2b92f4c9cd in read () at ../sysdeps/unix/syscall-template.S:84 #1 0x00007f2b73019b04 in unixRead () #2 0x00007f2b7302abe8 in walIndexReadHdr () #3 0x00007f2b7302ae4f in walTryBeginRead () #4 0x00007f2b73031886 in sqlite3PagerSharedLock () #5 0x00007f2b73031a26 in sqlite3BtreeBeginTrans () #6 0x00007f2b73059978 in sqlite3InitOne () #7 0x00007f2b73059c17 in sqlite3Init () #8 0x00007f2b73059cb2 in sqlite3ReadSchema () #9 0x00007f2b73060b48 in sqlite3Pragma () #10 0x00007f2b730663d1 in sqlite3Parser () #11 0x00007f2b730680a1 in sqlite3RunParser () #12 0x00007f2b73068450 in sqlite3Prepare () #13 0x00007f2b7306877c in sqlite3LockAndPrepare () #14 0x00007f2b7306897d in sqlite3_prepare_v2 () Thread #2. I'm fairly confident that it's inside of a call to sqlite3BtreeSchema which has been inlined. #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f2b92824dfd in __GI___pthread_mutex_lock (mutex=0x7f2b080018d8) at ../nptl/pthread_mutex_lock.c:80 #2 0x00007f2b7301c873 in btreeLockCarefully () #3 0x00007f2b7304804b in sqlite3BtreeOpen () #4 0x00007f2b7306bc77 in openDatabase () Thread #3 #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f2b92824dfd in __GI___pthread_mutex_lock (mutex=0x7f2b7328e350 <staticMutexes.9375+80>) at ../nptl/pthread_mutex_lock.c:80 #2 0x00007f2b7304777a in sqlite3BtreeOpen () #3 0x00007f2b7306bc77 in openDatabase () Thanks, Brian Vincent _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
