Got it. I'll add that one to our blacklist as well then. Thanks again! Ben
On Wed, May 23, 2018 at 11:29 AM, Dan Kennedy <danielk1...@gmail.com> wrote: > On 05/24/2018 01:01 AM, Ben Asher wrote: > >> I see. I think that makes sense! I've gone ahead and added these 2 >> functions to our local Tsan blacklist then. In researching this further, I >> found one more Tsan issue here in walTryBeginRead: >> >> - One thread is here: >> http://www.sqlite.org/src/artifact/aa9cffc7a2bad?ln=2558 >> - One thread is here: >> http://www.sqlite.org/src/artifact/aa9cffc7a2bad?ln=2571 >> >> Is this one expected as well? If so, I'll also add this function to our >> blacklist. >> > > It is expected, yes. > > The writer at line 2571 is holding an EXCLUSIVE posix lock when it writes > the aReadMark[] entry. The reader at 2558 is not holding any lock, but > later on in the same function it takes a SHARED lock and checks that the > value is as it was earlier. And retries everything if it is not. > > http://www.sqlite.org/src/artifact/aa9cffc7a2bad?2589..2591 > > Dan. > > > > > > > >> Thanks again! >> >> Ben >> >> >> On Wed, May 23, 2018 at 1:24 AM, Dan Kennedy <danielk1...@gmail.com> >> wrote: >> >> On 05/23/2018 06:21 AM, Ben Asher wrote: >>> >>> Hi there! I believe I've found a SQLite bug that triggers TSAN. I'm >>>> hoping >>>> to either confirm this bug or learn what I'm doing wrong. Some >>>> background: >>>> in our code, we make sure to synchronize writes to our database such >>>> that >>>> only one write can happen at any given time for the same database in our >>>> application (i.e. there's one synchronized writer connection). However, >>>> we >>>> allow any number of reads (readers create their own connections, which >>>> are >>>> similarly not shared with other threads) to happen on the same database >>>> (with a cap on the number of connections to respect resource limits). >>>> Additionally, the database has PRAGMA journal_mode=WAL; set. At the >>>> moment >>>> when TSAN is triggered, there is one reader and one writer each with >>>> their >>>> own connection to the same db performing a read and a write: >>>> >>>> - Reader is running: sqlite3WalFindFrame, specifically the line with >>>> this >>>> code (line 59679 in my sqlite.c): >>>> >>>> for(iKey=walHash(pgno); aHash[iKey]; iKey=walNextHash(iKey)){ >>>> >>>> >>>> - Writer is running walIndexAppend, specifically the line with this code >>>> (line 57856 in my sqlite.c): >>>> >>>> Thanks for the thread-testing. This one is a known condition. Tsan can't >>> see it, but we think it is safe. See the comment above the block in >>> sqlite3WalFindFrame() for details: >>> >>> http://www.sqlite.org/src/artifact/aa9cffc7a2bad?ln=2859..2883 >>> >>> Basically the race condition is only on the append-only hash-table. But >>> the hash-table is only used to accelerate lookups on the aPgno[] array. >>> And >>> since the hash-table is append-only, it can only produce false-positives >>> (which are filtered out when aPgno[] is inspected). >>> >>> Dan. >>> >>> >>> >>> >>> >>> aHash[iKey] = (ht_slot)idx; >>>> >>>> >>>> I apologize if those line numbers aren't helpful, however I hope the >>>> function names + code are unique enough to locate the lines in question. >>>> This is sqlite3 version 3.23.1. >>>> >>>> Please let me know if there's any other information I can provide. >>>> Thanks! >>>> >>>> Ben >>>> _______________________________________________ >>>> sqlite-users mailing list >>>> sqlite-users@mailinglists.sqlite.org >>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >>>> >>>> >>> _______________________________________________ >>> sqlite-users mailing list >>> sqlite-users@mailinglists.sqlite.org >>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >>> >>> >> >> > _______________________________________________ > sqlite-users mailing list > sqlite-users@mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > -- Ben _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
