On 02/07/2018 10:16 PM, Deon Brewis wrote:
That's an interesting thought, but we see this on Windows as well, and on 
Windows these files are opened with SQLITE_OPEN_EXCLUSIVE.

So this looks more like something is overwriting the memory of Page1 before 
SQLITE writes it back to disk.

I don't think SQLITE verifies the memory during every write. There's a:
if( memcmp(page1, zMagicHeader, 16)!=0 ){

during lockBtree() but that seems to be on read, not write.

Is there somewhere I can add a check of the header buffer before the write to 

Or does SQLITE literally never write this part of the header again once written 
the first time? (Seek past the zMagicHeader?).

If it does rewrite it, it writes all of the first database page at the same time.

You're using wal mode, so whenever page 1 (the page containing the db header) is written, it is appended to the wal file. Then, later on, a checkpoint copies it from the wal file into the db file.

Writing the page to the wal file happens here:


You could add (say):

  assert( p->pgno!=1 || 0==memcmp(pData, "SQLite", 6) );

Then copying from the wal file into the db file is here:


Add (again, say):

  assert( iDbpage!=1 || 0==memcmp(zBuf, "SQLite", 6) );


- Deon

-----Original Message-----
From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On 
Behalf Of Dan Kennedy
Sent: Tuesday, February 6, 2018 9:08 AM
To: sqlite-users@mailinglists.sqlite.org
Subject: Re: [sqlite] Header corruption

On 02/06/2018 11:57 PM, Deon Brewis wrote:
I’m trying to track down SQLITE corruptions that seems to corrupt our databases 
in a consistent way. (Running on 3.20.1).

This isn’t related to a crash or hardware failure. The app is running and 
reading and writing to the database fine, and then suddenly we start getting a 
SQLITE_NOTADB errors. The first error we got was on trying to execute a ‘BEGIN 
TRANSACTION’, and everything thereafter started failing.

When our database headers are fine, I see this:
00000000h: 53 51 4C 69 74 65 20 66 6F 72 6D 61 74 20 33 00 ; SQLite format 3.
00000010h: 10 00 02 02 00 40 20 20 00 00 10 9B 00 01 34 42 ; .....@

On corruption #1, I see this:
00000000h: 15 03 01 00 20 01 91 21 85 44 17 2C ED BE 21 FA ; ....
00000010h: 9E 74 E7 EA 9A 0A 15 2C 99 B0 B7 3B C6 F0 35 FB ;

On corruption #2, I see this:
00000000h: 15 03 01 00 20 0F 1E CB B1 FF 9C 1E D0 D6 BB 22 ; .... ..˱ÿœ.ÐÖ»"
00000010h: 15 64 D3 F4 DD 38 FB DF A3 E0 47 B8 D7 F6 21 BC ;

This bit pattern of “15 03 01 00” for the first 4 characters where it should be 
“SQLi” is a bit too much of a coincidence to ignore or write off as disk error.

The rest of the header and file after the first 25-ish bytes or so, seem fine. 
It’s just these first 25  bytes that gets corrupted. If I restore Offsets 0 to 
Offsets 27 out of a “working” database, I get a usable file.

Under what circumstances would SQLITE even attempt to re-write the header 
Is this on UNIX?

Quite often this sort of thing occurs because some other module is writing to a 
file-descriptor owned by SQLite. Usually because it closed its own fd, then 
SQLite opened the db file and was assigned the same integer fd value, then the 
rogue module wrote to the fd anyway. In other words, some other module is doing:

    write(fd, "1503010020...", 25);

and between those two calls SQLite is calling open() and is being assigned a 
file-descriptor with the same integer value as fd.


- Deon

sqlite-users mailing list

sqlite-users mailing list
sqlite-users mailing list

sqlite-users mailing list

Reply via email to