On 02/07/2018 10:16 PM, Deon Brewis wrote:
That's an interesting thought, but we see this on Windows as well, and on
Windows these files are opened with SQLITE_OPEN_EXCLUSIVE.
So this looks more like something is overwriting the memory of Page1 before
SQLITE writes it back to disk.
I don't think SQLITE verifies the memory during every write. There's a:
if( memcmp(page1, zMagicHeader, 16)!=0 ){
during lockBtree() but that seems to be on read, not write.
Is there somewhere I can add a check of the header buffer before the write to
disk?
Or does SQLITE literally never write this part of the header again once written
the first time? (Seek past the zMagicHeader?).
If it does rewrite it, it writes all of the first database page at the
same time.
You're using wal mode, so whenever page 1 (the page containing the db
header) is written, it is appended to the wal file. Then, later on, a
checkpoint copies it from the wal file into the db file.
Writing the page to the wal file happens here:
http://www.sqlite.org/src/artifact/5a3f464edd645?ln=3404
You could add (say):
assert( p->pgno!=1 || 0==memcmp(pData, "SQLite", 6) );
Then copying from the wal file into the db file is here:
http://www.sqlite.org/src/artifact/5a3f464edd645?ln=1861
Add (again, say):
assert( iDbpage!=1 || 0==memcmp(zBuf, "SQLite", 6) );
Dan.
- Deon
-----Original Message-----
From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On
Behalf Of Dan Kennedy
Sent: Tuesday, February 6, 2018 9:08 AM
To: sqlite-users@mailinglists.sqlite.org
Subject: Re: [sqlite] Header corruption
On 02/06/2018 11:57 PM, Deon Brewis wrote:
I’m trying to track down SQLITE corruptions that seems to corrupt our databases
in a consistent way. (Running on 3.20.1).
This isn’t related to a crash or hardware failure. The app is running and
reading and writing to the database fine, and then suddenly we start getting a
SQLITE_NOTADB errors. The first error we got was on trying to execute a ‘BEGIN
TRANSACTION’, and everything thereafter started failing.
When our database headers are fine, I see this:
00000000h: 53 51 4C 69 74 65 20 66 6F 72 6D 61 74 20 33 00 ; SQLite format 3.
00000010h: 10 00 02 02 00 40 20 20 00 00 10 9B 00 01 34 42 ; .....@
...›..4B
On corruption #1, I see this:
00000000h: 15 03 01 00 20 01 91 21 85 44 17 2C ED BE 21 FA ; ....
.‘!…D.,í¾!ú
00000010h: 9E 74 E7 EA 9A 0A 15 2C 99 B0 B7 3B C6 F0 35 FB ;
žtçêš..,™°·;Æð5û
On corruption #2, I see this:
00000000h: 15 03 01 00 20 0F 1E CB B1 FF 9C 1E D0 D6 BB 22 ; .... ..˱ÿœ.ÐÖ»"
00000010h: 15 64 D3 F4 DD 38 FB DF A3 E0 47 B8 D7 F6 21 BC ;
.dÓôÝ8ûߣàG¸×ö!¼
This bit pattern of “15 03 01 00” for the first 4 characters where it should be
“SQLi” is a bit too much of a coincidence to ignore or write off as disk error.
The rest of the header and file after the first 25-ish bytes or so, seem fine.
It’s just these first 25 bytes that gets corrupted. If I restore Offsets 0 to
Offsets 27 out of a “working” database, I get a usable file.
Under what circumstances would SQLITE even attempt to re-write the header
string?
Is this on UNIX?
Quite often this sort of thing occurs because some other module is writing to a
file-descriptor owned by SQLite. Usually because it closed its own fd, then
SQLite opened the db file and was assigned the same integer fd value, then the
rogue module wrote to the fd anyway. In other words, some other module is doing:
close(fd);
write(fd, "1503010020...", 25);
and between those two calls SQLite is calling open() and is being assigned a
file-descriptor with the same integer value as fd.
Dan.
- Deon
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmaili
nglists.sqlite.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsqlite-users&data=
02%7C01%7C%7C12190af418bc4a32192b08d56d8432f3%7C84df9e7fe9f640afb435aa
aaaaaaaaaa%7C1%7C0%7C636535336909688339&sdata=OMCv%2BtfSJ4PvlFTFw8Cqcc
1eizCrnIgFnNSJ7x5XUoo%3D&reserved=0
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailinglists.sqlite.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsqlite-users&data=02%7C01%7C%7C12190af418bc4a32192b08d56d8432f3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636535336909688339&sdata=OMCv%2BtfSJ4PvlFTFw8Cqcc1eizCrnIgFnNSJ7x5XUoo%3D&reserved=0
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
