That's an interesting thought, but we see this on Windows as well, and on
Windows these files are opened with SQLITE_OPEN_EXCLUSIVE.
So this looks more like something is overwriting the memory of Page1 before
SQLITE writes it back to disk.
I don't think SQLITE verifies the memory during every write. There's a:
if( memcmp(page1, zMagicHeader, 16)!=0 ){
during lockBtree() but that seems to be on read, not write.
Is there somewhere I can add a check of the header buffer before the write to
disk?
Or does SQLITE literally never write this part of the header again once written
the first time? (Seek past the zMagicHeader?).
- Deon
-----Original Message-----
From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On
Behalf Of Dan Kennedy
Sent: Tuesday, February 6, 2018 9:08 AM
To: sqlite-users@mailinglists.sqlite.org
Subject: Re: [sqlite] Header corruption
On 02/06/2018 11:57 PM, Deon Brewis wrote:
> I’m trying to track down SQLITE corruptions that seems to corrupt our
> databases in a consistent way. (Running on 3.20.1).
>
> This isn’t related to a crash or hardware failure. The app is running and
> reading and writing to the database fine, and then suddenly we start getting
> a SQLITE_NOTADB errors. The first error we got was on trying to execute a
> ‘BEGIN TRANSACTION’, and everything thereafter started failing.
>
> When our database headers are fine, I see this:
> 00000000h: 53 51 4C 69 74 65 20 66 6F 72 6D 61 74 20 33 00 ; SQLite format 3.
> 00000010h: 10 00 02 02 00 40 20 20 00 00 10 9B 00 01 34 42 ; .....@
> ...›..4B
>
> On corruption #1, I see this:
> 00000000h: 15 03 01 00 20 01 91 21 85 44 17 2C ED BE 21 FA ; ....
> .‘!…D.,í¾!ú
> 00000010h: 9E 74 E7 EA 9A 0A 15 2C 99 B0 B7 3B C6 F0 35 FB ;
> žtçêš..,™°·;Æð5û
>
> On corruption #2, I see this:
> 00000000h: 15 03 01 00 20 0F 1E CB B1 FF 9C 1E D0 D6 BB 22 ; .... ..˱ÿœ.ÐÖ»"
> 00000010h: 15 64 D3 F4 DD 38 FB DF A3 E0 47 B8 D7 F6 21 BC ;
> .dÓôÝ8ûߣàG¸×ö!¼
>
>
> This bit pattern of “15 03 01 00” for the first 4 characters where it should
> be “SQLi” is a bit too much of a coincidence to ignore or write off as disk
> error.
>
> The rest of the header and file after the first 25-ish bytes or so, seem
> fine. It’s just these first 25 bytes that gets corrupted. If I restore
> Offsets 0 to Offsets 27 out of a “working” database, I get a usable file.
>
> Under what circumstances would SQLITE even attempt to re-write the header
> string?
Is this on UNIX?
Quite often this sort of thing occurs because some other module is writing to a
file-descriptor owned by SQLite. Usually because it closed its own fd, then
SQLite opened the db file and was assigned the same integer fd value, then the
rogue module wrote to the fd anyway. In other words, some other module is doing:
close(fd);
write(fd, "1503010020...", 25);
and between those two calls SQLite is calling open() and is being assigned a
file-descriptor with the same integer value as fd.
Dan.
>
> - Deon
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmaili
> nglists.sqlite.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsqlite-users&data=
> 02%7C01%7C%7C12190af418bc4a32192b08d56d8432f3%7C84df9e7fe9f640afb435aa
> aaaaaaaaaa%7C1%7C0%7C636535336909688339&sdata=OMCv%2BtfSJ4PvlFTFw8Cqcc
> 1eizCrnIgFnNSJ7x5XUoo%3D&reserved=0
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailinglists.sqlite.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsqlite-users&data=02%7C01%7C%7C12190af418bc4a32192b08d56d8432f3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636535336909688339&sdata=OMCv%2BtfSJ4PvlFTFw8Cqcc1eizCrnIgFnNSJ7x5XUoo%3D&reserved=0
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
