On Mon, Dec 19, 2011 at 12:07 PM, David Fetter <da...@fetter.org> wrote: > On Mon, Dec 19, 2011 at 09:34:51AM -0500, Robert Haas wrote: >> On Mon, Dec 19, 2011 at 9:14 AM, Stephen Frost <sfr...@snowman.net> wrote: >> > * Aidan Van Dyk (ai...@highrise.ca) wrote: >> >> But the scary part is you don't know how long *ago* the crash was. >> >> Because a hint-bit-only change w/ a torn-page is a "non event" in >> >> PostgreSQL *DESIGN*, on crash recovery, it doesn't do anything to try >> >> and "scrub" every page in the database. >> > >> > Fair enough, but, could we distinguish these two cases? In other words, >> > would it be possible to detect if a page was torn due to a 'traditional' >> > crash and not complain in that case, but complain if there's a CRC >> > failure and it *doesn't* look like a torn page? >> >> No. > > Would you be so kind as to elucidate this a bit?
Well, basically, Stephen's proposal was pure hand-waving. :-) I don't know of any magic trick that would allow us to know whether a CRC failure "looks like a torn page". The only information we're going to get is the knowledge of whether the CRC matches or not. If it doesn't, it's fundamentally impossible for us to know why. We know the page contents are not as expected - that's it! It's been proposed before that we could examine the page, consider all the unset hint bits that could be set, and try all combinations of setting and clearing them to see whether any of them produce a valid CRC. But, as Tom has pointed out previously, that has a really quite large chance of making a page that's *actually* been corrupted look OK. If you have 30 or so unset hint bits, odds are very good that some combination will produce the 32-CRC you're expecting. To put this another way, we currently WAL-log just about everything. We get away with NOT WAL-logging some things when we don't care about whether they make it to disk. Hint bits, killed index tuple pointers, etc. cause no harm if they don't get written out, even if some other portion of the same page does get written out. But as soon as you CRC the whole page, now absolutely every single bit on that page becomes critical data which CANNOT be lost. IOW, it now requires the same sort of protection that we already need for our other critical updates - i.e. WAL logging. Or you could introduce some completely new mechanism that serves the same purpose, like MySQL's double-write buffer. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
