-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/13/2010 05:30 PM, Jim Wilcoxson wrote: > I don't think this would work, because the problem described is that the > writes aren't making it to disk. If pages don't make it to disk, the old > pages will be present, with the old, and valid checksums.
You are assuming the checksums are stored in the page they checksum. That would only detect corruption of that page. You could have pages that store the checksums of numerous other pages, so both the checksum page and the data page would have to fail to make it to disk. Yes, there are scenarios where you could still get old apparently valid pages, but those are harder to happen. > SQLite would have to verify the checksum on every page when the > database is opened and a hot journal exists, which could be quite a lot of > overhead for a large database. I would want checksums looked at when reading pages always. For more valuable data I would want a thorough open check. I already use the pragmas (integrity/quick check) on opens as appropriate. > I think a checksum on every page, and maybe even an error-correction code, > is a great idea as an optional feature. It would have to be optional, much like WAL for backwards compatibility issues, and because not everyone would want this. > But it would only detect hardware problems and bit rot. Plus normal errors, such as the undetectable one every 20TB of I/O rate I hope I remembered correctly from the ZFS paper. I don't know what the error rate of Flash is, nor how that interacts with SATA signalling error rates, but all these numbers are non-zero and we keep doing more and more I/O, have larger datasets, and have things around for longer. Plus we do like buying cheap storage :-) > This problem of not doing writes, or doing them in > the wrong order, is a different animal IMO. There will always be loopholes you could construct with that that are very hard to detect. Hopefully checksumming or some sort of similar data integrity scheme would make it easier to detect some of the scenarios. ie things could be made better than they are now. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkw9TJYACgkQmOOfHg372QRnkACfdJp+FZHvxXeLNN0DMCHSNJ1+ 3E8AnR1PXa4PTQ1ridHBaAC7WpSSwnyM =5sbp -----END PGP SIGNATURE----- _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
