Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-07 Thread Max Vlasov
On Sat, Feb 4, 2012 at 3:51 PM, Alexey Pechnikov wrote: > It's very important but there are some questions about > http://www.sqlite.org/src/info/72b01a982a > Some times ago DRH wrote that checksum calculation don't slow down > SQLite significantly. > But can be this

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-07 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/12 12:14, Nico Williams wrote: > You need to be able to store the checksums where the page pointers are > stored. The page containing the page pointers is itself checksummed. While there are still some extremely rare cases that isn't

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-07 Thread Nico Williams
On Tue, Feb 7, 2012 at 11:12 AM, Michael Stephenson wrote: > It's almost trivial to add per-page checksums at the page level.  Here are > basic steps: This is not enough, though it's a lot better than nothing. You need to be able to store the checksums where the page

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-07 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/12 09:12, Michael Stephenson wrote: > It's almost trivial to add per-page checksums at the page level. You omitted the steps of extensive testing and documentation :-) > Given the above, would per-page checksums be better served up as an >

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-07 Thread Michael Stephenson
. -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Nico Williams Sent: Monday, February 06, 2012 4:49 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] About new ticket "Per page/region checksums" On Mon, Feb 6,

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-07 Thread Alexey Pechnikov
Roger, it's good to know the reasons of SQLITE_CORRUPT signal. And very useful is possibility to mark all corrupted database pages as unused in repair process and so make the database workable again. And this repair may be processed only by SQLite internals because is needed ignore corrupted pages

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Simon Slavin
On 6 Feb 2012, at 11:31pm, Roger Binns wrote: > [nicely rebuffed my earlier post] Okay, you convinced me. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/02/12 14:38, Simon Slavin wrote: > File systems (FAT, NTFS, HTFS) already have block checksums. Huh? FAT has a checksum on the super block but not on datablocks. NTFS does not have block checksums. That is why they are introducing ReFS. What

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Nico Williams
On Mon, Feb 6, 2012 at 4:38 PM, Simon Slavin wrote: > On 6 Feb 2012, at 9:49pm, Nico Williams wrote: >> Encryption is not enough.  You really need block pointers to carry the >> block checksum/hash/MAC/integrity tag. > > File systems (FAT, NTFS, HTFS) already have block

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Simon Slavin
On 6 Feb 2012, at 9:49pm, Nico Williams wrote: > On Mon, Feb 6, 2012 at 2:27 PM, Roger Binns wrote: >> Approximately zero percent of SQLite databases will be on btrfs for the >> foreseeable future, even if you and I start using btrfs. Android 3 and >> above do support

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Nico Williams
On Mon, Feb 6, 2012 at 2:27 PM, Roger Binns wrote: > On 06/02/12 11:35, Nico Williams wrote: >> Indeed, but if you'd do integrity protection at the application layer >> then surely you'd have have a backup/restore strategy to deal with >> lower-layer corruption. > > Only if

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/02/12 11:35, Nico Williams wrote: > Indeed, but if you'd do integrity protection at the application layer > then surely you'd have have a backup/restore strategy to deal with > lower-layer corruption. Only if you know about that corruption!

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Nico Williams
On Mon, Feb 6, 2012 at 1:20 PM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 06/02/12 09:11, Nico Williams wrote: >> A checksum per-row is certainly a valuable thing at the layer above the >> RDBMS since it allows for integrity checking above

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/02/12 09:11, Nico Williams wrote: > A checksum per-row is certainly a valuable thing at the layer above the > RDBMS since it allows for integrity checking above the RBDMS, and in an > RBDMS-independent manner. It doesn't actually help that much

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Nico Williams
A checksum per-row is certainly a valuable thing at the layer above the RDBMS since it allows for integrity checking above the RBDMS, and in an RBDMS-independent manner. Of course, this approach is easiest to graft into SQLite3 simply because you'd be adding a hidden column and the machinery for

Re: [sqlite] About new ticket "Per page/region checksums"

2012-02-06 Thread Andy Gibbs
On 4th February 2012 12:51pm, Alexey Pechnikov wrote: > It's very important but there are some questions about > http://www.sqlite.org/src/info/72b01a982a > Some times ago DRH wrote that checksum calculation don't slow down > SQLite significantly. > But can be this realized in current SQLite 3.x

[sqlite] About new ticket "Per page/region checksums"

2012-02-04 Thread Alexey Pechnikov
It's very important but there are some questions about http://www.sqlite.org/src/info/72b01a982a Some times ago DRH wrote that checksum calculation don't slow down SQLite significantly. But can be this realized in current SQLite 3.x branch? When checksum can help to restore damaged pages/database?