Re: basebackup checksum verification

2019-03-28 Thread Michael Paquier
On Tue, Mar 26, 2019 at 05:23:01PM -0700, Peter Geoghegan wrote: > I have personally seen real world corruption that involved a page > image consisting of random noise. Several times. Failing to detect > blatant corruption is unacceptable IMV. Yeah, I have seen that as well. If we have a tool not

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-03-26 21:01:27 -0400, Stephen Frost wrote: > > I'm also not convinced that these changes to pg_basebackup will be free > > of issues that may impact users in a negative way, making me concerned > > that we're going to end up doing mo

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 21:01:27 -0400, Stephen Frost wrote: > I'm also not convinced that these changes to pg_basebackup will be free > of issues that may impact users in a negative way, making me concerned > that we're going to end up doing more harm than good with such a change > being back-patched.

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-03-26 20:18:31 -0400, Stephen Frost wrote: > > > >>I thought Robert's response was generally good, pointing out that > > > >>we're talking about this being an issue if the corruption happens in a > > > >>certain set of bytes. That s

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
-03-26 19:22:03 -0400, Stephen Frost wrote: > >>>>* Andres Freund (and...@anarazel.de) wrote: > >>>>> As detailed in > >>>>> https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > >>>>> the way the backend&#

Re: basebackup checksum verification

2019-03-26 Thread Tomas Vondra
e: >>> As detailed in >>> https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de >>> the way the backend's basebackup checksum verification works makes its >>> error detection capabilities very dubious. >> >>I disagree that it

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 20:18:31 -0400, Stephen Frost wrote: > > >>I thought Robert's response was generally good, pointing out that > > >>we're talking about this being an issue if the corruption happens in a > > >>certain set of bytes. That said, I'm happy to see improvements in > > >>this area but I

Re: basebackup checksum verification

2019-03-26 Thread Peter Geoghegan
On Tue, Mar 26, 2019 at 5:10 PM Tomas Vondra wrote: > Bogus might be a bit too harsh, but yeah - failure to reliably detect > obviously > invalid checksums when the LSN just happens to be high due to randomness is > not > a good thing. We'll still detect pages corrupted in other places, but this

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
gt;> https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > >>> the way the backend's basebackup checksum verification works makes its > >>> error detection capabilities very dubious. > >> > >>I disagree that it's 'very dubiou

Re: basebackup checksum verification

2019-03-26 Thread Tomas Vondra
backend's basebackup checksum verification works makes its > error detection capabilities very dubious. I disagree that it's 'very dubious', even with your analysis. I really don't know what to say. The current algorithm is flat out bogus. Bogus might be a bit too

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > As detailed in > > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > > the way the backend's basebackup checksum verifi

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > As detailed in > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > the way the backend's basebackup checksum verification works makes its > error detection capabilities very dubious. I disagre

basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, As detailed in https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de the way the backend's basebackup checksum verification works makes its error detection capabilities very dubious. I think we need to fix this before the next set of backbranch releases, or at the