Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-22 Thread Nathan Bossart
On Wed, Feb 22, 2023 at 12:30:20PM +0900, Michael Paquier wrote: > Okay, applied and backpatched with a minimal test set, then. I have > kept the tweaks I did to the tests with extra comments. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-21 Thread Michael Paquier
On Tue, Feb 21, 2023 at 10:00:11AM -0800, Nathan Bossart wrote: > What is the purpose of testing it without the checkpoint? Perhaps none, I was wondering whether it would be worth testing that with the flush phase, but perhaps that's just extra cycles wasted at this point. > Other than that > que

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-21 Thread Nathan Bossart
On Tue, Feb 21, 2023 at 03:13:10PM +0900, Michael Paquier wrote: > The test enforces a checkpoint after the table creation on the > template, so what about testing it also without a checkpoint, like the > extended version attached? I have tweaked a few things in the test, > while on it. What is t

RE: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-21 Thread Ryo Matsumura (Fujitsu)
> On Thu, Feb 16, 2023 at 10:24:13AM +0530, Dilip Kumar wrote: > > Yes, there is no reason to pass this as false, seems like this is > > passed false by mistake. And your patch fixes the issue. On Thu, Feb 16, 2023 at 02:26:55PM +0900, Michael Paquier wrote: > So, if I am understanding this stuff

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Michael Paquier
On Mon, Feb 20, 2023 at 04:43:22PM -0800, Nathan Bossart wrote: > On Mon, Feb 20, 2023 at 05:02:01PM +0900, Michael Paquier wrote: >> On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: >>> I'm happy to move this new test to wherever folks think it should go. I'll >>> look around to se

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Nathan Bossart
On Mon, Feb 20, 2023 at 05:02:01PM +0900, Michael Paquier wrote: > On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: >> I'm happy to move this new test to wherever folks think it should go. I'll >> look around to see if I can find a better place, too. > > I think that src/test/recov

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Michael Paquier
On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: > I'm happy to move this new test to wherever folks think it should go. I'll > look around to see if I can find a better place, too. I think that src/test/recovery/ is the best fit, because this stresses a code path for WAL replay on

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-17 Thread Nathan Bossart
On Fri, Feb 17, 2023 at 03:13:32PM +0100, Peter Eisentraut wrote: > On 16.02.23 22:29, Andres Freund wrote: >> What's the story behind 100_bugs.pl? This name clearly is copied from >> src/test/subscription/t/100_bugs.pl - but I've never understood why that is >> outside of the normal numbering spac

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-17 Thread Peter Eisentraut
On 16.02.23 22:29, Andres Freund wrote: What's the story behind 100_bugs.pl? This name clearly is copied from src/test/subscription/t/100_bugs.pl - but I've never understood why that is outside of the normal numbering space. Mainly to avoid awkwardness for backpatching. The number of tests in

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-16 Thread Amit Kapila
On Fri, Feb 17, 2023 at 2:59 AM Andres Freund wrote: > > On 2023-02-16 12:37:57 +0530, Robert Haas wrote: > > The patch creates 100_bugs.pl > > What's the story behind 100_bugs.pl? This name clearly is copied from > src/test/subscription/t/100_bugs.pl - but I've never understood why that is > outs

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-16 Thread Andres Freund
On 2023-02-16 12:37:57 +0530, Robert Haas wrote: > The patch creates 100_bugs.pl What's the story behind 100_bugs.pl? This name clearly is copied from src/test/subscription/t/100_bugs.pl - but I've never understood why that is outside of the normal numbering space.

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Dilip Kumar
On Thu, Feb 16, 2023 at 12:38 PM Robert Haas wrote: > > On Thu, Feb 16, 2023 at 12:32 PM Dilip Kumar wrote: > > I think the below change is not relevant to this bug right? > > > > diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build > > index 209118a639..6e9f8a7c7f 100644 >

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Michael Paquier
On Thu, Feb 16, 2023 at 12:37:57PM +0530, Robert Haas wrote: > Why not? The patch creates 100_bugs.pl so it also adds it to meson.build. It would not matter for REL_15_STABLE, but on HEAD all the new TAP test files have to be added in their respective meson.build. If you don't do that, the CFBot

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Robert Haas
On Thu, Feb 16, 2023 at 12:32 PM Dilip Kumar wrote: > I think the below change is not relevant to this bug right? > > diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build > index 209118a639..6e9f8a7c7f 100644 > --- a/src/test/recovery/meson.build > +++ b/src/test/recovery/mes

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Dilip Kumar
On Thu, Feb 16, 2023 at 12:11 PM Nathan Bossart wrote: > > On Thu, Feb 16, 2023 at 02:26:55PM +0900, Michael Paquier wrote: > > So, if I am understanding this stuff right, this issue can create data > > corruption once a DDL updates any pages of pg_class stored in a > > template database that gets

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Nathan Bossart
On Thu, Feb 16, 2023 at 02:26:55PM +0900, Michael Paquier wrote: > So, if I am understanding this stuff right, this issue can create data > corruption once a DDL updates any pages of pg_class stored in a > template database that gets copied by this routine. In this case, the > patch sent makes sur

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Michael Paquier
On Thu, Feb 16, 2023 at 10:24:13AM +0530, Dilip Kumar wrote: > Yes, there is no reason to pass this as false, seems like this is > passed false by mistake. And your patch fixes the issue. So, if I am understanding this stuff right, this issue can create data corruption once a DDL updates any page

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Dilip Kumar
On Thu, Feb 16, 2023 at 5:37 AM Nathan Bossart wrote: > > On Wed, Feb 15, 2023 at 04:49:38AM +, Ryo Matsumura (Fujitsu) wrote: > > The above is occured by the following call. > > The argument 'permanent' of ReadBufferWithoutRelcache() is passed to > > BufferAlloc() as 'relpersistence'. > > > >

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Nathan Bossart
On Wed, Feb 15, 2023 at 04:49:38AM +, Ryo Matsumura (Fujitsu) wrote: > The above is occured by the following call. > The argument 'permanent' of ReadBufferWithoutRelcache() is passed to > BufferAlloc() as 'relpersistence'. > > [src/backend/commands/] > 298 buf = ReadBufferWithoutRelca

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Justin Pryzby
On Wed, Feb 15, 2023 at 04:49:38AM +, Ryo Matsumura (Fujitsu) wrote: > Hi, hackers. > > I found that CREATE DATABASE occurs lost of DDL result after crash server. > The direct cause is that the checkpoint skips sync for page of template1's > main fork > because buffer status is not marked as