Re: [patch] demote

2020-09-01 Thread Jehan-Guillaume de Rorthais
On Tue, 18 Aug 2020 17:41:31 +0200 Jehan-Guillaume de Rorthais wrote: > Hi, > > Please find in attachment v5 of the patch set rebased on master after various > conflicts. > > Regards, > > On Wed, 5 Aug 2020 00:04:53 +0200 > Jehan-Guillaume de Rorthais wrote: > > > Demote now keeps backends

Re: [patch] demote

2020-08-18 Thread Jehan-Guillaume de Rorthais
Hi, Please find in attachment v5 of the patch set rebased on master after various conflicts. Regards, On Wed, 5 Aug 2020 00:04:53 +0200 Jehan-Guillaume de Rorthais wrote: > Demote now keeps backends with no active xid alive. Smart mode keeps all > backends: it waits for them to finish their

Re: [patch] demote

2020-08-04 Thread Jehan-Guillaume de Rorthais
Hi, Yet another summary + patch + tests. Demote now keeps backends with no active xid alive. Smart mode keeps all backends: it waits for them to finish their xact and enter read-only. Fast mode terminate backends wit an active xid and keeps all other ones. Backends enters "read-only" using

Re: [patch] demote

2020-07-14 Thread Jehan-Guillaume de Rorthais
On Tue, 14 Jul 2020 12:49:51 -0700 Andres Freund wrote: > Hi, > > On 2020-07-14 17:26:37 +0530, Amul Sul wrote: > > On Mon, Jul 13, 2020 at 8:35 PM Jehan-Guillaume de Rorthais > > wrote: > > > > > > Hi, > > > > > > Another summary + patch + tests. > > > > > > This patch supports 2PC. The

Re: [patch] demote

2020-07-14 Thread Andres Freund
Hi, On 2020-07-14 17:26:37 +0530, Amul Sul wrote: > On Mon, Jul 13, 2020 at 8:35 PM Jehan-Guillaume de Rorthais > wrote: > > > > Hi, > > > > Another summary + patch + tests. > > > > This patch supports 2PC. The goal is to keep them safe during demote/promote > > actions so they can be

Re: [patch] demote

2020-07-14 Thread Amul Sul
On Mon, Jul 13, 2020 at 8:35 PM Jehan-Guillaume de Rorthais wrote: > > Hi, > > Another summary + patch + tests. > > This patch supports 2PC. The goal is to keep them safe during demote/promote > actions so they can be committed/rollbacked later on a primary. See tests. > Wondering is it

Re: [patch] demote

2020-07-13 Thread Jehan-Guillaume de Rorthais
Hi, Another summary + patch + tests. This patch supports 2PC. The goal is to keep them safe during demote/promote actions so they can be committed/rollbacked later on a primary. See tests. The checkpointer is now shutdowned after the demote shutdown checkpoint. It removes some useless code

Re: [patch] demote

2020-07-02 Thread Jehan-Guillaume de Rorthais
Hi, Here is a small activity summary since last report. On Thu, 25 Jun 2020 19:27:54 +0200 Jehan-Guillaume de Rorthais wrote: [...] > I hadn't time to investigate Robert's concern about shared memory for snapshot > during recovery. I hadn't time to dig very far, but I suppose this might be

Re: [patch] demote

2020-07-01 Thread Jehan-Guillaume de Rorthais
On Fri, 26 Jun 2020 16:14:38 +0900 (JST) Kyotaro Horiguchi wrote: > Hello. > > At Thu, 25 Jun 2020 19:27:54 +0200, Jehan-Guillaume de Rorthais > wrote in > > Here is a summary of my work during the last few days on this demote > > approach. > > > > Please, find in attachment

Re: [patch] demote

2020-06-26 Thread Kyotaro Horiguchi
Mmm. Fat finger.. At Fri, 26 Jun 2020 16:14:38 +0900 (JST), Kyotaro Horiguchi wrote in > Hello. > > If we are going to dive so close to server shutdown, we can just > utilize the restart-after-crash path, which we can assume to work > reliably. The attached is a quite rough sketch, hijacking

Re: [patch] demote

2020-06-26 Thread Kyotaro Horiguchi
Hello. At Thu, 25 Jun 2020 19:27:54 +0200, Jehan-Guillaume de Rorthais wrote in > Here is a summary of my work during the last few days on this demote approach. > > Please, find in attachment v2-0001-Demote-PoC.patch and the comments in the > commit message and as FIXME in code. > > The

Re: [patch] demote

2020-06-25 Thread Jehan-Guillaume de Rorthais
Hi, Here is a summary of my work during the last few days on this demote approach. Please, find in attachment v2-0001-Demote-PoC.patch and the comments in the commit message and as FIXME in code. The patch is not finished or bug-free yet, I'm still not very happy with the coding style, it

Re: [patch] demote

2020-06-18 Thread Robert Haas
On Thu, Jun 18, 2020 at 1:24 PM Tom Lane wrote: > It seems like this is the core decision that needs to be taken. If > we're willing to have these state transitions include a server restart, > then many things get simpler. If we're not, it's gonna cost us in > code complexity and hence bugs.

Re: [patch] demote

2020-06-18 Thread Andres Freund
Hi, On 2020-06-18 13:24:38 -0400, Tom Lane wrote: > Robert Haas writes: > > ... To go back to recovery rather than just to a read-only > > state, I think you'd need to grapple with some additional issues that > > patch doesn't touch, like some of the snapshot-taking stuff, but I > > think you

Re: [patch] demote

2020-06-18 Thread Tom Lane
Robert Haas writes: > ... To go back to recovery rather than just to a read-only > state, I think you'd need to grapple with some additional issues that > patch doesn't touch, like some of the snapshot-taking stuff, but I > think you still need to solve all of the problems that it does deal >

Re: [patch] demote

2020-06-18 Thread Robert Haas
On Thu, Jun 18, 2020 at 12:55 PM Fujii Masao wrote: > Even with ASRO, the server restart is necessary and RO sessions are > kicked out when demoting RO primary to a standby, i.e., during a clean > switchover? The ASRO patch doesn't provide a way to put a running server to be put back into

Re: [patch] demote

2020-06-18 Thread Andres Freund
Hi, On 2020-06-18 21:41:45 +0900, Fujii Masao wrote: > ISTM that a clean switchover is possible without "ALTER SYSTEM READ ONLY". > What about the following procedure? > > 1. Demote the primary to a standby. Then this demoted standby is read-only. As far as I can tell this step includes ALTER

Re: [patch] demote

2020-06-18 Thread Andres Freund
Hi, On 2020-06-18 12:16:27 +0200, Jehan-Guillaume de Rorthais wrote: > On Wed, 17 Jun 2020 11:14:47 -0700 > > I don't think there's a fundamental issue, but I think it needs to deal > > with a lot more things than it does right now. StartupXLOG doesn't > > currently deal correctly with subsystems

Re: [patch] demote

2020-06-18 Thread Fujii Masao
On 2020/06/19 0:22, Robert Haas wrote: On Thu, Jun 18, 2020 at 8:41 AM Fujii Masao wrote: ISTM that a clean switchover is possible without "ALTER SYSTEM READ ONLY". What about the following procedure? 1. Demote the primary to a standby. Then this demoted standby is read-only. 2. The

Re: [patch] demote

2020-06-18 Thread Robert Haas
On Thu, Jun 18, 2020 at 11:56 AM Jehan-Guillaume de Rorthais wrote: > Considering the current demote patch improvement. I was considering to digg in > the following direction: > > * add a new state in the state machine where all backends are idle > * this new state forbid any new writes, the same

Re: [patch] demote

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Thu, 18 Jun 2020 11:22:47 -0400 Robert Haas wrote: > On Thu, Jun 18, 2020 at 8:41 AM Fujii Masao > wrote: > > ISTM that a clean switchover is possible without "ALTER SYSTEM READ ONLY". > > What about the following procedure? > > > > 1. Demote the primary to a standby. Then this demoted

Re: [patch] demote

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Thu, 18 Jun 2020 11:15:02 -0400 Robert Haas wrote: > On Thu, Jun 18, 2020 at 6:02 AM Jehan-Guillaume de Rorthais > wrote: > > At some expense, Admin can already set the system as readonly from the > > application point of view, using: > > > > alter system set default_transaction_read_only

Re: [patch] demote

2020-06-18 Thread Robert Haas
On Thu, Jun 18, 2020 at 8:41 AM Fujii Masao wrote: > ISTM that a clean switchover is possible without "ALTER SYSTEM READ ONLY". > What about the following procedure? > > 1. Demote the primary to a standby. Then this demoted standby is read-only. > 2. The orignal standby automatically establishes

Re: [patch] demote

2020-06-18 Thread Robert Haas
On Thu, Jun 18, 2020 at 6:02 AM Jehan-Guillaume de Rorthais wrote: > At some expense, Admin can already set the system as readonly from the > application point of view, using: > > alter system set default_transaction_read_only TO on; > select pg_reload_conf(); > > Current RW xact will finish,

Re: [patch] demote

2020-06-18 Thread Fujii Masao
On 2020/06/18 1:29, Robert Haas wrote: On Wed, Jun 17, 2020 at 11:45 AM Jehan-Guillaume de Rorthais wrote: As Amul sent a patch about "ALTER SYSTEM READ ONLY"[1], with similar futur objectives than mine, I decided to share the humble patch I am playing with to step down an instance from

Re: [patch] demote

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Wed, 17 Jun 2020 11:14:47 -0700 Andres Freund wrote: > Hi, > > On 2020-06-17 17:44:51 +0200, Jehan-Guillaume de Rorthais wrote: > > As Amul sent a patch about "ALTER SYSTEM READ ONLY"[1], with similar futur > > objectives than mine, I decided to share the humble patch I am playing with > >

Re: [patch] demote

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Wed, 17 Jun 2020 12:29:31 -0400 Robert Haas wrote: [...] > > Main difference with Amul's patch is that all backends must be disconnected > > to process with the demote. Either we wait for them to disconnect (smart) > > or we kill them (fast). This makes life much easier from the code point of

Re: [patch] demote

2020-06-17 Thread Andres Freund
Hi, On 2020-06-17 17:44:51 +0200, Jehan-Guillaume de Rorthais wrote: > As Amul sent a patch about "ALTER SYSTEM READ ONLY"[1], with similar futur > objectives than mine, I decided to share the humble patch I am playing with to > step down an instance from primary to standby status. To make sure

Re: [patch] demote

2020-06-17 Thread Robert Haas
On Wed, Jun 17, 2020 at 11:45 AM Jehan-Guillaume de Rorthais wrote: > As Amul sent a patch about "ALTER SYSTEM READ ONLY"[1], with similar futur > objectives than mine, I decided to share the humble patch I am playing with to > step down an instance from primary to standby status. Cool! This was