Re: AW: AW: AW: AW: AW: AW: AW: broken backup trail in case of quickly patroni switchback and forth

2019-11-08 Thread Brad Nicholson
"Zwettler Markus (OIZ)"  wrote on 2019/11/08
11:54:14 AM:

> > (please don't top post, makes the replies hard to follow)
>
> > patronictl switchover 
>
> > follow the prompts
>
> > there is also a /switchover API endpoint you can use.
>
> > Brad
>
> I wondered about your "patronictl switchover + systemd" hint. How
> would you do ("gate") this combination?


Change whatever process you are using today to shut things down to call the
patroni switchover first, check error codes, etc.


AW: AW: AW: AW: AW: AW: AW: broken backup trail in case of quickly patroni switchback and forth

2019-11-08 Thread Zwettler Markus (OIZ)
?  "Zwettler Markus (OIZ)" 
mailto:markus.zwett...@zuerich.ch>> wrote on 
2019/11/08 11:27:00 AM:

>> From: "Zwettler Markus (OIZ)" 
>> mailto:markus.zwett...@zuerich.ch>>
>> To: Brad Nicholson mailto:br...@ca.ibm.com>>
>> Cc: Adrian Klaver 
>> mailto:adrian.kla...@aklaver.com>>, "pgsql-
>> gene...@lists.postgresql.org<mailto:gene...@lists.postgresql.org>" 
>> mailto:pgsql-general@lists.postgresql.org>>
>> Date: 2019/11/08 11:27 AM
>>> Subject: [EXTERNAL] AW:  AW:  AW:  AW: AW: AW: broken backup trail
>> in case of quickly patroni switchback and forth
>>
>> How exactly? Please clarify.

> (please don't top post, makes the replies hard to follow)

> patronictl switchover 

> follow the prompts

> there is also a /switchover API endpoint you can use.

> Brad



I wondered about your "patronictl switchover + systemd" hint. How would you do 
("gate") this combination?

Markus


Re: AW: AW: AW: AW: AW: AW: broken backup trail in case of quickly patroni switchback and forth

2019-11-08 Thread Brad Nicholson
"Zwettler Markus (OIZ)"  wrote on 2019/11/08
11:27:00 AM:

> From: "Zwettler Markus (OIZ)" 
> To: Brad Nicholson 
> Cc: Adrian Klaver , "pgsql-
> gene...@lists.postgresql.org" 
> Date: 2019/11/08 11:27 AM
> Subject: [EXTERNAL] AW:  AW:  AW:  AW: AW: AW: broken backup trail
> in case of quickly patroni switchback and forth
>
> How exactly? Please clarify.

(please don't top post, makes the replies hard to follow)

patronictl switchover 

follow the prompts

there is also a /switchover API endpoint you can use.

Brad


AW: AW: AW: AW: AW: AW: broken backup trail in case of quickly patroni switchback and forth

2019-11-08 Thread Zwettler Markus (OIZ)
How exactly? Please clarify.



"Zwettler Markus (OIZ)" 
mailto:markus.zwett...@zuerich.ch>> wrote on 
2019/11/08 11:02:49 AM:

> From: "Zwettler Markus (OIZ)" 
> mailto:markus.zwett...@zuerich.ch>>
> To: Brad Nicholson mailto:br...@ca.ibm.com>>
> Cc: Adrian Klaver 
> mailto:adrian.kla...@aklaver.com>>, "pgsql-
> gene...@lists.postgresql.org<mailto:gene...@lists.postgresql.org>" 
> mailto:pgsql-general@lists.postgresql.org>>
> Date: 2019/11/08 11:02 AM
> Subject: [EXTERNAL] AW:  AW:  AW: AW: AW: broken backup trail in
> case of quickly patroni switchback and forth
>
> Let me clarify: "But, it might start killing processes after a
> certain period if a _fast_ shutdown after SIGTERM didn't happen".
>
> I am talking about stopping the Patroni master process with a systemd scipt.


Use the switchover functionality in Patroni first, and gate youur shutdown via 
systemd on the success of that operation.

Brad.


Re: AW: AW: AW: AW: AW: broken backup trail in case of quickly patroni switchback and forth

2019-11-08 Thread Brad Nicholson
"Zwettler Markus (OIZ)"  wrote on 2019/11/08
11:02:49 AM:

> From: "Zwettler Markus (OIZ)" 
> To: Brad Nicholson 
> Cc: Adrian Klaver , "pgsql-
> gene...@lists.postgresql.org" 
> Date: 2019/11/08 11:02 AM
> Subject: [EXTERNAL] AW:  AW:  AW: AW: AW: broken backup trail in
> case of quickly patroni switchback and forth
>
> Let me clarify: "But, it might start killing processes after a
> certain period if a _fast_ shutdown after SIGTERM didn't happen".
>
> I am talking about stopping the Patroni master process with a systemd
scipt.


Use the switchover functionality in Patroni first, and gate youur shutdown
via systemd on the success of that operation.

Brad.


AW: AW: AW: AW: AW: broken backup trail in case of quickly patroni switchback and forth

2019-11-08 Thread Zwettler Markus (OIZ)
Let me clarify: "But, it might start killing processes after a certain period 
if a _fast_ shutdown after SIGTERM didn't happen".

I am talking about stopping the Patroni master process with a systemd scipt.



Von: Brad Nicholson 
Gesendet: Freitag, 8. November 2019 15:58
An: Zwettler Markus (OIZ) 
Cc: Adrian Klaver ; 
pgsql-general@lists.postgresql.org
Betreff: Re: AW: AW: AW: AW: broken backup trail in case of quickly patroni 
switchback and forth


"Zwettler Markus (OIZ)" 
mailto:markus.zwett...@zuerich.ch>> wrote on 
2019/11/08 07:51:33 AM:

> From: "Zwettler Markus (OIZ)" 
> mailto:markus.zwett...@zuerich.ch>>
> To: Brad Nicholson mailto:br...@ca.ibm.com>>
> Cc: Adrian Klaver 
> mailto:adrian.kla...@aklaver.com>>, "pgsql-
> gene...@lists.postgresql.org<mailto:gene...@lists.postgresql.org>" 
> mailto:pgsql-general@lists.postgresql.org>>
> Date: 2019/11/08 07:51 AM
> Subject: [EXTERNAL] AW:  AW: AW: AW: broken backup trail in case of
> quickly patroni switchback and forth
>
> It depends. It is a switchover if Patroni could to a clean shutdown.
> But, it might start killing processes after a certain period if a
> normal shutdown after SIGTERM didn't happen. This would not be a
> switchover anymore. In other words there is no guarantee for a
> "clean" switchover. This might be the reason why the Patroni guys
> are always talking about failover only.


If it can't do a clean shutdown, that points to something wrong with Postgres 
itself.  Why doesn't a fast shutdown work for you in those cases?

> It's not a Patroni issue but it's triggered by Patroni as it will do
> "some kind of switchover" on a regular shutdown.

Sure, but you should be looking at why Postgres can't cleanly shutdown.

How are you telling Patroni to switchover? Are you using the Patroni switchover 
command via patronictl or the API, or sending a signal to the Patroni process?  
I think the explicit switchover
command will not behave this way.  It will return you a 503 if it can't 
switchover and not change the primary (that is something you can confirm with 
the Patroni developers).

Brad.