Re: [HACKERS] Remove secondary checkpoint

2017-11-07 Thread Michael Paquier
On Wed, Nov 8, 2017 at 2:23 AM, Simon Riggs wrote: > On 31 October 2017 at 12:01, Michael Paquier > wrote: >> While the mention about a manual checkpoint happening after a timed >> one will cause a full range of WAL segments to be recycled, it

Re: [HACKERS] Remove secondary checkpoint

2017-11-07 Thread Simon Riggs
On 31 October 2017 at 12:01, Michael Paquier wrote: > On Tue, Oct 31, 2017 at 2:00 PM, Simon Riggs wrote: >> On 30 October 2017 at 18:58, Simon Riggs wrote: >>> On 30 October 2017 at 15:22, Simon Riggs

Re: [HACKERS] Remove secondary checkpoint

2017-10-31 Thread Michael Paquier
On Tue, Oct 31, 2017 at 2:00 PM, Simon Riggs wrote: > On 30 October 2017 at 18:58, Simon Riggs wrote: >> On 30 October 2017 at 15:22, Simon Riggs wrote: >> You forgot to update this formula in xlog.c: distance =

Re: [HACKERS] Remove secondary checkpoint

2017-10-31 Thread Simon Riggs
On 30 October 2017 at 18:58, Simon Riggs wrote: > On 30 October 2017 at 15:22, Simon Riggs wrote: > >>> You forgot to update this formula in xlog.c: >>> distance = (2.0 + CheckPointCompletionTarget) * >>> CheckPointDistanceEstimate; >>> /*

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 15:22, Simon Riggs wrote: >> You forgot to update this formula in xlog.c: >> distance = (2.0 + CheckPointCompletionTarget) * >> CheckPointDistanceEstimate; >> /* add 10% for good measure. */ >> distance *= 1.10; >> You can guess easily

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Andres Freund
On 2017-10-30 10:10:19 -0400, Tom Lane wrote: > Robert Haas writes: > > I was mostly just thinking out loud, listing another option rather > > than advocating for it. > > FWIW, I just wanted the question to be debated and resolved properly. > > After rereading the thread

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 15:31, Michael Paquier wrote: > On Mon, Oct 30, 2017 at 2:22 PM, Simon Riggs wrote: >> On 25 October 2017 at 00:17, Michael Paquier >> wrote: >>> -* Delete old log files (those no longer

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 2:22 PM, Simon Riggs wrote: > On 25 October 2017 at 00:17, Michael Paquier > wrote: >> -* Delete old log files (those no longer needed even for previous >> -* checkpoint or the standbys in XLOG streaming). >> +

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 25 October 2017 at 00:17, Michael Paquier wrote: > On Mon, Oct 23, 2017 at 11:20 PM, Simon Riggs wrote: >> Remove the code that maintained two checkpoint's WAL files and all >> associated stuff. >> >> Try to avoid breaking anything else >> >>

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Tom Lane
Robert Haas writes: > I was mostly just thinking out loud, listing another option rather > than advocating for it. FWIW, I just wanted the question to be debated and resolved properly. After rereading the thread Andres pointed to, I thought of a hazard that I think Andres

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 7:04 PM, Alvaro Herrera wrote: >> A sort of middle way would be to keep the secondary checkpoint around >> but never try to replay from that point, or only if a specific flag is >> provided. > > Why do you want to keep the secondary checkpoint? If

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 7:04 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Oct 24, 2017 at 7:25 PM, Andres Freund wrote: >> > I think it does the contrary. The current mechanism is, in my opinion, >> > downright dangerous: >> >

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Oct 24, 2017 at 7:25 PM, Andres Freund wrote: > > I think it does the contrary. The current mechanism is, in my opinion, > > downright dangerous: > > https://www.postgresql.org/message-id/20160201235854.go8...@awork2.anarazel.de > > A sort of

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Robert Haas
On Tue, Oct 24, 2017 at 7:25 PM, Andres Freund wrote: > I think it does the contrary. The current mechanism is, in my opinion, > downright dangerous: > https://www.postgresql.org/message-id/20160201235854.go8...@awork2.anarazel.de A sort of middle way would be to keep the

Re: [HACKERS] Remove secondary checkpoint

2017-10-26 Thread Tsunakawa, Takayuki
From: Alvaro Herrera [mailto:alvhe...@alvh.no-ip.org] > Tsunakawa, Takayuki wrote: > > > (Although unrelated to this, I've also been wondering why PostgreSQL > > flushes WAL to disk when writing a page in the shared buffer, because > > PostgreSQL doesn't use WAL for undo.) > > The reason is that

Re: [HACKERS] Remove secondary checkpoint

2017-10-26 Thread Alvaro Herrera
Tsunakawa, Takayuki wrote: > (Although unrelated to this, I've also been wondering why PostgreSQL > flushes WAL to disk when writing a page in the shared buffer, because > PostgreSQL doesn't use WAL for undo.) The reason is that if the system crashes after writing the data page to disk, but

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Michael Paquier
On Tue, Oct 24, 2017 at 7:24 PM, Tsunakawa, Takayuki wrote: > (3) > Should we change the default value of max_wal_size from 1 GB to a smaller > size? I vote for "no" for performance. The default has just changed in v10, so changing it again could be confusing,

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Simon Riggs > This > * reduces disk space requirements on master > * removes a minor bug in fast failover > * simplifies code I welcome this patch. I was wondering why PostgreSQL retains the

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Michael Paquier > On Tue, Oct 24, 2017 at 5:58 PM, Tsunakawa, Takayuki > wrote: > > If the latest checkpoint record is unreadable (the WAL > segment/block/record is

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Michael Paquier
On Tue, Oct 24, 2017 at 5:58 PM, Tsunakawa, Takayuki wrote: > If the latest checkpoint record is unreadable (the WAL segment/block/record > is corrupt?), recovery from the previous checkpoint would also stop at the > latest checkpoint. And we don't need to

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tom Lane > Doesn't it also make crash recovery less robust? The whole point > of that mechanism is to be able to cope if the latest checkpoint > record is unreadable. If the latest checkpoint

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Michael Paquier
On Mon, Oct 23, 2017 at 11:20 PM, Simon Riggs wrote: > Remove the code that maintained two checkpoint's WAL files and all > associated stuff. > > Try to avoid breaking anything else > > This > * reduces disk space requirements on master > * removes a minor bug in fast

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Simon Riggs
On 24 October 2017 at 09:50, Tom Lane wrote: > Simon Riggs writes: >> Remove the code that maintained two checkpoint's WAL files and all >> associated stuff. > >> Try to avoid breaking anything else > >> This >> * reduces disk space requirements on

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Andres Freund
On 2017-10-24 09:50:12 -0400, Tom Lane wrote: > Simon Riggs writes: > > Remove the code that maintained two checkpoint's WAL files and all > > associated stuff. > > > Try to avoid breaking anything else > > > This > > * reduces disk space requirements on master > > *

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Tom Lane
Simon Riggs writes: > Remove the code that maintained two checkpoint's WAL files and all > associated stuff. > Try to avoid breaking anything else > This > * reduces disk space requirements on master > * removes a minor bug in fast failover > * simplifies code Doesn't it