Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-25 Thread Simon Riggs
On 25 January 2013 12:15, Heikki Linnakangas wrote: >> 1) an immediate checkpoint can cause a disk/resource usage spike, >> which is definitely not what you need just when a spike of connections >> and new SQL hits the system. > > > It doesn't need to be an "immediate" checkpoint, ie. you don't n

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-25 Thread Tom Lane
Heikki Linnakangas writes: > There's no hard correctness reason here for any particular behavior, I > just feel that that would make most sense. It seems prudent to initiate > a checkpoint right after timeline switch, so that you get a new > checkpoint on the new timeline fairly soon - it could

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-25 Thread Heikki Linnakangas
On 24.01.2013 19:44, Simon Riggs wrote: On 24 January 2013 16:52, Heikki Linnakangas wrote: I may be missing something, but it looks like after a "fast" promotion, you don't request a new checkpoint. So it can take quite a while for the next checkpoint to be triggered by checkpoint_timeout/segm

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Simon Riggs
On 24 January 2013 17:44, Simon Riggs wrote: >> At replay, an end-of-recovery record should be a signal to the hot standby >> mechanism that there are no transactions running in the master at that >> point, same as a shutdown checkpoint. > > I had a reason why I didn't do that, but it seems to ha

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Simon Riggs
On 24 January 2013 16:52, Heikki Linnakangas wrote: > On 24.01.2013 18:24, Simon Riggs wrote: >> >> On 6 January 2013 21:58, Simon Riggs wrote: >>> >>> I've been torn between the need to remove the checkpoint for speed and >>> >>> being worried about the implications of doing so. >>> >>> We promo

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Heikki Linnakangas
On 24.01.2013 18:24, Simon Riggs wrote: On 6 January 2013 21:58, Simon Riggs wrote: I've been torn between the need to remove the checkpoint for speed and being worried about the implications of doing so. We promote in multiple use cases. When we end a PITR, or are performing a switchover, it

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Simon Riggs
On 6 January 2013 21:58, Simon Riggs wrote: > On 9 August 2012 10:45, Simon Riggs wrote: >> On 22 June 2012 05:03, Kyotaro HORIGUCHI >> wrote: >> >>>I hope this is promising. >> >> I've reviewed this and thought about it over some time. > > I've been torn between the need to remove the check

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-06 Thread Simon Riggs
On 9 August 2012 10:45, Simon Riggs wrote: > On 22 June 2012 05:03, Kyotaro HORIGUCHI > wrote: > >>I hope this is promising. > > I've reviewed this and thought about it over some time. I've been torn between the need to remove the checkpoint for speed and being worried about the implications

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-10-18 Thread Simon Riggs
On 18 October 2012 21:22, Alvaro Herrera wrote: > This patch seems to have been neglected by both its submitter and the > reviewer. Also, Simon said he was going to set it > returned-with-feedback on his last reply, but I see it as needs-review > still in the CF app. Is this something that is g

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-10-18 Thread Alvaro Herrera
This patch seems to have been neglected by both its submitter and the reviewer. Also, Simon said he was going to set it returned-with-feedback on his last reply, but I see it as needs-review still in the CF app. Is this something that is going to be reconsidered and resubmitted for the next commi

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-08-30 Thread Kyotaro HORIGUCHI
Hello, sorry for long absense. > At first I was unhappy that you'd removed the restriction that > timelines only change on a shutdown checkpoint. But the reality is > timelines change at any point in the WAL stream - the only way to tell > between end of WAL and a timeline change is by looking for

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-08-09 Thread Simon Riggs
On 22 June 2012 05:03, Kyotaro HORIGUCHI wrote: >I hope this is promising. I've reviewed this and thought about it over some time. At first I was unhappy that you'd removed the restriction that timelines only change on a shutdown checkpoint. But the reality is timelines change at any point

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-21 Thread Kyotaro HORIGUCHI
Hello, > Is it guaranteed that all the files (e.g., the latest timeline history file) > required for such crash recovery exist in pg_xlog? If yes, your > approach might work well. Particularly regarding the promotion, the files reuiqred are the history file of the latest timeline, the WAL file in

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-19 Thread Fujii Masao
On Tue, Jun 19, 2012 at 5:30 PM, Kyotaro HORIGUCHI wrote: > Thank you. > >> What happens if the server skips an end-of-recovery checkpoint, >> is promoted to the master, runs some write transactions, >> crashes and restarts automatically before it completes >> checkpoint? In this case, the server

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-19 Thread Kyotaro HORIGUCHI
Thank you. > What happens if the server skips an end-of-recovery checkpoint, > is promoted to the master, runs some write transactions, > crashes and restarts automatically before it completes > checkpoint? In this case, the server needs to do crash recovery > from the last checkpoint record with

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-18 Thread Fujii Masao
On Mon, Jun 18, 2012 at 5:42 PM, Kyotaro HORIGUCHI wrote: > What do you think about this? What happens if the server skips an end-of-recovery checkpoint, is promoted to the master, runs some write transactions, crashes and restarts automatically before it completes checkpoint? In this case, the s

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-18 Thread Kyotaro HORIGUCHI
Hello, This is the new version of the patch. Your patch introduced new WAL record type XLOG_END_OF_RECOVERY to mark the chenge point of TLI. But I think the information is already stored in history files and already ready to use in current code. I looked into your first patch and looked over the

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-12 Thread Kyotaro HORIGUCHI
Hello, Thank you to head me the previous discussion. I'll consider them from now. > > I want the standby to start to serve as soon as possible even by > > a few seconds on failover in a HA cluster. > > Please implement a prototype and measure how many seconds we > are discussing. I'm sorry to ha

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-12 Thread Simon Riggs
On 12 June 2012 03:38, Kyotaro HORIGUCHI wrote: > Hello, sorry for vague understanding. > >> > I depend on this and suppose we can omit it if latest checkpoint >> > has been taken so as to be able to do crash recovery thereafter. >> >> I don't see any reason to special case this. If a checkpoint h

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-11 Thread Kyotaro HORIGUCHI
Hello, sorry for vague understanding. > > I depend on this and suppose we can omit it if latest checkpoint > > has been taken so as to be able to do crash recovery thereafter. > > I don't see any reason to special case this. If a checkpoint has no > work to do, then it will go very quickly. Why s

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-08 Thread Simon Riggs
On 8 June 2012 09:22, Kyotaro HORIGUCHI wrote: > I have a problem with promoting from hot-standby that exclusive > checkpointing retards completion of promotion. Agreed, we have that problem. > I depend on this and suppose we can omit it if latest checkpoint > has been taken so as to be able to

[HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-08 Thread Kyotaro HORIGUCHI
Hello, I have a problem with promoting from hot-standby that exclusive checkpointing retards completion of promotion. This checkpoint is "shutdown checkpoint" as a convention in realtion to TLI increment according to the comment shown below. I suppose "shutdown checkpoint" means exclusive checkpo