Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Heikki Linnakangas
Tom Lane wrote: > Speaking of which, just where is the defense that makes sure that > walsender and walreceiver are compatible? We should be checking not > only version, but all of the configuration variables that are embedded > in pg_control. That happens at startup when pg_control is read, befo

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> Whether or not anyone bothers with the timestamp message, I think adding >> a message type header is a Must Fix item. A protocol with no provision >> for extension is certainly going to bite us in the rear before long. > Agreed a message type heade

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> Simon Riggs wrote: >>> Do we need a new record type for that, is there a handy record type to >>> bounce from? > >> After starting streaming, slices of WAL are sent as CopyData messages. >> The CopyData payload begins with an XLogRecPtr, followed by

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Tom Lane
Heikki Linnakangas writes: > Simon Riggs wrote: >> Do we need a new record type for that, is there a handy record type to >> bounce from? > After starting streaming, slices of WAL are sent as CopyData messages. > The CopyData payload begins with an XLogRecPtr, followed by the WAL > data. That pay

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Heikki Linnakangas
Simon Riggs wrote: > What were the blockers that prevented sync rep from being included? I > must have missed the discussion on that part. For one, figuring out how to send back the notifications about WAL applied in standby, and all the IPC required for that. Streaming replication is a complex e

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Heikki Linnakangas
Fujii Masao wrote: > On Mon, Jan 18, 2010 at 6:35 PM, Fujii Masao wrote: >> On Mon, Jan 18, 2010 at 5:45 PM, Simon Riggs wrote: >>> How accurate is this now? With regard to remaining items of work. >>> http://wiki.postgresql.org/wiki/Streaming_Replication >> Not accurate. I'll correct that and pr

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Simon Riggs
On Mon, 2010-01-18 at 20:18 +0900, Fujii Masao wrote: > On Mon, Jan 18, 2010 at 6:35 PM, Fujii Masao wrote: > > On Mon, Jan 18, 2010 at 5:45 PM, Simon Riggs wrote: > >> How accurate is this now? With regard to remaining items of work. > >> http://wiki.postgresql.org/wiki/Streaming_Replication > >

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Fujii Masao
On Mon, Jan 18, 2010 at 6:35 PM, Fujii Masao wrote: > On Mon, Jan 18, 2010 at 5:45 PM, Simon Riggs wrote: >> How accurate is this now? With regard to remaining items of work. >> http://wiki.postgresql.org/wiki/Streaming_Replication > > Not accurate. I'll correct that and provide the link from > "

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Fujii Masao
On Mon, Jan 18, 2010 at 5:45 PM, Simon Riggs wrote: > How accurate is this now? With regard to remaining items of work. > http://wiki.postgresql.org/wiki/Streaming_Replication Not accurate. I'll correct that and provide the link from "v8.5 Open Items page" to that. Regards, -- Fujii Masao NIPP

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Simon Riggs
On Mon, 2010-01-18 at 08:28 +0200, Heikki Linnakangas wrote: > > Do we need a new record type for that, is there a handy record type to > > bounce from? > > After starting streaming, slices of WAL are sent as CopyData messages. > The CopyData payload begins with an XLogRecPtr, followed by the WAL

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-18 Thread Simon Riggs
On Mon, 2010-01-18 at 08:28 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Sun, 2010-01-17 at 22:57 +0200, Heikki Linnakangas wrote: > >>> It would be good if there was a keepalive WAL record with a > >> timestamp on it generated every N seconds while in streaming mode. > >> > >> Yeah

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-17 Thread Heikki Linnakangas
Simon Riggs wrote: > On Sun, 2010-01-17 at 22:57 +0200, Heikki Linnakangas wrote: >>> It would be good if there was a keepalive WAL record with a >> timestamp on it generated every N seconds while in streaming mode. >> >> Yeah, that would help. In streaming replication we could also send >> such ti

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-17 Thread Simon Riggs
On Sun, 2010-01-17 at 22:57 +0200, Heikki Linnakangas wrote: > > > It would be good if there was a keepalive WAL record with a > timestamp on it generated every N seconds while in streaming mode. > > Yeah, that would help. In streaming replication we could also send > such timestamp as a separate

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-17 Thread Heikki Linnakangas
Simon Riggs wrote: > On Fri, 2010-01-15 at 20:50 +0200, Heikki Linnakangas wrote: > >>> So in either case, when we are waiting for new input we reset the timer >>> as soon as new WAL is received. The resolution/accuracy of standby_delay >>> will be no more than the time taken to replay a single fi

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-16 Thread Joshua D. Drake
On Sat, 2010-01-16 at 18:20 +, Simon Riggs wrote: > On Sat, 2010-01-16 at 11:37 -0500, Tom Lane wrote: > > Simon Riggs writes: > > > I'm wondering if it wouldn't just be easier to put in a plugin for > > > recovery conflict handling, so the user can decide what to do > > > themselves. That see

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-16 Thread Simon Riggs
On Sat, 2010-01-16 at 11:37 -0500, Tom Lane wrote: > Simon Riggs writes: > > I'm wondering if it wouldn't just be easier to put in a plugin for > > recovery conflict handling, so the user can decide what to do > > themselves. That seems like a better plan than chewing through these > > issues now.

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-16 Thread Simon Riggs
On Fri, 2010-01-15 at 20:50 +0200, Heikki Linnakangas wrote: > That would change the meaning of max_standby_delay. Currently, it's the > delay between *generating* and applying a WAL record, your proposal > would change it to mean delay between receiving and applying it. That > seems a lot less us

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-16 Thread Tom Lane
Simon Riggs writes: > I'm wondering if it wouldn't just be easier to put in a plugin for > recovery conflict handling, so the user can decide what to do > themselves. That seems like a better plan than chewing through these > issues now. Making it a plugin doesn't solve anything. This is not th

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-16 Thread Simon Riggs
On Sat, 2010-01-16 at 14:08 +0100, Dimitri Fontaine wrote: > Simon Riggs writes: > > On Fri, 2010-01-15 at 20:50 +0200, Heikki Linnakangas wrote: > > Yes, it does. And I know you're thinking along those lines because we > > are concurrently discussing how to handle re-connection after updates. >

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-16 Thread Dimitri Fontaine
Simon Riggs writes: > On Fri, 2010-01-15 at 20:50 +0200, Heikki Linnakangas wrote: > Yes, it does. And I know you're thinking along those lines because we > are concurrently discussing how to handle re-connection after updates. With my State Machine proposal, we could only apply max_standby_delay

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-15 Thread Simon Riggs
On Fri, 2010-01-15 at 20:50 +0200, Heikki Linnakangas wrote: > > So in either case, when we are waiting for new input we reset the timer > > as soon as new WAL is received. The resolution/accuracy of standby_delay > > will be no more than the time taken to replay a single file. This > > shouldn't

Re: [HACKERS] Hot Standby and handling max_standby_delay

2010-01-15 Thread Heikki Linnakangas
Simon Riggs wrote: > We need to calculate a more accurate time since WAL arrived to make > max_standby_delay sensible in all cases. Difficult to know exactly when > to record new timestamps for received WAL. So, proposal is... > > if (Base time is earlier than WAL record time) > standby_dela

[HACKERS] Hot Standby and handling max_standby_delay

2010-01-15 Thread Simon Riggs
We need to calculate a more accurate time since WAL arrived to make max_standby_delay sensible in all cases. Difficult to know exactly when to record new timestamps for received WAL. So, proposal is... if (Base time is earlier than WAL record time) standby_delay = WAL record time - Base ti