Re: [HACKERS] Time-Delayed Standbys

On 2013-12-13 13:44:30 +, Simon Riggs wrote: > On 13 December 2013 13:22, Andres Freund wrote: > > On 2013-12-13 13:09:13 +, Simon Riggs wrote: > >> On 13 December 2013 11:58, Andres Freund wrote: > >> >> I removed it because it was after the pause. I'll replace it, but > >> >> before the

Re: [HACKERS] Time-Delayed Standbys

On Fri, Dec 13, 2013 at 11:44 AM, Simon Riggs wrote: > > On 13 December 2013 13:22, Andres Freund wrote: > > On 2013-12-13 13:09:13 +, Simon Riggs wrote: > >> On 13 December 2013 11:58, Andres Freund wrote: > >> > On 2013-12-13 11:56:47 +, Simon Riggs wrote: > >> >> On 12 December 2013 2

Re: [HACKERS] Time-Delayed Standbys

On 13 December 2013 13:22, Andres Freund wrote: > On 2013-12-13 13:09:13 +, Simon Riggs wrote: >> On 13 December 2013 11:58, Andres Freund wrote: >> > On 2013-12-13 11:56:47 +, Simon Riggs wrote: >> >> On 12 December 2013 21:58, Fabrízio de Royes Mello >> >> wrote: >> >> > Reviewing the

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-13 13:09:13 +, Simon Riggs wrote: > On 13 December 2013 11:58, Andres Freund wrote: > > On 2013-12-13 11:56:47 +, Simon Riggs wrote: > >> On 12 December 2013 21:58, Fabrízio de Royes Mello > >> wrote: > >> > Reviewing the committed patch I noted that the "CheckForStandbyTrigger

Re: [HACKERS] Time-Delayed Standbys

On 13 December 2013 11:58, Andres Freund wrote: > On 2013-12-13 11:56:47 +, Simon Riggs wrote: >> On 12 December 2013 21:58, Fabrízio de Royes Mello >> wrote: >> > Reviewing the committed patch I noted that the "CheckForStandbyTrigger()" >> > after the delay was removed. >> > >> > If we promo

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-13 11:56:47 +, Simon Riggs wrote: > On 12 December 2013 21:58, Fabrízio de Royes Mello > wrote: > > Reviewing the committed patch I noted that the "CheckForStandbyTrigger()" > > after the delay was removed. > > > > If we promote the standby during the delay and don't check the trigg

Re: [HACKERS] Time-Delayed Standbys

On 12 December 2013 21:58, Fabrízio de Royes Mello wrote: > > On Thu, Dec 12, 2013 at 3:42 PM, Fabrízio de Royes Mello > wrote: >> >> On Thu, Dec 12, 2013 at 3:39 PM, Simon Riggs >> wrote: >> > >> > On 12 December 2013 15:19, Simon Riggs wrote: >> > >> > > Don't panic guys! I meant UTC offset o

Re: [HACKERS] Time-Delayed Standbys

On Thu, Dec 12, 2013 at 3:42 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > On Thu, Dec 12, 2013 at 3:39 PM, Simon Riggs wrote: > > > > On 12 December 2013 15:19, Simon Riggs wrote: > > > > > Don't panic guys! I meant UTC offset only. And yes, it may not be > > > needed, will c

Re: [HACKERS] Time-Delayed Standbys

On Thu, Dec 12, 2013 at 3:39 PM, Simon Riggs wrote: > > On 12 December 2013 15:19, Simon Riggs wrote: > > > Don't panic guys! I meant UTC offset only. And yes, it may not be > > needed, will check. > > Checked, all non-UTC TZ offsets work without further effort here. > Thanks! -- Fabrízio de Ro

Re: [HACKERS] Time-Delayed Standbys

On 12 December 2013 15:19, Simon Riggs wrote: > Don't panic guys! I meant UTC offset only. And yes, it may not be > needed, will check. Checked, all non-UTC TZ offsets work without further effort here. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Time-Delayed Standbys

On 12 December 2013 15:03, Robert Haas wrote: > On Thu, Dec 12, 2013 at 9:52 AM, Tom Lane wrote: >> Simon Riggs writes: >>> On 12 December 2013 11:05, Andres Freund wrote: > My suggestion would be to add the TZ to the checkpoint record. This > way all users of WAL can see the TZ of the

Re: [HACKERS] Time-Delayed Standbys

On Thu, Dec 12, 2013 at 9:52 AM, Tom Lane wrote: > Simon Riggs writes: >> On 12 December 2013 11:05, Andres Freund wrote: My suggestion would be to add the TZ to the checkpoint record. This way all users of WAL can see the TZ of the master and act accordingly. I'll do a separate p

Re: [HACKERS] Time-Delayed Standbys

Simon Riggs writes: > On 12 December 2013 11:05, Andres Freund wrote: >>> My suggestion would be to add the TZ to the checkpoint record. This >>> way all users of WAL can see the TZ of the master and act accordingly. >>> I'll do a separate patch for that. >> Intuitively I'd say that might be use

Re: [HACKERS] Time-Delayed Standbys

2013/12/12 Simon Riggs > On 12 December 2013 10:42, KONDO Mitsumasa > wrote: > > >> I agree with your request here, but I don't think negative values are > >> the right way to implement that, at least it would not be very usable. > > > > I think that my proposal is the easiest and simplist way t

Re: [HACKERS] Time-Delayed Standbys

On 12 December 2013 11:05, Andres Freund wrote: >> My suggestion would be to add the TZ to the checkpoint record. This >> way all users of WAL can see the TZ of the master and act accordingly. >> I'll do a separate patch for that. > > Intuitively I'd say that might be useful - but I am not reall

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-12 09:09:21 +, Simon Riggs wrote: > >> * Add functionality (I propose) > >> We can set negative number at min_standby_apply_delay. I think that > >> this feature > >> is for world wide replication situation. For example, master server is > >> in > >> Japan and slave s

Re: [HACKERS] Time-Delayed Standbys

On 12 December 2013 10:42, KONDO Mitsumasa wrote: >> I agree with your request here, but I don't think negative values are >> the right way to implement that, at least it would not be very usable. > > I think that my proposal is the easiest and simplist way to solve this > problem. And I believe

Re: [HACKERS] Time-Delayed Standbys

(2013/12/12 18:09), Simon Riggs wrote: On 9 December 2013 10:54, KONDO Mitsumasa wrote: (2013/12/09 19:35), Pavel Stehule wrote: 2013/12/9 KONDO Mitsumasa mailto:kondo.mitsum...@lab.ntt.co.jp>> Hi Fabrízio, I test your v4 patch, and send your review comments. * Fix typo

Re: [HACKERS] Time-Delayed Standbys

On 9 December 2013 10:54, KONDO Mitsumasa wrote: > (2013/12/09 19:35), Pavel Stehule wrote: >> >> >> >> >> 2013/12/9 KONDO Mitsumasa > > >> >> >> Hi Fabrízio, >> >> I test your v4 patch, and send your review comments. >> >> * Fix typo >> > 49

Re: [HACKERS] Time-Delayed Standbys

On 12 December 2013 08:19, KONDO Mitsumasa wrote: > (2013/12/12 7:23), Fabrízio de Royes Mello wrote: >> >> On Wed, Dec 11, 2013 at 7:47 PM, Andres Freund > > * hot_standby=off: Makes delay useable with wal_level=archive (and thus >> > a lower WAL volume) >> > * standby_mode=off: Configuratio

Re: [HACKERS] Time-Delayed Standbys

(2013/12/12 7:23), Fabrízio de Royes Mello wrote: On Wed, Dec 11, 2013 at 7:47 PM, Andres Freund * hot_standby=off: Makes delay useable with wal_level=archive (and thus > a lower WAL volume) > * standby_mode=off: Configurations that use tools like pg_standby and > similar simply don't nee

Re: [HACKERS] Time-Delayed Standbys

On Wed, Dec 11, 2013 at 7:47 PM, Andres Freund wrote: > > I don't think that position has any merit, sorry: Think about the way > this stuff gets setup. The user creates a new basebackup (pg_basebackup, > manual pg_start/stop_backup, shutdown primary). Then he creates a > recovery conf by either s

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-11 16:37:54 -0200, Fabrízio de Royes Mello wrote: > On Wed, Dec 11, 2013 at 6:27 AM, Simon Riggs wrote: > > >> I think this feature will be used in a lot of scenarios in > > >> which PITR is currently used. > > > > > > We have to judge which is better, we get something potential or to p

Re: [HACKERS] Time-Delayed Standbys

On Wed, Dec 11, 2013 at 6:27 AM, Simon Riggs wrote: > > On 11 December 2013 06:36, KONDO Mitsumasa > wrote: > > >> I think this feature will be used in a lot of scenarios in > >> which PITR is currently used. > > > > We have to judge which is better, we get something potential or to protect > > s

Re: [HACKERS] Time-Delayed Standbys

On 11 December 2013 06:36, KONDO Mitsumasa wrote: >> I think this feature will be used in a lot of scenarios in >> which PITR is currently used. > > We have to judge which is better, we get something potential or to protect > stupid. > And we had better to wait author's comment... I'd say just d

Re: [HACKERS] Time-Delayed Standbys

(2013/12/10 18:38), Andres Freund wrote: "master PITR"? What's that? All PITR is based on recovery.conf and thus not really a "master"? "master PITR" is PITR with "standby_mode = off". It's just recovery from basebackup. They have difference between "master PITR" and "standby" that the former w

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-10 13:26:27 +0900, KONDO Mitsumasa wrote: > (2013/12/09 20:29), Andres Freund wrote: > >On 2013-12-09 19:51:01 +0900, KONDO Mitsumasa wrote: > >>Add my comment. We have to consider three situations. > >> > >>1. PITR > >>2. replication standby > >>3. replication standby with restore_comma

Re: [HACKERS] Time-Delayed Standbys

(2013/12/09 20:29), Andres Freund wrote: On 2013-12-09 19:51:01 +0900, KONDO Mitsumasa wrote: Add my comment. We have to consider three situations. 1. PITR 2. replication standby 3. replication standby with restore_command I think this patch cannot delay in 1 situation. Why? I have three r

Re: [HACKERS] Time-Delayed Standbys

On 9 Dec 2013 12:16, "Craig Ringer" wrote: > The only way to "deal with" clock drift that isn't fragile in the face > of variable latency, etc, is to basically re-implement (S)NTP in order > to find out what the clock difference with the remote is. There's actually an entirely different way to "

Re: [HACKERS] Time-Delayed Standbys

On 12/04/2013 02:46 AM, Robert Haas wrote: >> Thanks for your review Christian... > > So, I proposed this patch previously and I still think it's a good > idea, but it got voted down on the grounds that it didn't deal with > clock drift. I view that as insufficient reason to reject the > feature,

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-09 19:51:01 +0900, KONDO Mitsumasa wrote: > Add my comment. We have to consider three situations. > > 1. PITR > 2. replication standby > 3. replication standby with restore_command > > I think this patch cannot delay in 1 situation. Why? Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] Time-Delayed Standbys

(2013/12/09 19:35), Pavel Stehule wrote: 2013/12/9 KONDO Mitsumasa mailto:kondo.mitsum...@lab.ntt.co.jp>> Hi Fabrízio, I test your v4 patch, and send your review comments. * Fix typo > 49 -# commited transactions from the master, specify a recovery time delay. > 49 +#

Re: [HACKERS] Time-Delayed Standbys

(2013/12/09 19:36), KONDO Mitsumasa wrote: * Problem 1 I read your wittened document. There is "PITR has not affected". However, when I run PITR with min_standby_apply_delay=300, it cannot start server. The log is under following. [mitsu-ko@localhost postgresql]$ bin/pg_ctl -D data2 start se

Re: [HACKERS] Time-Delayed Standbys

2013/12/9 KONDO Mitsumasa > Hi Fabrízio, > > I test your v4 patch, and send your review comments. > > * Fix typo > > 49 -# commited transactions from the master, specify a recovery time > delay. > > 49 +# committed transactions from the master, specify a recovery time > delay. > > * Fix white spa

Re: [HACKERS] Time-Delayed Standbys

Hi Fabrízio, I test your v4 patch, and send your review comments. * Fix typo > 49 -# commited transactions from the master, specify a recovery time delay. > 49 +# committed transactions from the master, specify a recovery time delay. * Fix white space > 177 - if (secs <= 0 && micr

Re: [HACKERS] Time-Delayed Standbys

On Fri, Dec 6, 2013 at 1:36 PM, Robert Haas wrote: > On Thu, Dec 5, 2013 at 11:07 PM, Fabrízio de Royes Mello > wrote: > > On Tue, Dec 3, 2013 at 5:33 PM, Simon Riggs > wrote: > >> > >> > - compute recoveryUntilDelayTime in XLOG_XACT_COMMIT and > >> > XLOG_XACT_COMMIT_COMPACT checks > >> > >> W

Re: [HACKERS] Time-Delayed Standbys

On Thu, Dec 5, 2013 at 11:07 PM, Fabrízio de Royes Mello wrote: > On Tue, Dec 3, 2013 at 5:33 PM, Simon Riggs wrote: >> >> > - compute recoveryUntilDelayTime in XLOG_XACT_COMMIT and >> > XLOG_XACT_COMMIT_COMPACT checks >> >> Why just those? Why not aborts and restore points also? >> > > I think m

Re: [HACKERS] Time-Delayed Standbys

On Tue, Dec 3, 2013 at 5:33 PM, Simon Riggs wrote: > > > - compute recoveryUntilDelayTime in XLOG_XACT_COMMIT and > > XLOG_XACT_COMMIT_COMPACT checks > > Why just those? Why not aborts and restore points also? > I think make no sense execute the delay after aborts and/or restore points, because i

Re: [HACKERS] Time-Delayed Standbys

On Thu, Dec 5, 2013 at 7:57 AM, Simon Riggs wrote: > On 5 December 2013 08:51, Magnus Hagander wrote: > > > Not recalling the older thread, but it seems the "breaks on clock > drift", I > > think we can fairly easily make that situation "good enough". Just have > > IDENTIFY_SYSTEM return the cur

Re: [HACKERS] Time-Delayed Standbys

On 5 December 2013 08:51, Magnus Hagander wrote: > Not recalling the older thread, but it seems the "breaks on clock drift", I > think we can fairly easily make that situation "good enough". Just have > IDENTIFY_SYSTEM return the current timestamp on the master, and refuse to > start if the time

Re: [HACKERS] Time-Delayed Standbys

On Thu, Dec 5, 2013 at 1:45 AM, Simon Riggs wrote: > On 3 December 2013 18:46, Robert Haas wrote: > > On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello > > wrote: > >> On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse < > christ...@2ndquadrant.com> > >> wrote: > >>> > >>> Hi Fabrizio, > >>

Re: [HACKERS] Time-Delayed Standbys

On 3 December 2013 18:46, Robert Haas wrote: > On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello > wrote: >> On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse >> wrote: >>> >>> Hi Fabrizio, >>> >>> looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It >>> applies and compiles w/

Re: [HACKERS] Time-Delayed Standbys

src/backend/access/transam/xlog.c:5889: trailing whitespace. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Time-Delayed Standbys

Hi, On 04/12/13 07:22, Kevin Grittner wrote: > There are many things that a system admin can get wrong.  Failing > to supply this feature because the sysadmin might not be running > ntpd (or equivalent) correctly seems to me to be like not having > the software do fsync because the sysadmin might

Re: [HACKERS] Time-Delayed Standbys

Robert Haas wrote: > So, I proposed this patch previously and I still think it's a > good idea, but it got voted down on the grounds that it didn't > deal with clock drift.  I view that as insufficient reason to > reject the feature, but others disagreed.  Unless some of those > people have chang

Re: [HACKERS] Time-Delayed Standbys

2013/12/4 Andres Freund > On 2013-12-04 22:47:47 +0900, Mitsumasa KONDO wrote: > > 2013/12/4 Andres Freund > > When it happened, psql cannot connect standby server at all. I think this > > behavior is not good. > > It should only delay recovery position and can seen old delay table data. > > Tha

Re: [HACKERS] Time-Delayed Standbys

2013/12/4 Christian Kruse > You created a master node and a hot standby with 300 delay. Then > you stopped the standby, did the pgbench and startet the hot standby > again. It did not get in line with the master. Is this correct? > No. First, I start master, and execute pgbench. Second, I sta

Re: [HACKERS] Time-Delayed Standbys

Hi, On 2013-12-03 19:33:16 +, Simon Riggs wrote: > > - compute recoveryUntilDelayTime in XLOG_XACT_COMMIT and > > XLOG_XACT_COMMIT_COMPACT checks > > Why just those? Why not aborts and restore points also? What would the advantage of waiting on anything but commits be? If it's not a commit,

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-04 22:47:47 +0900, Mitsumasa KONDO wrote: > 2013/12/4 Andres Freund > When it happened, psql cannot connect standby server at all. I think this > behavior is not good. > It should only delay recovery position and can seen old delay table data. That doesn't sound like a good plan - even

Re: [HACKERS] Time-Delayed Standbys

2013/12/4 Andres Freund > On 2013-12-04 11:13:58 +0900, KONDO Mitsumasa wrote: > > >4) Start the slave and connect to it using psql and in another session > I can see > > >all archive recovery log > > Hmm... I had thought my mistake in reading your email, but it reproduce > again. > > When I sat

Re: [HACKERS] Time-Delayed Standbys

Hi, On 04/12/13 11:13, KONDO Mitsumasa wrote: > >1) Clusters > >- build master > >- build slave and attach to the master using SR and config > >recovery_time_delay to > >1min. > > > >2) Stop de Slave > > > >3) Run some transactions on the master using pgbench to generate a lot of > >archives > >

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-04 11:13:58 +0900, KONDO Mitsumasa wrote: > >4) Start the slave and connect to it using psql and in another session I can > >see > >all archive recovery log > Hmm...　I had thought my mistake in reading your email, but it reproduce again. > When I sat small recovery_time_delay(=3), i

Re: [HACKERS] Time-Delayed Standbys

(2013/12/04 4:00), Andres Freund wrote: On 2013-12-03 13:46:28 -0500, Robert Haas wrote: On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello wrote: On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse wrote: Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It app

Re: [HACKERS] Time-Delayed Standbys

(2013/11/30 5:34), Fabrízio de Royes Mello wrote: On Fri, Nov 29, 2013 at 5:49 AM, KONDO Mitsumasa mailto:kondo.mitsum...@lab.ntt.co.jp>> wrote: > * Problem1 > Your patch does not code recovery.conf.sample about recovery_time_delay. > Please add it. Fixed. OK. It seems no problem. > * Prob

Re: [HACKERS] Time-Delayed Standbys

On 18 October 2013 19:03, Fabrízio de Royes Mello wrote: > The attached patch is a continuation of Robert's work [1]. Reviewing v2... > I made some changes: > - use of Latches instead of pg_usleep, so we don't have to wakeup regularly. OK > - call HandleStartupProcInterrupts() before CheckFo

Re: [HACKERS] Time-Delayed Standbys

On 2013-12-03 13:46:28 -0500, Robert Haas wrote: > On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello > wrote: > > On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse > > wrote: > >> > >> Hi Fabrizio, > >> > >> looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It > >> applies and c

Re: [HACKERS] Time-Delayed Standbys

On 12/03/2013 10:46 AM, Robert Haas wrote: On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello wrote: On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse wrote: Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It applies and compiles w/o errors or warnings. I s

Re: [HACKERS] Time-Delayed Standbys

On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello wrote: > On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse > wrote: >> >> Hi Fabrizio, >> >> looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It >> applies and compiles w/o errors or warnings. I set up a master and two >> hot st

Re: [HACKERS] Time-Delayed Standbys

On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse wrote: > Hi Fabrizio, > > looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It > applies and compiles w/o errors or warnings. I set up a master and two > hot standbys replicating from the master, one with 5 minutes delay and > one withou

Re: [HACKERS] Time-Delayed Standbys

Hi Fabrizio, looks good to me. I did some testing on 9.2.4, 9.2.5 and HEAD. It applies and compiles w/o errors or warnings. I set up a master and two hot standbys replicating from the master, one with 5 minutes delay and one without delay. After that I created a new database and generated some tes

Re: [HACKERS] Time-Delayed Standbys

On Fri, Nov 29, 2013 at 5:49 AM, KONDO Mitsumasa < kondo.mitsum...@lab.ntt.co.jp> wrote: > > Hi Royes, > > I'm sorry for my late review... > No problem... > I feel potential of your patch in PG replication function, and it might be something useful for all people. I check your patch and have som

Re: [HACKERS] Time-Delayed Standbys

Hi Royes, I'm sorry for my late review... I feel potential of your patch in PG replication function, and it might be something useful for all people. I check your patch and have some comment for improvement. I haven't executed detail of unexpected sutuation yet. But I think that under followi

Re: [HACKERS] time-delayed standbys

On Thu, Jun 30, 2011 at 6:25 PM, Robert Haas wrote: >> I think the time problems are more complex than said. The patch relies >> upon transaction completion times, but not all WAL records have a time >> attached to them. Plus you only used commits anyway, not sure why. > > For the same reason we

Re: [HACKERS] time-delayed standbys

Fujii Masao writes: > On Fri, Jul 1, 2011 at 2:25 AM, Robert Haas wrote: >>> Some actions aren't even transactional, such as DROP DATABASE, amongst >> >> Good point.  We'd probably need to add a timestamp to the drop >> database record, as that's a case that people would likely want to >> defend

Re: [HACKERS] time-delayed standbys

On Fri, Jul 1, 2011 at 3:25 AM, Robert Haas wrote: > On Thu, Jun 30, 2011 at 1:51 PM, Kevin Grittner > wrote: >> I think doing anything in PostgreSQL around this beyond allowing >> DBAs to trust their server clocks is insane.  The arguments for >> using and trusting ntpd is pretty much identical

Re: [HACKERS] time-delayed standbys

On Fri, Jul 1, 2011 at 2:25 AM, Robert Haas wrote: >> Some actions aren't even transactional, such as DROP DATABASE, amongst > > Good point.  We'd probably need to add a timestamp to the drop > database record, as that's a case that people would likely want to > defend against with this feature.

Re: [HACKERS] time-delayed standbys

On Thu, Jun 30, 2011 at 1:51 PM, Kevin Grittner wrote: > I think doing anything in PostgreSQL around this beyond allowing > DBAs to trust their server clocks is insane.  The arguments for > using and trusting ntpd is pretty much identical to the arguments > for using and trusting the OS file syste

Re: [HACKERS] time-delayed standbys

Kevin, > I think doing anything in PostgreSQL around this beyond allowing > DBAs to trust their server clocks is insane. The arguments for > using and trusting ntpd is pretty much identical to the arguments > for using and trusting the OS file systems. Oh, you don't want to implement our own NTP

Re: [HACKERS] time-delayed standbys

Josh Berkus wrote: > when we last had the argument about time synchronization, > Kevin Grittner (I believe) pointed out that unsynchronized > replication servers have an assortment of other issues ... like > any read query involving now(). I don't remember making that point, although I think i

Re: [HACKERS] time-delayed standbys

On 6/30/11 10:25 AM, Robert Haas wrote: > So I think keeping it defined it terms of time is the > right way forward, even though the need for external time > synchronization is, certainly, not ideal. Actually, when we last had the argument about time synchronization, Kevin Grittner (I believe) poi

Re: [HACKERS] time-delayed standbys

On Thu, Jun 30, 2011 at 6:45 AM, Simon Riggs wrote: > The only way to control this is with a time delay that can be changed > while the server is running. A recovery.conf parameter doesn't allow > that, so another way is preferable. True. We've talked about making the recovery.conf parameters in

Re: [HACKERS] time-delayed standbys

On Thu, Jun 30, 2011 at 1:00 PM, Josh Berkus wrote: > On 6/30/11 2:00 AM, Simon Riggs wrote: Manual (or scripted) intervention is always necessary if you reach disk >> 100% full. >>> > >>> > Wow, that's a pretty crappy failure mode... but I don't think we need >>> > to fix it just on acc

Re: [HACKERS] time-delayed standbys

On 6/30/11 2:00 AM, Simon Riggs wrote: >>> Manual (or scripted) intervention is always necessary if you reach disk >>> >> 100% full. >> > >> > Wow, that's a pretty crappy failure mode... but I don't think we need >> > to fix it just on account of this patch. It would be nice to fix, of >> > course

Re: [HACKERS] time-delayed standbys

On Wed, Jun 29, 2011 at 7:11 PM, Robert Haas wrote: > I don't really see how that's any different from what happens now.  If > (for whatever reason) the master is generating WAL faster than a > streaming standby can replay it, then the excess WAL is going to pile > up someplace, and you might run

Re: [HACKERS] time-delayed standbys

On Thu, Jun 30, 2011 at 2:56 AM, Robert Haas wrote: > On Wed, Jun 29, 2011 at 9:54 PM, Josh Berkus wrote: >>> I am not sure exactly how walreceiver handles it if the disk is full. >>> I assume it craps out and eventually retries, so probably what will >>> happen is that, after the standby's pg_xl

Re: [HACKERS] time-delayed standbys

On Thu, Jun 30, 2011 at 12:14 PM, Fujii Masao wrote: > We should disable this feature also after recovery reaches the stop > point (specified in recovery_target_xxx)? Another comment; it's very helpful to document the behavior of delayed standby when promoting or after reaching the stop point. R

Re: [HACKERS] time-delayed standbys

On Thu, Jun 30, 2011 at 10:56 AM, Robert Haas wrote: >>> Nope, it gets stuck and stops there.  Replay doesn't advance unless you >> can somehow clear out some space manually; if the disk is full, the disk >> is full, and PostgreSQL doesn't remove WAL files without being able to >> write files firs

Re: [HACKERS] time-delayed standbys

On Wed, Jun 29, 2011 at 11:14 AM, Robert Haas wrote: > On Wed, Jun 15, 2011 at 1:58 AM, Fujii Masao wrote: >> After we run "pg_ctl promote", time-delayed replication should be disabled? >> Otherwise, failover might take very long time when we set recovery_time_delay >> to high value. > > PFA a pa

Re: [HACKERS] time-delayed standbys

On Wed, Jun 29, 2011 at 9:54 PM, Josh Berkus wrote: >> I am not sure exactly how walreceiver handles it if the disk is full. >> I assume it craps out and eventually retries, so probably what will >> happen is that, after the standby's pg_xlog directory fills up, >> walreceiver will sit there and e

Re: [HACKERS] time-delayed standbys

On 6/29/11 11:11 AM, Robert Haas wrote: > If the standby gets far enough behind the master that the required > files are no longer there, then it will switch to the archive, if > available. One more thing: As I understand it (and my testing shows this), the standby *prefers* the archive logs, and

Re: [HACKERS] time-delayed standbys

Robert, > I don't really see how that's any different from what happens now. If > (for whatever reason) the master is generating WAL faster than a > streaming standby can replay it, then the excess WAL is going to pile > up someplace, and you might run out of disk space. Time-delaying the > sta

Re: [HACKERS] time-delayed standbys

On Wed, Jun 29, 2011 at 1:50 PM, Simon Riggs wrote: >> As implemented, the feature will work with either streaming >> replication or with file-based replication. > > That sounds like the exact opposite of yours and Fujii's comments > above. Please explain. I think our comments above were addressi

Re: [HACKERS] time-delayed standbys

On Wed, Jun 29, 2011 at 1:24 PM, Robert Haas wrote: > On Wed, Jun 29, 2011 at 4:00 AM, Simon Riggs wrote: >> On Thu, Jun 16, 2011 at 7:29 PM, Robert Haas wrote: >>> On Wed, Jun 15, 2011 at 1:58 AM, Fujii Masao wrote: When the replication connection is terminated, the standby tries to read

Re: [HACKERS] time-delayed standbys

On Wed, Jun 29, 2011 at 4:00 AM, Simon Riggs wrote: > On Thu, Jun 16, 2011 at 7:29 PM, Robert Haas wrote: >> On Wed, Jun 15, 2011 at 1:58 AM, Fujii Masao wrote: >>> When the replication connection is terminated, the standby tries to read >>> WAL files from the archive. In this case, there is no

Re: [HACKERS] time-delayed standbys

On Thu, Jun 16, 2011 at 7:29 PM, Robert Haas wrote: > On Wed, Jun 15, 2011 at 1:58 AM, Fujii Masao wrote: >> When the replication connection is terminated, the standby tries to read >> WAL files from the archive. In this case, there is no walreceiver process, >> so how does the standby calculate

Re: [HACKERS] time-delayed standbys

On Wed, Jun 15, 2011 at 6:58 AM, Fujii Masao wrote: > Or, we should > implement new "promote" mode which finishes a recovery as soon as > "promote" is requested (i.e., not replay all the available WAL records)? That's not a new feature. We had it in 8.4, but it was removed. Originally, we suppo

Re: [HACKERS] time-delayed standbys

On Wed, Jun 15, 2011 at 1:58 AM, Fujii Masao wrote: > After we run "pg_ctl promote", time-delayed replication should be disabled? > Otherwise, failover might take very long time when we set recovery_time_delay > to high value. PFA a patch that I believe will disable recovery_time_delay after prom

Re: [HACKERS] time-delayed standbys

On Fri, Jun 17, 2011 at 11:34 AM, Robert Haas wrote: > On Thu, Jun 16, 2011 at 10:10 PM, Fujii Masao wrote: According to the above page, one purpose of time-delayed replication is to protect against user mistakes on master. But, when an user notices his wrong operation on mas

Re: [HACKERS] time-delayed standbys

On Thu, Jun 16, 2011 at 10:10 PM, Fujii Masao wrote: >>> According to the above page, one purpose of time-delayed replication is to >>> protect against user mistakes on master. But, when an user notices his wrong >>> operation on master, what should he do next? The WAL records of his wrong >>> ope

Re: [HACKERS] time-delayed standbys

On Fri, Jun 17, 2011 at 3:29 AM, Robert Haas wrote: > Even if that were not an issue, I'm still more or less of the opinion > that trying to solve the time synchronization problem is a rathole > anyway.  To really solve this problem well, you're going to need the > standby to send a message contai

Re: [HACKERS] time-delayed standbys

On Wed, Jun 15, 2011 at 1:58 AM, Fujii Masao wrote: > When the replication connection is terminated, the standby tries to read > WAL files from the archive. In this case, there is no walreceiver process, > so how does the standby calculate the clock difference? Good question. Also, just because

Re: [HACKERS] time-delayed standbys

On Wed, Jun 15, 2011 at 12:58 AM, Fujii Masao wrote: > > http://forge.mysql.com/worklog/task.php?id=344 > According to the above page, one purpose of time-delayed replication is to > protect against user mistakes on master. But, when an user notices his wrong > operation on master, what should he

Re: [HACKERS] time-delayed standbys

On Thu, Apr 21, 2011 at 12:18 PM, Robert Haas wrote: > On Wed, Apr 20, 2011 at 11:15 AM, Tom Lane wrote: >> Robert Haas writes: >>> I am a bit concerned about the reliability of this approach.  If there >>> is some network lag, or some lag in processing from the master, we >>> could easily get t

Re: [HACKERS] time-delayed standbys

On 11.05.2011 14:16, Fujii Masao wrote: On Wed, May 11, 2011 at 6:50 PM, Heikki Linnakangas wrote: I think we can just always call ShutdownWalRcv(). It should be gone if the server was promoted while streaming, but that's just an implementation detail of what the promotion code does. There's n

Re: [HACKERS] time-delayed standbys

On Wed, May 11, 2011 at 6:50 PM, Heikki Linnakangas wrote: > I think we can just always call ShutdownWalRcv(). It should be gone if the > server was promoted while streaming, but that's just an implementation > detail of what the promotion code does. There's no hard reason why it > shouldn't be ru

Re: [HACKERS] time-delayed standbys

On 11.05.2011 08:29, Fujii Masao wrote: On Sat, May 7, 2011 at 10:48 PM, Robert Haas wrote: I was able to reproduce something very like this in unpatched master, just by letting recovery pause at a named restore point, and then resuming it. I was able to reproduce the same problem even in 9.0

Re: [HACKERS] time-delayed standbys

On 07.05.2011 16:48, Robert Haas wrote: I was able to reproduce something very like this in unpatched master, just by letting recovery pause at a named restore point, and then resuming it. LOG: recovery stopping at restore point "stop", time 2011-05-07 09:28:01.652958-04 LOG: recovery has paus

Re: [HACKERS] time-delayed standbys

On Sat, May 7, 2011 at 10:48 PM, Robert Haas wrote: > I was able to reproduce something very like this in unpatched master, > just by letting recovery pause at a named restore point, and then > resuming it. I was able to reproduce the same problem even in 9.0. When the standby reaches the recover

Re: [HACKERS] time-delayed standbys

On Sat, Apr 23, 2011 at 9:46 PM, Jaime Casanova wrote: > On Tue, Apr 19, 2011 at 9:47 PM, Robert Haas wrote: >> >> That is, a standby configured such that replay lags a prescribed >> amount of time behind the master. >> >> This seemed easy to implement, so I did.  Patch (for 9.2, obviously) >> a

Re: [HACKERS] time-delayed standbys

On Tue, Apr 19, 2011 at 9:47 PM, Robert Haas wrote: > > That is, a standby configured such that replay lags a prescribed > amount of time behind the master. > > This seemed easy to implement, so I did.  Patch (for 9.2, obviously) attached. > This crashes when stoping recovery to a target (i tried

  1   2   >