Re: [HACKERS] Switching timeline over streaming replication

2012-12-26 Thread Heikki Linnakangas
On 23.12.2012 16:37, Fujii Masao wrote: On Fri, Dec 21, 2012 at 1:48 AM, Fujii Masaomasao.fu...@gmail.com wrote: On Sat, Dec 15, 2012 at 9:36 AM, Fujii Masaomasao.fu...@gmail.com wrote: I found another requested timeline does not contain minimum recovery point error scenario in HEAD: 1. Set

Re: [HACKERS] Switching timeline over streaming replication

2012-12-23 Thread Fujii Masao
On Fri, Dec 21, 2012 at 1:48 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Dec 15, 2012 at 9:36 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Dec 8, 2012 at 12:51 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December

Re: [HACKERS] Switching timeline over streaming replication

2012-12-21 Thread Heikki Linnakangas
On 21.12.2012 01:50, Thom Brown wrote: Now I'm getting this on all standbys after promoting the first standby in a chain. ... TRAP: FailedAssertion(!(((sentPtr)= (SendRqstPtr))), File: walsender.c, Line: 1425) Sigh. I'm sounding like a broken record, but I just committed another fix for

Re: [HACKERS] Switching timeline over streaming replication

2012-12-21 Thread Thom Brown
On 21 December 2012 18:13, Heikki Linnakangas hlinnakan...@vmware.comwrote: On 21.12.2012 01:50, Thom Brown wrote: Now I'm getting this on all standbys after promoting the first standby in a chain. ... TRAP: FailedAssertion(!(((sentPtr)**= (SendRqstPtr))), File: walsender.c, Line:

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Heikki Linnakangas
On 17.12.2012 15:05, Thom Brown wrote: I just set up 120 chained standbys, and for some reason I'm seeing these errors: LOG: replication terminated by primary server DETAIL: End of WAL reached on timeline 1 LOG: record with zero length at 0/301EC10 LOG: fetching timeline history file for

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Andres Freund
On 2012-12-20 14:45:05 +0200, Heikki Linnakangas wrote: On 17.12.2012 15:05, Thom Brown wrote: I just set up 120 chained standbys, and for some reason I'm seeing these errors: LOG: replication terminated by primary server DETAIL: End of WAL reached on timeline 1 LOG: record with zero

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Fujii Masao
On Sat, Dec 15, 2012 at 9:36 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Dec 8, 2012 at 12:51 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Joshua Berkus
I just committed a patch that should make the requested WAL segment 00020003 has already been removed errors go away. The trick was for walsenders to not switch to the new timeline until at least one record has been replayed on it. That closes the window where the

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Thom Brown
On 20 December 2012 12:45, Heikki Linnakangas hlinnakan...@vmware.comwrote: On 17.12.2012 15:05, Thom Brown wrote: I just set up 120 chained standbys, and for some reason I'm seeing these errors: LOG: replication terminated by primary server DETAIL: End of WAL reached on timeline 1 LOG:

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Heikki Linnakangas
On 19.12.2012 04:57, Josh Berkus wrote: Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran into a weird issue when starting up r3: it refused to come out of the database is starting up

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Heikki Linnakangas
On 19.12.2012 15:55, Heikki Linnakangas wrote: On 19.12.2012 04:57, Josh Berkus wrote: Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran into a weird issue when starting up r3: it

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Heikki Linnakangas
On 19.12.2012 17:27, Heikki Linnakangas wrote: On 19.12.2012 15:55, Heikki Linnakangas wrote: On 19.12.2012 04:57, Josh Berkus wrote: Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Joshua Berkus
Heikki, The problem goes away after some time, after the 1st standby has streamed the contents of 00020003 and written it to disk, and the cascaded standby reconnects. But it would be nice to avoid that situation. I'm not sure how to do that yet, we might need to track the

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Joshua Berkus
Heikki, The next time I get the issue, and I'm not paying for 5 cloud servers by the hour, I'll give you a login. --Josh - Original Message - On 19.12.2012 17:27, Heikki Linnakangas wrote: On 19.12.2012 15:55, Heikki Linnakangas wrote: On 19.12.2012 04:57, Josh Berkus wrote:

Re: [HACKERS] Switching timeline over streaming replication

2012-12-18 Thread Josh Berkus
Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran into a weird issue when starting up r3: it refused to come out of the database is starting up mode until I did a write on the master.

Re: [HACKERS] Switching timeline over streaming replication

2012-12-17 Thread Heikki Linnakangas
On 15.12.2012 01:09, Josh Berkus wrote: Tested this on yesterday's snapshot. Worked great. Thanks for the testing! Now I wanna test a chain of cascading replicas ... how far can we chain these? There's no limit in theory. I tested with one master and two chained standbys myself. Give it

Re: [HACKERS] Switching timeline over streaming replication

2012-12-17 Thread Thom Brown
On 17 December 2012 12:07, Heikki Linnakangas hlinnakan...@vmware.comwrote: On 15.12.2012 01:09, Josh Berkus wrote: Tested this on yesterday's snapshot. Worked great. Thanks for the testing! Now I wanna test a chain of cascading replicas ... how far can we chain these? There's no

Re: [HACKERS] Switching timeline over streaming replication

2012-12-17 Thread Josh Berkus
Since Thom already did the destruction test, I only chained 7 standbies, just to see if I could reproduce his error. In the process, I accidentally connected one standby to itself. This failed, but the error message wasn't very helpful; it just gave me FATAL: could not connect, the database

Re: [HACKERS] Switching timeline over streaming replication

2012-12-14 Thread Josh Berkus
Heikki, Tested this on yesterday's snapshot. Worked great. Test: 4 Ubuntu 10.04 LTS Cloud Servers (GoGrid) Configuration: Compiled 9.3(12-12-12) with: pg_stat_statements, citext, ISN, btree_gist, pl/perl Setup Test: Master-Master Replicated to: master-replica using pg_basebackup -x.

Re: [HACKERS] Switching timeline over streaming replication

2012-12-14 Thread Fujii Masao
On Sat, Dec 8, 2012 at 12:51 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas

Re: [HACKERS] Switching timeline over streaming replication

2012-12-10 Thread Amit Kapila
From: Heikki Linnakangas [mailto:hlinnakan...@vmware.com] Sent: Friday, December 07, 2012 9:22 PM To: Amit Kapila Cc: 'PostgreSQL-development'; 'Thom Brown' Subject: Re: [HACKERS] Switching timeline over streaming replication On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December

Re: [HACKERS] Switching timeline over streaming replication

2012-12-07 Thread Heikki Linnakangas
On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a

Re: [HACKERS] Switching timeline over streaming replication

2012-12-06 Thread Amit Kapila
On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch,

Re: [HACKERS] Switching timeline over streaming replication

2012-12-05 Thread Amit Kapila
On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch, which just add some sanity checks to notice incorrect PITR scenarios. Here's a new version of the main patch

Re: [HACKERS] Switching timeline over streaming replication

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch, which just add some sanity checks to notice incorrect PITR scenarios.

Re: [HACKERS] Switching timeline over streaming replication

2012-12-04 Thread Heikki Linnakangas
After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch, which just add some sanity checks to notice incorrect PITR scenarios. Here's a new version of the main patch based on current HEAD. - Heikki streaming-tli-switch-8.patch.gz

Re: [HACKERS] Switching timeline over streaming replication

2012-12-03 Thread senthilnathan
Is this patch available in version 9.2.1 ? Senthil -- View this message in context: http://postgresql.1045698.n5.nabble.com/Switching-timeline-over-streaming-replication-tp5723547p5734744.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via pgsql-hackers

Re: [HACKERS] Switching timeline over streaming replication

2012-12-03 Thread Heikki Linnakangas
On 03.12.2012 14:21, senthilnathan wrote: Is this patch available in version 9.2.1 ? Nope, this is for 9.3. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-27 Thread Heikki Linnakangas
On 26.11.2012 14:53, Amit Kapila wrote: On Friday, November 23, 2012 7:03 PM Heikki Linnakangas This is what I came up with. It adds a new function, OpenFile, that returns a raw file descriptor like BasicOpenFile, but the file descriptor is associated with the current subtransaction and

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Amit Kapila
On Friday, November 23, 2012 7:03 PM Heikki Linnakangas On 15.11.2012 17:16, Heikki Linnakangas wrote: On 15.11.2012 16:55, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Heikki Linnakangas
On 26.11.2012 14:53, Amit Kapila wrote: I have one usecase in feature (SQL Command to edit postgresql.conf) very similar to OpenFile/CloseFile, but I want that when CloseFile is called from abort, it should remove(unlink) the file as well and during open it has to retry few times if open is not

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Amit Kapila
On Monday, November 26, 2012 7:01 PM Heikki Linnakangas wrote: On 26.11.2012 14:53, Amit Kapila wrote: I have one usecase in feature (SQL Command to edit postgresql.conf) very similar to OpenFile/CloseFile, but I want that when CloseFile is called from abort, it should remove(unlink) the

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: On Monday, November 26, 2012 7:01 PM Heikki Linnakangas wrote: Hmm, if it's just for locking purposes, how about using a lwlock or a heavy-weight lock instead? Its not only for lock, the main idea is that we create temp file and write modified

Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-23 Thread Heikki Linnakangas
On 15.11.2012 17:16, Heikki Linnakangas wrote: On 15.11.2012 16:55, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file

Re: [HACKERS] Switching timeline over streaming replication

2012-11-22 Thread Amit Kapila
On Wednesday, November 21, 2012 11:36 PM Heikki Linnakangas wrote: On 20.11.2012 15:33, Amit Kapila wrote: Defect-2: 1. start primary A 2. start standby B following A 3. start cascade standby C following B. 4. Start another standby D following C. 5. Execute

Re: [HACKERS] Switching timeline over streaming replication

2012-11-21 Thread Heikki Linnakangas
On 20.11.2012 15:33, Amit Kapila wrote: Defect-2: 1. start primary A 2. start standby B following A 3. start cascade standby C following B. 4. Start another standby D following C. 5. Execute the following commands in the primary A. create table tbl(f int);

Re: [HACKERS] Switching timeline over streaming replication

2012-11-20 Thread Amit Kapila
On Monday, November 19, 2012 10:54 PM Heikki Linnakangas wrote: On 10.10.2012 17:54, Thom Brown wrote: Hmm... I get something different. When I promote standby B, standby C's log shows: The following problems are observed while testing of the patch. Defect-1: 1. start

Re: [HACKERS] Switching timeline over streaming replication

2012-11-19 Thread Heikki Linnakangas
On 10.10.2012 17:54, Thom Brown wrote: Hmm... I get something different. When I promote standby B, standby C's log shows: LOG: walreceiver ended streaming and awaits new instructions LOG: re-handshaking at position 0/400 on tli 1 LOG: fetching timeline history file for timeline 2

Re: [HACKERS] Switching timeline over streaming replication

2012-11-16 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote:

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: ... ./pg_basebackup -P -D ../../data_sub -X fetch -p 2303

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 10.10.2012 17:26, Amit Kapila wrote: 36.+SendTimeLineHistory(TimeLineHistoryCmd *cmd) { .. if (nread= 0) +ereport(ERROR, +(errcode_for_file_access(), + errmsg(could not read file \%s\:

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: ...

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote:

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file descriptors on error: Um, don't we automatically clean those up during transaction

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 16:55, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file descriptors on error: Um, don't we automatically

Re: [HACKERS] Switching timeline over streaming replication

2012-10-23 Thread Alvaro Herrera
Heikki Linnakangas wrote: Attached is a new version of the patch. I committed the refactoring of XLogPageRead() already, as that was a readability improvement even without this patch. All the reported issues should be fixed now, although I will continue testing this tomorrow. I added various

Re: [HACKERS] Switching timeline over streaming replication

2012-10-12 Thread Amit Kapila
-Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- ow...@postgresql.org] On Behalf Of Amit Kapila Sent: Wednesday, October 10, 2012 7:57 PM To: 'Heikki Linnakangas' Cc: 'PostgreSQL-development' Subject: Re: [HACKERS] Switching timeline over

Re: [HACKERS] Switching timeline over streaming replication

2012-10-10 Thread Amit Kapila
On Tuesday, October 09, 2012 10:32 PM Heikki Linnakangas wrote: On 06.10.2012 15:58, Amit Kapila wrote: One more test seems to be failed. Apart from this, other tests are passed. It seems there is one more defect, please check the same Defect: 1. start primary A 2. start

Re: [HACKERS] Switching timeline over streaming replication

2012-10-09 Thread Heikki Linnakangas
On 06.10.2012 15:58, Amit Kapila wrote: One more test seems to be failed. Apart from this, other tests are passed. 2. a. Master M-1 b. Standby S-1 follows M-1 c. insert 10 records on M-1. verify all records are visible on M-1,S-1 d. Stop S-1 e. insert 2 records on M-1. f.

Re: Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-08 Thread Heikki Linnakangas
On 04.10.2012 20:07, Fujii Masao wrote: On Thu, Oct 4, 2012 at 4:59 PM, Heikki Linnakangas But I wonder why promoting a standby renders the backup invalid in the first place? Fujii, Simon, can you explain that? Simon had the same question and I answered it before.

Re: Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-08 Thread Simon Riggs
On 4 October 2012 18:07, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 4, 2012 at 4:59 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03.10.2012 18:15, Amit Kapila wrote: On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Hmm, should a base backup be aborted

Re: [HACKERS] Switching timeline over streaming replication

2012-10-06 Thread Amit Kapila
On Thursday, October 04, 2012 7:22 PM Heikki Linnakangas wrote: On Wednesday, October 03, 2012 8:45 PM Heikki Linnakangas wrote: On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Thanks for the thorough review! I committed the xlog.c refactoring patch now. Attached is a

Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Heikki Linnakangas
On 03.10.2012 18:15, Amit Kapila wrote: On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Hmm, should a base backup be aborted when the standby is promoted? Does the promotion render the backup corrupt? I think currently it does so. Pls refer 1. do_pg_stop_backup(char *labelfile,

Re: [HACKERS] Switching timeline over streaming replication

2012-10-04 Thread Amit Kapila
On Wednesday, October 03, 2012 8:45 PM Heikki Linnakangas wrote: On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Thanks for the thorough review! I committed the xlog.c refactoring patch now. Attached is a new version of the main patch, comments on specific points below. I

Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Heikki Linnakangas
On 03.10.2012 18:15, Amit Kapila wrote: 35.WalSenderMain(void) { .. +if (walsender_shutdown_requested) +ereport(FATAL, +(errcode(ERRCODE_ADMIN_SHUTDOWN), + errmsg(terminating

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: So I propose the attached patch. I made small changes to postgres.c to make it call exec_replication_command() instead of exec_simple_query(), and reject extend query protocol, in a WAL sender process. A lot of code related to handling the

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Heikki Linnakangas
On 04.10.2012 19:00, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: So I propose the attached patch. I made small changes to postgres.c to make it call exec_replication_command() instead of exec_simple_query(), and reject extend query protocol, in a WAL sender process. A lot

Re: Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Fujii Masao
On Thu, Oct 4, 2012 at 4:59 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03.10.2012 18:15, Amit Kapila wrote: On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Hmm, should a base backup be aborted when the standby is promoted? Does the promotion render the backup

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Simon Riggs
On 4 October 2012 17:23, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04.10.2012 19:00, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: So I propose the attached patch. I made small changes to postgres.c to make it call exec_replication_command() instead of

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 4 October 2012 17:23, Heikki Linnakangas hlinnakan...@vmware.com wrote: Perhaps we could make walsenders even more like regular backends than what I was proposing, so that the replication commands are parsed and executed just like regular utility

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Amit Kapila
On Thursday, October 04, 2012 8:40 PM Heikki Linnakangas wrote: On 03.10.2012 18:15, Amit Kapila wrote: 35.WalSenderMain(void) { .. +if (walsender_shutdown_requested) +ereport(FATAL, + (errcode(ERRCODE_ADMIN_SHUTDOWN), +

Re: [HACKERS] Switching timeline over streaming replication

2012-10-03 Thread Amit Kapila
On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Thanks for the thorough review! I committed the xlog.c refactoring patch now. Attached is a new version of the main patch, comments on specific points below. I didn't adjust the docs per your comments yet, will do that next. I

Re: [HACKERS] Switching timeline over streaming replication

2012-09-30 Thread Amit kapila
On Friday, September 28, 2012 6:38 PM Amit Kapila wrote: On Tuesday, September 25, 2012 6:29 PM Heikki Linnakangas wrote: On 25.09.2012 10:08, Heikki Linnakangas wrote: On 24.09.2012 16:33, Amit Kapila wrote: In any case, it will be better if you can split it into multiple patches: 1.

Re: [HACKERS] Switching timeline over streaming replication

2012-09-28 Thread Amit Kapila
On Tuesday, September 25, 2012 6:29 PM Heikki Linnakangas wrote: On 25.09.2012 10:08, Heikki Linnakangas wrote: On 24.09.2012 16:33, Amit Kapila wrote: In any case, it will be better if you can split it into multiple patches: 1. Having new functionality of Switching timeline over

Re: [HACKERS] Switching timeline over streaming replication

2012-09-27 Thread Hannu Krosing
On 09/26/2012 01:02 AM, m...@rpzdesign.com wrote: John: Who has the money for oracle RAC or funding arrogant bastard Oracle CEO Ellison to purchase another island? Postgres needs CHEAP, easy to setup, self healing, master-master-master-master and it needs it yesterday. I was able to patch

Re: [HACKERS] Switching timeline over streaming replication

2012-09-27 Thread Euler Taveira
On 27-09-2012 01:30, Amit Kapila wrote: I understood this point, but currently in documentation of Timelines, this usecase is not documented (Section 24.3.5). Timeline documentation was written during PITR implementation. There wasn't SR yet. AFAICS it doesn't cite SR but is sufficiently

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread Josh Berkus
I was able to patch the 9.2.0 code base in 1 day and change my entire architecture strategy for replication into self healing async master-master-master and the tiniest bit of sharding code imaginable Sounds cool. Do you have a fork available on Github? I'll try it out. -- Josh Berkus

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread Josh Berkus
Yes that is correct. I thought timeline change happens only when somebody does PITR. Can you please tell me why we change timeline after promotion, because the original Timeline concept was for PITR and I am not able to trace from code the reason why on promotion it is required?

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread m...@rpzdesign.com
Josh: The good part is you are the first person to ask for a copy and I will send you the hook code that I have and you can be a good sport and put it on GitHub, that is great, you can give us both credit for a joint effort, I do the code, you put it GitHub. The not so good part is that the

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread Amit Kapila
On Thursday, September 27, 2012 6:30 AM Josh Berkus wrote: Yes that is correct. I thought timeline change happens only when somebody does PITR. Can you please tell me why we change timeline after promotion, because the original Timeline concept was for PITR and I am not able to

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Heikki Linnakangas
On 24.09.2012 16:33, Amit Kapila wrote: On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: I've been working on the often-requested feature to handle timeline changes over streaming replication. At the moment, if you kill the master and promote a standby server, and you have

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Amit Kapila
On Tuesday, September 25, 2012 12:39 PM Heikki Linnakangas wrote: On 24.09.2012 16:33, Amit Kapila wrote: On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: I've been working on the often-requested feature to handle timeline changes over streaming replication. At the

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Heikki Linnakangas
On 25.09.2012 14:10, Amit Kapila wrote: On Tuesday, September 25, 2012 12:39 PM Heikki Linnakangas wrote: On 24.09.2012 16:33, Amit Kapila wrote: On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: I've been working on the often-requested feature to handle timeline changes

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Amit Kapila
On Tuesday, September 25, 2012 6:29 PM Heikki Linnakangas wrote: On 25.09.2012 10:08, Heikki Linnakangas wrote: On 24.09.2012 16:33, Amit Kapila wrote: In any case, it will be better if you can split it into multiple patches: 1. Having new functionality of Switching timeline over

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread m...@rpzdesign.com
Amit: At some point, every master - slave replicator gets to the point where they need to start thinking about master-master replication. Instead of getting stuck in the weeds to finally realize that master-master is the ONLY way to go, many developers do not start out planning for master -

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Daniel Farina
On Tue, Sep 25, 2012 at 11:01 AM, m...@rpzdesign.com m...@rpzdesign.com wrote: Amit: At some point, every master - slave replicator gets to the point where they need to start thinking about master-master replication. Even in a master-master system, the ability to cleanly swap leaders

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread John R Pierce
On 09/25/12 11:01 AM, m...@rpzdesign.com wrote: At some point, every master - slave replicator gets to the point where they need to start thinking about master-master replication. master-master and transactional integrity are mutually exclusive, except perhaps in special cases like Oracle

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread m...@rpzdesign.com
John: Who has the money for oracle RAC or funding arrogant bastard Oracle CEO Ellison to purchase another island? Postgres needs CHEAP, easy to setup, self healing, master-master-master-master and it needs it yesterday. I was able to patch the 9.2.0 code base in 1 day and change my entire

Re: [HACKERS] Switching timeline over streaming replication

2012-09-24 Thread Amit Kapila
On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: I've been working on the often-requested feature to handle timeline changes over streaming replication. At the moment, if you kill the master and promote a standby server, and you have another standby server that you'd like to

Re: [HACKERS] Switching timeline over streaming replication

2012-09-24 Thread Amit Kapila
On Monday, September 24, 2012 9:08 PM m...@rpzdesign.com wrote: What a disaster waiting to happen. Maybe the only replication should be master-master replication so there is no need to sequence timelines or anything, all servers are ready masters, no backups or failovers. If you really do