Re: [HACKERS] pg_receivexlog and replication slots

2014-10-06 Thread Michael Paquier
On Mon, Oct 6, 2014 at 7:14 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-04 14:25:27 +0900, Michael Paquier wrote: And as I am on it, attached is a patch that can be applied to master and REL9_4_STABLE to rename the --create and --drop to --create-slot and --drop-slot.

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-06 Thread Andres Freund
Hi, On 2014-10-04 15:01:03 +0900, Michael Paquier wrote: On Fri, Oct 3, 2014 at 8:57 PM, Andres Freund and...@2ndquadrant.com wrote: para +applicationpg_receivexlog/application can run in one of two following +modes, which control physical replication slot: I don't

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-06 Thread Michael Paquier
On Mon, Oct 6, 2014 at 8:01 PM, Andres Freund and...@2ndquadrant.com wrote: Pushed with these adjustments. Thanks. The portions changed look fine to me. -- Michael

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Michael Paquier
On Fri, Oct 3, 2014 at 8:57 PM, Andres Freund and...@2ndquadrant.com wrote: para +applicationpg_receivexlog/application can run in one of two following +modes, which control physical replication slot: I don't think that's good enough. There's also the important mode where

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Andres Freund
On 2014-10-04 09:24:07 +0900, Michael Paquier wrote: On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-03 14:02:08 -0400, Robert Haas wrote: On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund and...@2ndquadrant.com wrote: I do wonder whether

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Andres Freund
On 2014-10-04 14:25:27 +0900, Michael Paquier wrote: On Sat, Oct 4, 2014 at 9:24 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-03 14:02:08 -0400, Robert Haas wrote: On Fri, Oct 3, 2014

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Michael Paquier
On Sun, Oct 5, 2014 at 12:09 AM, Andres Freund and...@2ndquadrant.com wrote: Any opinion on whether we should accept both --create and --create-slot or only the latter? Accepting both would get rid of problems due to potential usages of the old syntax - and it's easier to type... My vote goes

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Robert Haas
On Sat, Oct 4, 2014 at 7:03 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sun, Oct 5, 2014 at 12:09 AM, Andres Freund and...@2ndquadrant.com wrote: Any opinion on whether we should accept both --create and --create-slot or only the latter? Accepting both would get rid of problems due

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Andres Freund
On 2014-10-03 10:30:19 +0900, Michael Paquier wrote: On Thu, Oct 2, 2014 at 12:44 AM, Andres Freund and...@2ndquadrant.com wrote: I pushed the first part. Thanks. Attached is a rebased version of patch 2, implementing the actual feature. One thing I noticed with more testing is that if

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund and...@2ndquadrant.com wrote: I do wonder whether --create/--drop aren't somewhat wierd for pg_receivexlog. It's not that clear what it means. It'd be ugly, but we could rename them --create-slot/drop-slot. +1 on doing it, -1 on it being ugly. --

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Andres Freund
On 2014-10-03 14:02:08 -0400, Robert Haas wrote: On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund and...@2ndquadrant.com wrote: I do wonder whether --create/--drop aren't somewhat wierd for pg_receivexlog. It's not that clear what it means. It'd be ugly, but we could rename them

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Michael Paquier
On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-03 14:02:08 -0400, Robert Haas wrote: On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund and...@2ndquadrant.com wrote: I do wonder whether --create/--drop aren't somewhat wierd for pg_receivexlog. It's not

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Michael Paquier
On Sat, Oct 4, 2014 at 9:24 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-03 14:02:08 -0400, Robert Haas wrote: On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund and...@2ndquadrant.com wrote: I

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-02 Thread Michael Paquier
On Thu, Oct 2, 2014 at 12:44 AM, Andres Freund and...@2ndquadrant.com wrote: I pushed the first part. Thanks. Attached is a rebased version of patch 2, implementing the actual feature. One thing I noticed with more testing is that if --create is used and that the destination folder does not

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
Hi, I've split and edited patch 0001: 0001) Old WIP patch for pg_recvlogical tests. Useful for easier development. 0002) Copy editing that should be in 9.4 0003) Rebased patches of yours 0004) My changes to 0003 besides the rebase. This'll be squashed, but I thought it might be interesting

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Michael Paquier
Thanks! On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund and...@2ndquadrant.com wrote: 0001) Old WIP patch for pg_recvlogical tests. Useful for easier development. From where is this patch? Is it some rest from the time when pg_recvlogical was developed? 0002) Copy editing that should be in

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
On 2014-10-01 21:54:56 +0900, Michael Paquier wrote: Thanks! On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund and...@2ndquadrant.com wrote: 0001) Old WIP patch for pg_recvlogical tests. Useful for easier development. From where is this patch? Is it some rest from the time when

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Alvaro Herrera
Andres Freund wrote: From d667f7a63cd62733d88ec5b7228dfd5d7136b9d7 Mon Sep 17 00:00:00 2001 From: Michael Paquier mich...@otacoo.com Date: Mon, 1 Sep 2014 20:48:43 +0900 Subject: [PATCH 3/4] Refactoring of pg_basebackup utilities Code duplication is reduced with the introduction of new

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
On 2014-10-01 11:21:12 -0300, Alvaro Herrera wrote: Andres Freund wrote: From d667f7a63cd62733d88ec5b7228dfd5d7136b9d7 Mon Sep 17 00:00:00 2001 From: Michael Paquier mich...@otacoo.com Date: Mon, 1 Sep 2014 20:48:43 +0900 Subject: [PATCH 3/4] Refactoring of pg_basebackup utilities

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
Hi, I pushed the first part. On 2014-10-01 21:54:56 +0900, Michael Paquier wrote: On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund and...@2ndquadrant.com wrote: 0001) Old WIP patch for pg_recvlogical tests. Useful for easier development. From where is this patch? Is it some rest from the

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Jan Wieck
On 10/01/2014 08:59 AM, Andres Freund wrote: On 2014-10-01 21:54:56 +0900, Michael Paquier wrote: Thanks! On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund and...@2ndquadrant.com wrote: 0001) Old WIP patch for pg_recvlogical tests. Useful for easier development. From where is this patch? Is it

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Jan Wieck
On 10/01/2014 01:19 PM, Andres Freund wrote: On 2014-10-01 13:17:17 -0400, Jan Wieck wrote: -static void StreamLog(); +static void StreamLogicalLog(); Not related at all to those patches, but for correctness it is better to add a declaration (void). Agreed. Except those small things the

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
On 2014-10-01 13:22:53 -0400, Jan Wieck wrote: On 10/01/2014 01:19 PM, Andres Freund wrote: On 2014-10-01 13:17:17 -0400, Jan Wieck wrote: -static void StreamLog(); +static void StreamLogicalLog(); Not related at all to those patches, but for correctness it is better to add a declaration

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Jan Wieck
On 10/01/2014 01:32 PM, Andres Freund wrote: On 2014-10-01 13:22:53 -0400, Jan Wieck wrote: On 10/01/2014 01:19 PM, Andres Freund wrote: On 2014-10-01 13:17:17 -0400, Jan Wieck wrote: -static void StreamLog(); +static void StreamLogicalLog(); Not related at all to those patches, but for

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-29 Thread Andres Freund
On 2014-09-22 15:40:41 +0900, Michael Paquier wrote: Updated patches attached. These are patches about pg_dump. I don't think these were the ones you wanted to attach... :) Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-29 Thread Michael Paquier
On Mon, Sep 29, 2014 at 9:42 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-22 15:40:41 +0900, Michael Paquier wrote: Updated patches attached. These are patches about pg_dump. I don't think these were the ones you wanted to attach... :) Wah, sorry (bakabakashii...). Here they

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-22 Thread Michael Paquier
On Mon, Sep 22, 2014 at 2:25 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Sat, Sep 20, 2014 at 10:06 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Sep 20, 2014 at 7:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: 3. I find existing comments okay, is there a need to

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-21 Thread Amit Kapila
On Sat, Sep 20, 2014 at 10:06 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Sep 20, 2014 at 7:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: 3. I find existing comments okay, is there a need to change in this case? Part of the new comment mentions obtaining start LSN

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-20 Thread Amit Kapila
On Wed, Sep 10, 2014 at 10:09 AM, Michael Paquier michael.paqu...@gmail.com wrote: New patches taking into account all those comments are attached. I have looked into refactoring related patch and would like to share my observations with you: 1. + * Run IDENTIFY_SYSTEM through a given

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-20 Thread Michael Paquier
On Sat, Sep 20, 2014 at 7:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: I have looked into refactoring related patch and would like to share my observations with you: Thanks! Useful input is always welcome. 1. + * Run IDENTIFY_SYSTEM through a given connection and give back to caller

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-09 Thread Michael Paquier
On Mon, Sep 8, 2014 at 8:50 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Sep 3, 2014 at 11:40 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Aug 31, 2014 at 9:45 AM, Magnus Hagander mag...@hagander.net wrote: Do we really want those Asserts? There is not a single Assert in

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-08 Thread Michael Paquier
On Wed, Sep 3, 2014 at 11:40 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Aug 31, 2014 at 9:45 AM, Magnus Hagander mag...@hagander.net wrote: Do we really want those Asserts? There is not a single Assert in bin/pg_basebackup today - as is the case for most things in bin/. We typically

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-03 Thread Robert Haas
On Sun, Aug 31, 2014 at 9:45 AM, Magnus Hagander mag...@hagander.net wrote: Do we really want those Asserts? There is not a single Assert in bin/pg_basebackup today - as is the case for most things in bin/. We typically use regular if statements for things that can happen, and just ignore the

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-01 Thread Michael Paquier
On Sun, Aug 31, 2014 at 10:45 PM, Magnus Hagander mag...@hagander.net wrote: As this is a number of patches rolled into one - do you happen to keep them separate in your local repo? If so can you send them as separate ones (refactor identify_system should be quite unrelated to supporting

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-01 Thread Andres Freund
On 2014-09-01 20:58:29 +0900, Michael Paquier wrote: On Sun, Aug 31, 2014 at 10:45 PM, Magnus Hagander mag...@hagander.net wrote: As this is a number of patches rolled into one - do you happen to keep them separate in your local repo? If so can you send them as separate ones (refactor

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-31 Thread Magnus Hagander
On Tue, Aug 26, 2014 at 9:43 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Aug 19, 2014 at 2:49 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-26 Thread Michael Paquier
On Tue, Aug 19, 2014 at 2:49 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Fujii Masao
On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier michael.paqu...@gmail.com wrote: - IDENTIFY_SYSTEM checks were incorrect (even in HEAD).

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Andres Freund
On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following check was done but in 9.4 this command returns 4 fields: (PQntuples(res) != 1 || PQnfields(res) 3) Which is correct. We don't want to error out in the case where 3

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Fujii Masao
On Tue, Aug 19, 2014 at 5:52 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following check was done but in 9.4 this command returns 4 fields: (PQntuples(res) != 1 ||

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Andres Freund
On 2014-08-19 18:02:32 +0900, Fujii Masao wrote: On Tue, Aug 19, 2014 at 5:52 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following check was done but in 9.4 this command

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Fujii Masao
On Tue, Aug 19, 2014 at 6:03 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-08-19 18:02:32 +0900, Fujii Masao wrote: On Tue, Aug 19, 2014 at 5:52 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: - IDENTIFY_SYSTEM checks were

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-18 Thread Fujii Masao
On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Aug 15, 2014 at 5:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: Thanks for your review. On Fri, Aug 15, 2014 at 12:56 AM, furu...@pm.nttdata.co.jp wrote: At consistency with pg_recvlogical,

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-18 Thread Fujii Masao
On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Aug 15, 2014 at 5:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: Thanks for your review. On Fri, Aug 15, 2014 at 12:56

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-18 Thread Michael Paquier
On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier michael.paqu...@gmail.com wrote: - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following check was done but in 9.4 this command returns 4 fields:

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-18 Thread Michael Paquier
On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier michael.paqu...@gmail.com wrote: And now looking at your patch there is additional

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-17 Thread Michael Paquier
On Fri, Aug 15, 2014 at 5:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: Thanks for your review. On Fri, Aug 15, 2014 at 12:56 AM, furu...@pm.nttdata.co.jp wrote: At consistency with pg_recvlogical, do you think about --start? I did not add that for the sake of

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-15 Thread furuyao
Actually I came up with the same need as Magnus, but a bit later, so... Attached is a patch to support physical slot creation and drop in pg_receivexlog with the addition of new options --create and --drop. It would be nice to have that in 9.4, but I will not the one deciding that at the end

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-15 Thread Michael Paquier
Thanks for your review. On Fri, Aug 15, 2014 at 12:56 AM, furu...@pm.nttdata.co.jp wrote: At consistency with pg_recvlogical, do you think about --start? I did not add that for the sake of backward-compatibility as in pg_recvlogical an action is mandatory. It is not the case now of

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-12 Thread Michael Paquier
On Fri, Jul 11, 2014 at 6:23 PM, Andres Freund and...@2ndquadrant.com wrote: Ok. Do you plan to take care of it? If, I'd be fine with backpatching it. I'm not likely to get to it right now :( Actually I came up with the same need as Magnus, but a bit later, so... Attached is a patch to support

[HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Magnus Hagander
Is there a particular reason why pg_receivexlog only supports using manually created slots but pg_recvlogical supports creating and dropping them? Wouldn't it be good for consistency there? I'm guessing it's related to not being exposed over the replication protocol? We had a discussion earlier

Re: [HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Andres Freund
On 2014-07-11 11:08:48 +0200, Magnus Hagander wrote: Is there a particular reason why pg_receivexlog only supports using manually created slots but pg_recvlogical supports creating and dropping them? Wouldn't it be good for consistency there? I've added it to recvlogical because logical

Re: [HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Magnus Hagander
On Fri, Jul 11, 2014 at 11:14 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-07-11 11:08:48 +0200, Magnus Hagander wrote: Is there a particular reason why pg_receivexlog only supports using manually created slots but pg_recvlogical supports creating and dropping them? Wouldn't it be

Re: [HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Andres Freund
On 2014-07-11 11:18:58 +0200, Magnus Hagander wrote: On Fri, Jul 11, 2014 at 11:14 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-07-11 11:08:48 +0200, Magnus Hagander wrote: Is there a particular reason why pg_receivexlog only supports using manually created slots but