Re: [HACKERS] pg_receivexlog and replication slots

2014-10-06 Thread Michael Paquier
On Mon, Oct 6, 2014 at 8:01 PM, Andres Freund wrote: > Pushed with these adjustments. > Thanks. The portions changed look fine to me. -- Michael

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-06 Thread Andres Freund
Hi, On 2014-10-04 15:01:03 +0900, Michael Paquier wrote: > On Fri, Oct 3, 2014 at 8:57 PM, Andres Freund wrote: > > > > > > +pg_receivexlog can run in one of two > > > following > > > +modes, which control physical replication slot: > > > > I don't think that's good enough. There's

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-06 Thread Michael Paquier
On Mon, Oct 6, 2014 at 7:14 PM, Andres Freund wrote: > On 2014-10-04 14:25:27 +0900, Michael Paquier wrote: > > And as I am on it, attached is a patch that can be applied to master and > > REL9_4_STABLE to rename the --create and --drop to --create-slot and > > --drop-slot. > > Thanks. > > > diff

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-06 Thread Andres Freund
On 2014-10-04 14:25:27 +0900, Michael Paquier wrote: > And as I am on it, attached is a patch that can be applied to master and > REL9_4_STABLE to rename the --create and --drop to --create-slot and > --drop-slot. Thanks. > diff --git a/src/bin/pg_basebackup/pg_recvlogical.c > b/src/bin/pg_baseb

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Robert Haas
On Sat, Oct 4, 2014 at 7:03 PM, Michael Paquier wrote: > On Sun, Oct 5, 2014 at 12:09 AM, Andres Freund wrote: >> Any opinion on whether we should accept both --create and --create-slot >> or only the latter? Accepting both would get rid of problems due to >> potential usages of the old syntax -

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Michael Paquier
On Sun, Oct 5, 2014 at 12:09 AM, Andres Freund wrote: > Any opinion on whether we should accept both --create and --create-slot > or only the latter? Accepting both would get rid of problems due to > potential usages of the old syntax - and it's easier to type... My vote goes for only --create-slo

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Andres Freund
On 2014-10-04 14:25:27 +0900, Michael Paquier wrote: > On Sat, Oct 4, 2014 at 9:24 AM, Michael Paquier > wrote: > > > > > > > On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund > > wrote: > > > >> On 2014-10-03 14:02:08 -0400, Robert Haas wrote: > >> > On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund >

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-04 Thread Andres Freund
On 2014-10-04 09:24:07 +0900, Michael Paquier wrote: > On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund > wrote: > > > On 2014-10-03 14:02:08 -0400, Robert Haas wrote: > > > On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund > > wrote: > > > > I do wonder whether --create/--drop aren't somewhat wierd fo

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Michael Paquier
On Fri, Oct 3, 2014 at 8:57 PM, Andres Freund wrote: > > > > +pg_receivexlog can run in one of two > > following > > +modes, which control physical replication slot: > > I don't think that's good enough. There's also the important mode where > it's not doing --create/--drop at all. W

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Michael Paquier
On Sat, Oct 4, 2014 at 9:24 AM, Michael Paquier wrote: > > > On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund > wrote: > >> On 2014-10-03 14:02:08 -0400, Robert Haas wrote: >> > On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund >> wrote: >> > > I do wonder whether --create/--drop aren't somewhat wierd

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Michael Paquier
On Sat, Oct 4, 2014 at 6:48 AM, Andres Freund wrote: > On 2014-10-03 14:02:08 -0400, Robert Haas wrote: > > On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund > wrote: > > > I do wonder whether --create/--drop aren't somewhat wierd for > > > pg_receivexlog. It's not that clear what it means. It'd be

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Andres Freund
On 2014-10-03 14:02:08 -0400, Robert Haas wrote: > On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund wrote: > > I do wonder whether --create/--drop aren't somewhat wierd for > > pg_receivexlog. It's not that clear what it means. It'd be ugly, but we > > could rename them --create-slot/drop-slot. > >

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund wrote: > I do wonder whether --create/--drop aren't somewhat wierd for > pg_receivexlog. It's not that clear what it means. It'd be ugly, but we > could rename them --create-slot/drop-slot. +1 on doing it, -1 on it being ugly. -- Robert Haas Enterpr

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Andres Freund
On 2014-10-03 10:30:19 +0900, Michael Paquier wrote: > On Thu, Oct 2, 2014 at 12:44 AM, Andres Freund > wrote: > > > I pushed the first part. > > > Thanks. Attached is a rebased version of patch 2, implementing the actual > feature. One thing I noticed with more testing is that if --create is use

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-02 Thread Michael Paquier
On Thu, Oct 2, 2014 at 12:44 AM, Andres Freund wrote: > I pushed the first part. > Thanks. Attached is a rebased version of patch 2, implementing the actual feature. One thing I noticed with more testing is that if --create is used and that the destination folder does not exist, pg_receivexlog wa

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Jan Wieck
On 10/01/2014 01:32 PM, Andres Freund wrote: On 2014-10-01 13:22:53 -0400, Jan Wieck wrote: On 10/01/2014 01:19 PM, Andres Freund wrote: >On 2014-10-01 13:17:17 -0400, Jan Wieck wrote: -static void StreamLog(); +static void StreamLogicalLog(); Not related at all to those patches, but

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
On 2014-10-01 13:22:53 -0400, Jan Wieck wrote: > On 10/01/2014 01:19 PM, Andres Freund wrote: > >On 2014-10-01 13:17:17 -0400, Jan Wieck wrote: > -static void StreamLog(); > +static void StreamLogicalLog(); > Not related at all to those patches, but for correctness it is better to > >>>

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Jan Wieck
On 10/01/2014 01:19 PM, Andres Freund wrote: On 2014-10-01 13:17:17 -0400, Jan Wieck wrote: >>-static void StreamLog(); >>+static void StreamLogicalLog(); >>Not related at all to those patches, but for correctness it is better to >>add a declaration "(void)". > >Agreed. > >>Except those small th

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
On 2014-10-01 13:17:17 -0400, Jan Wieck wrote: > >>-static void StreamLog(); > >>+static void StreamLogicalLog(); > >>Not related at all to those patches, but for correctness it is better to > >>add a declaration "(void)". > > > >Agreed. > > > >>Except those small things the changes look good to me

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Jan Wieck
On 10/01/2014 08:59 AM, Andres Freund wrote: On 2014-10-01 21:54:56 +0900, Michael Paquier wrote: Thanks! On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund wrote: > 0001) Old WIP patch for pg_recvlogical tests. Useful for easier > development. > From where is this patch? Is it some rest from the

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
Hi, I pushed the first part. On 2014-10-01 21:54:56 +0900, Michael Paquier wrote: > On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund > wrote: > > 0001) Old WIP patch for pg_recvlogical tests. Useful for easier > > development. > From where is this patch? Is it some rest from the time when pg_recvlo

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
On 2014-10-01 11:21:12 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > From d667f7a63cd62733d88ec5b7228dfd5d7136b9d7 Mon Sep 17 00:00:00 2001 > > From: Michael Paquier > > Date: Mon, 1 Sep 2014 20:48:43 +0900 > > Subject: [PATCH 3/4] Refactoring of pg_basebackup utilities > > > > Code

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Alvaro Herrera
Andres Freund wrote: > From d667f7a63cd62733d88ec5b7228dfd5d7136b9d7 Mon Sep 17 00:00:00 2001 > From: Michael Paquier > Date: Mon, 1 Sep 2014 20:48:43 +0900 > Subject: [PATCH 3/4] Refactoring of pg_basebackup utilities > > Code duplication is reduced with the introduction of new APIs for each >

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
On 2014-10-01 21:54:56 +0900, Michael Paquier wrote: > Thanks! > > On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund > wrote: > > > 0001) Old WIP patch for pg_recvlogical tests. Useful for easier > > development. > > > From where is this patch? Is it some rest from the time when pg_recvlogical > was

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Michael Paquier
Thanks! On Wed, Oct 1, 2014 at 8:38 PM, Andres Freund wrote: > 0001) Old WIP patch for pg_recvlogical tests. Useful for easier > development. > >From where is this patch? Is it some rest from the time when pg_recvlogical was developed? > 0002) Copy editing that should be in 9.4 > Definitely ma

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-01 Thread Andres Freund
Hi, I've split and edited patch 0001: 0001) Old WIP patch for pg_recvlogical tests. Useful for easier development. 0002) Copy editing that should be in 9.4 0003) Rebased patches of yours 0004) My changes to 0003 besides the rebase. This'll be squashed, but I thought it might be interesting f

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-29 Thread Michael Paquier
On Mon, Sep 29, 2014 at 9:42 PM, Andres Freund wrote: > On 2014-09-22 15:40:41 +0900, Michael Paquier wrote: > > Updated patches attached. > > These are patches about pg_dump. I don't think these were the ones you > wanted to attach... :) > Wah, sorry (bakabakashii...). Here they are. -- Michael

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-29 Thread Andres Freund
On 2014-09-22 15:40:41 +0900, Michael Paquier wrote: > Updated patches attached. These are patches about pg_dump. I don't think these were the ones you wanted to attach... :) Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-21 Thread Michael Paquier
On Mon, Sep 22, 2014 at 2:25 PM, Amit Kapila wrote: > On Sat, Sep 20, 2014 at 10:06 PM, Michael Paquier > wrote: >> On Sat, Sep 20, 2014 at 7:09 AM, Amit Kapila >> wrote: >> > 3. >> > I find existing comments okay, is there a need to change >> > in this case? Part of the new comment mentions >>

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-21 Thread Amit Kapila
On Sat, Sep 20, 2014 at 10:06 PM, Michael Paquier wrote: > On Sat, Sep 20, 2014 at 7:09 AM, Amit Kapila wrote: > > 3. > > I find existing comments okay, is there a need to change > > in this case? Part of the new comment mentions > > "obtaining start LSN position", actually the start position is

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-20 Thread Michael Paquier
On Sat, Sep 20, 2014 at 7:09 AM, Amit Kapila wrote: > I have looked into refactoring related patch and would like > to share my observations with you: Thanks! Useful input is always welcome. > 1. > + * Run IDENTIFY_SYSTEM through a given connection and give back to caller > This API also gets plu

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-20 Thread Amit Kapila
On Wed, Sep 10, 2014 at 10:09 AM, Michael Paquier wrote: > > > New patches taking into account all those comments are attached. I have looked into refactoring related patch and would like to share my observations with you: 1. + * Run IDENTIFY_SYSTEM through a given connection and give back to ca

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-09 Thread Michael Paquier
On Mon, Sep 8, 2014 at 8:50 PM, Michael Paquier wrote: > On Wed, Sep 3, 2014 at 11:40 PM, Robert Haas wrote: >> On Sun, Aug 31, 2014 at 9:45 AM, Magnus Hagander wrote: >>> Do we really want those Asserts? There is not a single Assert in >>> bin/pg_basebackup today - as is the case for most thing

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-08 Thread Michael Paquier
On Wed, Sep 3, 2014 at 11:40 PM, Robert Haas wrote: > On Sun, Aug 31, 2014 at 9:45 AM, Magnus Hagander wrote: >> Do we really want those Asserts? There is not a single Assert in >> bin/pg_basebackup today - as is the case for most things in bin/. We >> typically use regular if statements for thin

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-03 Thread Robert Haas
On Sun, Aug 31, 2014 at 9:45 AM, Magnus Hagander wrote: > Do we really want those Asserts? There is not a single Assert in > bin/pg_basebackup today - as is the case for most things in bin/. We > typically use regular if statements for things that "can happen", and > just ignore the others I think

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-01 Thread Andres Freund
On 2014-09-01 20:58:29 +0900, Michael Paquier wrote: > On Sun, Aug 31, 2014 at 10:45 PM, Magnus Hagander wrote: > > As this is a number of patches rolled into one - do you happen to keep > > them separate in your local repo? If so can you send them as separate > > ones (refactor identify_system sh

Re: [HACKERS] pg_receivexlog and replication slots

2014-09-01 Thread Michael Paquier
On Sun, Aug 31, 2014 at 10:45 PM, Magnus Hagander wrote: > As this is a number of patches rolled into one - do you happen to keep > them separate in your local repo? If so can you send them as separate > ones (refactor identify_system should be quite unrelated to supporting > replication slots, ri

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-31 Thread Magnus Hagander
On Tue, Aug 26, 2014 at 9:43 AM, Michael Paquier wrote: > On Tue, Aug 19, 2014 at 2:49 PM, Michael Paquier > wrote: >> On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier >> wrote: >>> On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao wrote: On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier w

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-26 Thread Michael Paquier
On Tue, Aug 19, 2014 at 2:49 PM, Michael Paquier wrote: > On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier > wrote: >> On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao wrote: >>> On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier >>> wrote: >> And now looking at your patch there is additional refacto

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Fujii Masao
On Tue, Aug 19, 2014 at 6:03 PM, Andres Freund wrote: > On 2014-08-19 18:02:32 +0900, Fujii Masao wrote: >> On Tue, Aug 19, 2014 at 5:52 PM, Andres Freund >> wrote: >> > On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: >> >> - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The followi

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Andres Freund
On 2014-08-19 18:02:32 +0900, Fujii Masao wrote: > On Tue, Aug 19, 2014 at 5:52 PM, Andres Freund wrote: > > On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: > >> - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following > >> check was done but in 9.4 this command returns 4 fields:

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Fujii Masao
On Tue, Aug 19, 2014 at 5:52 PM, Andres Freund wrote: > On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: >> - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following >> check was done but in 9.4 this command returns 4 fields: >> (PQntuples(res) != 1 || PQnfields(res) < 3) > > Which

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Andres Freund
On 2014-08-18 14:38:06 +0900, Michael Paquier wrote: > - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following > check was done but in 9.4 this command returns 4 fields: > (PQntuples(res) != 1 || PQnfields(res) < 3) Which is correct. We don't want to error out in the case where 3 col

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-19 Thread Fujii Masao
On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier wrote: > On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao wrote: >> On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier >> wrote: >>> - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following >>> check was done but in 9.4 this command returns

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-18 Thread Michael Paquier
On Mon, Aug 18, 2014 at 4:01 PM, Michael Paquier wrote: > On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao wrote: >> On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier >> wrote: > And now looking at your patch there is additional refactoring possible > with IDENTIFY_SYSTEM and pg_basebackup as well...

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-18 Thread Michael Paquier
On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao wrote: > On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier > wrote: >> - IDENTIFY_SYSTEM checks were incorrect (even in HEAD). The following >> check was done but in 9.4 this command returns 4 fields: >> (PQntuples(res) != 1 || PQnfields(res) < 3) >> Th

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-17 Thread Fujii Masao
On Mon, Aug 18, 2014 at 3:48 PM, Fujii Masao wrote: > On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier > wrote: >> On Fri, Aug 15, 2014 at 5:17 PM, Michael Paquier >> wrote: >>> Thanks for your review. >>> >>> On Fri, Aug 15, 2014 at 12:56 AM, wrote: At consistency with pg_recvlogical, do

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-17 Thread Fujii Masao
On Mon, Aug 18, 2014 at 2:38 PM, Michael Paquier wrote: > On Fri, Aug 15, 2014 at 5:17 PM, Michael Paquier > wrote: >> Thanks for your review. >> >> On Fri, Aug 15, 2014 at 12:56 AM, wrote: >>> At consistency with pg_recvlogical, do you think about --start? >> I did not add that for the sake of

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-17 Thread Michael Paquier
On Fri, Aug 15, 2014 at 5:17 PM, Michael Paquier wrote: > Thanks for your review. > > On Fri, Aug 15, 2014 at 12:56 AM, wrote: >> At consistency with pg_recvlogical, do you think about --start? > I did not add that for the sake of backward-compatibility as in > pg_recvlogical an action is mandat

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-15 Thread Michael Paquier
Thanks for your review. On Fri, Aug 15, 2014 at 12:56 AM, wrote: > At consistency with pg_recvlogical, do you think about --start? I did not add that for the sake of backward-compatibility as in pg_recvlogical an action is mandatory. It is not the case now of pg_receivexlog. > [postgres postgre

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-15 Thread furuyao
> Actually I came up with the same need as Magnus, but a bit later, so... > Attached is a patch to support physical slot creation and drop in > pg_receivexlog with the addition of new options --create and --drop. It > would be nice to have that in 9.4, but I will not the one deciding that > at the

Re: [HACKERS] pg_receivexlog and replication slots

2014-08-12 Thread Michael Paquier
On Fri, Jul 11, 2014 at 6:23 PM, Andres Freund wrote: > Ok. Do you plan to take care of it? If, I'd be fine with backpatching > it. I'm not likely to get to it right now :( Actually I came up with the same need as Magnus, but a bit later, so... Attached is a patch to support physical slot creatio

Re: [HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Andres Freund
On 2014-07-11 11:18:58 +0200, Magnus Hagander wrote: > On Fri, Jul 11, 2014 at 11:14 AM, Andres Freund > wrote: > > On 2014-07-11 11:08:48 +0200, Magnus Hagander wrote: > >> Is there a particular reason why pg_receivexlog only supports using > >> manually created slots but pg_recvlogical supports

Re: [HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Magnus Hagander
On Fri, Jul 11, 2014 at 11:14 AM, Andres Freund wrote: > On 2014-07-11 11:08:48 +0200, Magnus Hagander wrote: >> Is there a particular reason why pg_receivexlog only supports using >> manually created slots but pg_recvlogical supports creating and >> dropping them? Wouldn't it be good for consiste

Re: [HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Andres Freund
On 2014-07-11 11:08:48 +0200, Magnus Hagander wrote: > Is there a particular reason why pg_receivexlog only supports using > manually created slots but pg_recvlogical supports creating and > dropping them? Wouldn't it be good for consistency there? I've added it to recvlogical because logical deco

[HACKERS] pg_receivexlog and replication slots

2014-07-11 Thread Magnus Hagander
Is there a particular reason why pg_receivexlog only supports using manually created slots but pg_recvlogical supports creating and dropping them? Wouldn't it be good for consistency there? I'm guessing it's related to not being exposed over the replication protocol? We had a discussion earlier th