Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-29 Thread Simon Riggs 
On 23 August 2016 at 14:57, Michael Paquier  wrote:
> On Tue, Aug 23, 2016 at 9:48 PM, Gabriele Bartolini
>  wrote:
>> Hi Simon and Michael,
>>
>> 2016-08-23 10:39 GMT+02:00 Simon Riggs :
>>>
>>> Agreed, but I'd move all the comments above the block.
>>
>>
>> That's fine with me.
>
> +1.

Committed and backpatched to 9.5. Thank you both.

-- 
Simon Riggshttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-23 Thread Michael Paquier 
On Tue, Aug 23, 2016 at 9:48 PM, Gabriele Bartolini
 wrote:
> Hi Simon and Michael,
>
> 2016-08-23 10:39 GMT+02:00 Simon Riggs :
>>
>> Agreed, but I'd move all the comments above the block.
>
>
> That's fine with me.

+1.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-23 Thread Gabriele Bartolini 
Hi Simon and Michael,

2016-08-23 10:39 GMT+02:00 Simon Riggs :
>
> Agreed, but I'd move all the comments above the block.
>

That's fine with me.

Thanks,
Gabriele
--
 Gabriele Bartolini - 2ndQuadrant Italia - Director
 PostgreSQL Training, Services and Support
 gabriele.bartol...@2ndquadrant.it | www.2ndQuadrant.it

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-23 Thread Simon Riggs 
On 23 August 2016 at 08:56, Michael Paquier  wrote:
> On Tue, Aug 23, 2016 at 7:34 AM, Gabriele Bartolini
>  wrote:

 I'd suggest rewording it a bit instead, please see attached.
> And of course this needs a backpatch.

Agreed, but I'd move all the comments above the block.

-- 
Simon Riggshttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


pg-receivexlog-not-report-sync-flush.v2.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-23 Thread Michael Paquier 
On Tue, Aug 23, 2016 at 7:34 AM, Gabriele Bartolini
 wrote:
>   Looking up the code in more details I see that, unless replication slot
> are enabled, pg_receivexlog does not report the flush position (this is a
> precaution that's been taken when '--synchronous' was probably not around).
> Please find this very short patch which - in case replication slots are not
> present but synchronous is - reports the flush position.

Ouch.

>I am not sure if it is a bug or not. I any case I guess we should
> probably improve the documentation - it's a bit late here so maybe I can try
> better tomorrow with a fresher mind. :)

Looking at c4f99d20, there is the following thing from the commit log:
This commit removes -F option from and adds --synchronous to pg_receivexlog.
If --synchronous is specified, like the standby's wal receiver, pg_receivexlog
flushes WAL data as soon as there is WAL data which has not been flushed yet.
Then it sends back the feedback message identifying the latest flush location
to the server. This option is useful to make pg_receivexlog behave as sync
standby by using replication slot, for example.

The docs also clearly state what --sync should do:
Also send a status packet back to the server immediately after
flushing, regardless of --status-interval.

So it means that you are right, this is a bug and we should report the
flush position back to the server so as pg_receivexlog can be thought
as a synchronous replica. Looking at your patch, I think that this is
the right shot, but I don't think that the first comment block should
be completely removed as it still applies when replication slots are
enabled. I'd suggest rewording it a bit instead, please see attached.
And of course this needs a backpatch.
-- 
Michael
diff --git a/src/bin/pg_basebackup/receivelog.c 
b/src/bin/pg_basebackup/receivelog.c
index 595213f..94f1f37 100644
--- a/src/bin/pg_basebackup/receivelog.c
+++ b/src/bin/pg_basebackup/receivelog.c
@@ -509,7 +509,7 @@ ReceiveXlogStream(PGconn *conn, StreamCtl *stream)
 * Report the flush position, so the primary can know what WAL 
we'll
 * possibly re-request, and remove older WAL safely.
 *
-* We only report it when a slot has explicitly been used, 
because
+* We always report it when a slot has explicitly been used, 
because
 * reporting the flush position makes one eligible as a 
synchronous
 * replica. People shouldn't include generic names in
 * synchronous_standby_names, but we've protected them against 
it so
@@ -522,7 +522,14 @@ ReceiveXlogStream(PGconn *conn, StreamCtl *stream)
}
else
{
-   reportFlushPosition = false;
+   /*
+* If 'synchronous' is true, report flush position 
unconditionally
+* to make pg_receivexlog eligible as a synchronous replica.
+*/
+   if (stream->synchronous)
+   reportFlushPosition = true;
+   else
+   reportFlushPosition = false;
slotcmd[0] = 0;
}
 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-22 Thread Gabriele Bartolini 
Hi guys,

  while adding synchronous WAL streaming to Barman, I noticed that
pg_receivexlog - unless a replication slot is specified and --synchronous
is passed - does not become a synchronous receiver (if the application_name
is in the synchronous_standby_names value). I was a bit surprised by this
behaviour.

  By reading the pg_receivexlog documentation, I assumed that:

1) if I set application_name properly for pg_receivexlog (let's say
'barman_receive_wal')
2) then I set synchronous_standby_names so that barman_receive_wal is first
in the list
3) then I run pg_receivexlog with --synchronous

  I would find the pg_receivexlog in 'sync' state in the
pg_stat_replication view on the master.

  However, I kept receiving the 'async' state. After looking at the
documentation once more, I noticed that '--synchronous' was mentioned also
in the '--slot-name' option but the explanation - at least to me - was not
very clear.

  I tried again by creating a replication slot and passing it to
pg_receivexlog and this time I could see 'sync' as streaming state.

  Looking up the code in more details I see that, unless replication slot
are enabled, pg_receivexlog does not report the flush position (this is a
precaution that's been taken when '--synchronous' was probably not around).
Please find this very short patch which - in case replication slots are not
present but synchronous is - reports the flush position.

   I am not sure if it is a bug or not. I any case I guess we should
probably improve the documentation - it's a bit late here so maybe I can
try better tomorrow with a fresher mind. :)

Thanks,
Gabriele
--
 Gabriele Bartolini - 2ndQuadrant Italia - Director
 PostgreSQL Training, Services and Support
 gabriele.bartol...@2ndquadrant.it | www.2ndQuadrant.it


0001-pg_receivexlog-does-not-report-flush-position-with-s.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers