Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 September 2016 at 16:33, Petr Jelinek wrote: >> The better alternative is to add a variant on >> pg_logical_slot_get_changes(...) etc that accepts a start LSN. But >> it's not convenient or easy for SQL callers to extract the last commit >> LSN received from the last

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 September 2016 at 03:41, Craig Ringer wrote: > On 2 September 2016 at 17:49, Craig Ringer wrote: > >> So the main change becomes the one-liner in my prior mail. > > Per feedback from Simon, updated with a new test in src/test/recovery . > > If

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 05/09/16 04:41, Craig Ringer wrote: On 2 September 2016 at 17:49, Craig Ringer wrote: So the main change becomes the one-liner in my prior mail. Per feedback from Simon, updated with a new test in src/test/recovery . If you revert the change to

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 September 2016 at 10:41, Craig Ringer wrote: > On 2 September 2016 at 17:49, Craig Ringer wrote: > >> So the main change becomes the one-liner in my prior mail. > > Per feedback from Simon, updated with a new test in src/test/recovery . ...

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 2 September 2016 at 17:49, Craig Ringer wrote: > So the main change becomes the one-liner in my prior mail. Per feedback from Simon, updated with a new test in src/test/recovery . If you revert the change to src/backend/replication/logical/logicalfuncs.c then the test

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 2016-09-02 10:50:17 +0200, Petr Jelinek wrote: > Okay that sounds reasonable, the SQL interface is already somewhat different > than walsender as it does not really "stream" so makes sense to improve the > behavior there. As a side note, I would really like to have cursor-like SQL > interface

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 2 September 2016 at 16:50, Petr Jelinek wrote: >> So here's a simpler patch that just dirties the slot when it's >> replayed something from it on the SQL interface [...] > > Okay that sounds reasonable, the SQL interface is already somewhat different > than walsender as

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 02/09/16 09:58, Craig Ringer wrote: On 1 September 2016 at 21:19, Simon Riggs wrote: I agree the doc patch should go in, though I suggest reword it slightly, like attached patch. Thanks. The rewording looks good to me and I think that

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 1 September 2016 at 21:19, Simon Riggs wrote: > I agree the doc patch should go in, though I suggest reword it > slightly, like attached patch. Thanks. The rewording looks good to me and I think that Doc-correction-each-change-once.v2.patch is ready for commit.

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 20 August 2016 at 15:04, Petr Jelinek wrote: > On 20/08/16 16:01, Craig Ringer wrote: >> >> On 5 June 2016 at 09:54, David G. Johnston > > wrote: >> >> On Thursday, March 17, 2016, Craig Ringer

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 20/08/16 16:01, Craig Ringer wrote: On 5 June 2016 at 09:54, David G. Johnston > wrote: On Thursday, March 17, 2016, Craig Ringer > wrote: The first patch was

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 June 2016 at 09:54, David G. Johnston wrote: > On Thursday, March 17, 2016, Craig Ringer wrote: > >> The first patch was incorrectly created on top of failover slots not >> HEAD. Attached patch applies on HEAD. >> > > Lots of logical

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On Thursday, March 17, 2016, Craig Ringer wrote: > The first patch was incorrectly created on top of failover slots not HEAD. > Attached patch applies on HEAD. > Lots of logical decoding work ongoing but this one shows as active in the September cf and the comments by

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

The first patch was incorrectly created on top of failover slots not HEAD. Attached patch applies on HEAD. From 87d839f8a2e78abb17fa985502fd5b66f0872b57 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Wed, 16 Mar 2016 15:45:16 +0800 Subject: [PATCH 1/2] Correct incorrect

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

Attached is a patch to mark a logical replication slot as dirty if its confirmed lsn is changed. Aesthetically I'm not sure if it's better to do it per this patch and call ReplicationSlotMarkDirty after we release the spinlock, add a new ReplicationSlotMarkDirtyLocked() that skips the spinlock

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 14/03/16 10:48, Craig Ringer wrote: You btw can emulate asking for the specific LSN in SQL interface by first calling the pg_logical_slot_get_changes function with upto_lsn set to whatever lsn you expect to start at, but it's ugly. Ugh. I didn't realise

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 14 March 2016 at 17:16, Petr Jelinek wrote: > It will not change the fact that slot can go backwards Sure. I don't consider that a problem in general though. It's similar to the way we lose cached sequence chunks on crash - a performance optimisation with a user

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 14/03/16 08:08, Craig Ringer wrote: On 11 March 2016 at 20:15, Alvaro Herrera > wrote: Craig Ringer wrote: > Hi all > > I think I found a couple of logical decoding issues while writing tests for > failover

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 11 March 2016 at 20:15, Alvaro Herrera wrote: > Craig Ringer wrote: > > Hi all > > > > I think I found a couple of logical decoding issues while writing tests > for > > failover slots. > > > > Despite the docs' claim that a logical slot will replay data "exactly > >

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

Craig Ringer wrote: > Hi all > > I think I found a couple of logical decoding issues while writing tests for > failover slots. > > Despite the docs' claim that a logical slot will replay data "exactly > once", a slot's confirmed_lsn can go backwards and the SQL functions can > replay the same

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 11 March 2016 at 17:00, Simon Riggs wrote: > > >> Also, pg_logical_slot_get_changes and its _peek_ variant should have a >> param specifying the starting LSN to read and return. If this is lower than >> the restart_lsn but non-null it should ERROR; if it's greater than

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 11 March 2016 at 08:19, Craig Ringer wrote: > Hi all > > I think I found a couple of logical decoding issues while writing tests > for failover slots. > > Despite the docs' claim that a logical slot will replay data "exactly > once", a slot's confirmed_lsn can go