Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 September 2016 at 16:33, Petr Jelinek wrote: >> The better alternative is to add a variant on >> pg_logical_slot_get_changes(...) etc that accepts a start LSN. But >> it's not convenient or easy for SQL callers to extract the last commit >> LSN received from the last call to pass it to the n

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 September 2016 at 03:41, Craig Ringer wrote: > On 2 September 2016 at 17:49, Craig Ringer wrote: > >> So the main change becomes the one-liner in my prior mail. > > Per feedback from Simon, updated with a new test in src/test/recovery . > > If you revert the change to > src/backend/replicati

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 05/09/16 04:41, Craig Ringer wrote: On 2 September 2016 at 17:49, Craig Ringer wrote: So the main change becomes the one-liner in my prior mail. Per feedback from Simon, updated with a new test in src/test/recovery . If you revert the change to src/backend/replication/logical/logicalfunc

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 September 2016 at 10:41, Craig Ringer wrote: > On 2 September 2016 at 17:49, Craig Ringer wrote: > >> So the main change becomes the one-liner in my prior mail. > > Per feedback from Simon, updated with a new test in src/test/recovery . ... attached this time. -- Craig Ringer

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 2 September 2016 at 17:49, Craig Ringer wrote: > So the main change becomes the one-liner in my prior mail. Per feedback from Simon, updated with a new test in src/test/recovery . If you revert the change to src/backend/replication/logical/logicalfuncs.c then the test will start failing. I'

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 2016-09-02 10:50:17 +0200, Petr Jelinek wrote: > Okay that sounds reasonable, the SQL interface is already somewhat different > than walsender as it does not really "stream" so makes sense to improve the > behavior there. As a side note, I would really like to have cursor-like SQL > interface wh

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 2 September 2016 at 16:50, Petr Jelinek wrote: >> So here's a simpler patch that just dirties the slot when it's >> replayed something from it on the SQL interface [...] > > Okay that sounds reasonable, the SQL interface is already somewhat different > than walsender as it does not really "str

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 02/09/16 09:58, Craig Ringer wrote: On 1 September 2016 at 21:19, Simon Riggs wrote: I agree the doc patch should go in, though I suggest reword it slightly, like attached patch. Thanks. The rewording looks good to me and I think that Doc-correction-each-change-once.v2.patch is ready for

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 1 September 2016 at 21:19, Simon Riggs wrote: > I agree the doc patch should go in, though I suggest reword it > slightly, like attached patch. Thanks. The rewording looks good to me and I think that Doc-correction-each-change-once.v2.patch is ready for commit. Meanwhile, thinking about the

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 20 August 2016 at 15:04, Petr Jelinek wrote: > On 20/08/16 16:01, Craig Ringer wrote: >> >> On 5 June 2016 at 09:54, David G. Johnston > > wrote: >> >> On Thursday, March 17, 2016, Craig Ringer > > wrote: >> >> The

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 20/08/16 16:01, Craig Ringer wrote: On 5 June 2016 at 09:54, David G. Johnston mailto:david.g.johns...@gmail.com>> wrote: On Thursday, March 17, 2016, Craig Ringer mailto:cr...@2ndquadrant.com>> wrote: The first patch was incorrectly created on top of failover slots not H

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 5 June 2016 at 09:54, David G. Johnston wrote: > On Thursday, March 17, 2016, Craig Ringer wrote: > >> The first patch was incorrectly created on top of failover slots not >> HEAD. Attached patch applies on HEAD. >> > > Lots of logical decoding work ongoing but this one shows as active in the

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On Thursday, March 17, 2016, Craig Ringer wrote: > The first patch was incorrectly created on top of failover slots not HEAD. > Attached patch applies on HEAD. > Lots of logical decoding work ongoing but this one shows as active in the September cf and the comments by Craig indicate potential re

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

The first patch was incorrectly created on top of failover slots not HEAD. Attached patch applies on HEAD. From 87d839f8a2e78abb17fa985502fd5b66f0872b57 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Wed, 16 Mar 2016 15:45:16 +0800 Subject: [PATCH 1/2] Correct incorrect claim that slots output

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

Attached is a patch to mark a logical replication slot as dirty if its confirmed lsn is changed. Aesthetically I'm not sure if it's better to do it per this patch and call ReplicationSlotMarkDirty after we release the spinlock, add a new ReplicationSlotMarkDirtyLocked() that skips the spinlock acq

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 14/03/16 10:48, Craig Ringer wrote: You btw can emulate asking for the specific LSN in SQL interface by first calling the pg_logical_slot_get_changes function with upto_lsn set to whatever lsn you expect to start at, but it's ugly. Ugh. I didn't realise pg_logical_slot_get_chan

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 14 March 2016 at 17:16, Petr Jelinek wrote: > It will not change the fact that slot can go backwards Sure. I don't consider that a problem in general though. It's similar to the way we lose cached sequence chunks on crash - a performance optimisation with a user visible effect. It needs to

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 14/03/16 08:08, Craig Ringer wrote: On 11 March 2016 at 20:15, Alvaro Herrera mailto:alvhe...@2ndquadrant.com>> wrote: Craig Ringer wrote: > Hi all > > I think I found a couple of logical decoding issues while writing tests for > failover slots. > > Despite the do

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 11 March 2016 at 20:15, Alvaro Herrera wrote: > Craig Ringer wrote: > > Hi all > > > > I think I found a couple of logical decoding issues while writing tests > for > > failover slots. > > > > Despite the docs' claim that a logical slot will replay data "exactly > > once", a slot's confirmed_l

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

Craig Ringer wrote: > Hi all > > I think I found a couple of logical decoding issues while writing tests for > failover slots. > > Despite the docs' claim that a logical slot will replay data "exactly > once", a slot's confirmed_lsn can go backwards and the SQL functions can > replay the same dat

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 11 March 2016 at 17:00, Simon Riggs wrote: > > >> Also, pg_logical_slot_get_changes and its _peek_ variant should have a >> param specifying the starting LSN to read and return. If this is lower than >> the restart_lsn but non-null it should ERROR; if it's greater than or equal >> it should u

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

On 11 March 2016 at 08:19, Craig Ringer wrote: > Hi all > > I think I found a couple of logical decoding issues while writing tests > for failover slots. > > Despite the docs' claim that a logical slot will replay data "exactly > once", a slot's confirmed_lsn can go backwards and the SQL function

[HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

Hi all I think I found a couple of logical decoding issues while writing tests for failover slots. Despite the docs' claim that a logical slot will replay data "exactly once", a slot's confirmed_lsn can go backwards and the SQL functions can replay the same data more than once.We don't mark a slo