Re: [HACKERS] Logical decoding on standby

On 21 June 2017 at 17:30, sanyam jain wrote: > Hi, > After changing > sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; > to > sendTimeLineIsHistoric = state->currTLI != ThisTimeLineID; > > I was facing another issue. > On promotion of a cascaded server

Re: [HACKERS] Logical decoding on standby

On 21 June 2017 at 13:28, sanyam jain wrote: > Hi, > > In this patch in walsender.c sendTimeLineIsHistoric is set to true when > current and ThisTimeLineID are equal. > > sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; > > > Shouldn't sendTimeLineIsHistoric is

Re: [HACKERS] Logical decoding on standby

Hi, >After changing >sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; >to >sendTimeLineIsHistoric = state->currTLI != ThisTimeLineID; > >I was facing another issue. >On promotion of a cascaded server ThisTimeLineID in the standby server having >>logical slot becomes 0. >Then i added a

Re: [HACKERS] Logical decoding on standby

Hi, After changing sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; to sendTimeLineIsHistoric = state->currTLI != ThisTimeLineID; I was facing another issue. On promotion of a cascaded server ThisTimeLineID in the standby server having logical slot becomes 0. Then i added a function

Re: [HACKERS] Logical decoding on standby

resql.org> on behalf of Robert Haas <robertmh...@gmail.com> Sent: Wednesday, April 5, 2017 3:25:50 PM To: Andres Freund Cc: Craig Ringer; Simon Riggs; Thom Brown; Michael Paquier; Petr Jelinek; PostgreSQL Hackers Subject: Re: [HACKERS] Logical decoding on standby On Wed, Apr 5, 2017 at 10:32

Re: [HACKERS] Logical decoding on standby

On 5 April 2017 at 23:25, Robert Haas wrote: > On Wed, Apr 5, 2017 at 10:32 AM, Andres Freund wrote: >> On 2017-04-05 17:18:24 +0800, Craig Ringer wrote: >>> On 5 April 2017 at 04:19, Andres Freund wrote: >>> > On 2017-04-04

Re: [HACKERS] Logical decoding on standby

On Wed, Apr 5, 2017 at 10:32 AM, Andres Freund wrote: > On 2017-04-05 17:18:24 +0800, Craig Ringer wrote: >> On 5 April 2017 at 04:19, Andres Freund wrote: >> > On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: >> >> I'm much happier with this. I'm still

Re: [HACKERS] Logical decoding on standby

On 2017-04-05 17:18:24 +0800, Craig Ringer wrote: > On 5 April 2017 at 04:19, Andres Freund wrote: > > On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: > >> I'm much happier with this. I'm still fixing some issues in the tests > >> for 03 and tidying them up, but 03 should

Re: [HACKERS] Logical decoding on standby

On 5 April 2017 at 04:19, Andres Freund wrote: > On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: >> I'm much happier with this. I'm still fixing some issues in the tests >> for 03 and tidying them up, but 03 should allow 01 and 02 to be >> reviewed in their proper context

Re: [HACKERS] Logical decoding on standby

On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: > I'm much happier with this. I'm still fixing some issues in the tests > for 03 and tidying them up, but 03 should allow 01 and 02 to be > reviewed in their proper context now. To me this very clearly is too late for v10, and now should be moved

Re: [HACKERS] Logical decoding on standby

On 4 April 2017 at 22:32, Craig Ringer wrote: > Hi all > > Here's the final set of three patches on top of what's already committed. > > The first is catalog_xmin logging, which is unchanged from the prior post. > > The 2nd is support for conflict with recovery, with

Re: [HACKERS] Logical decoding on standby

Hi all Here's the final set of three patches on top of what's already committed. The first is catalog_xmin logging, which is unchanged from the prior post. The 2nd is support for conflict with recovery, with changes that should address Andres's concerns there. The 3rd actually enables decoding

Re: [HACKERS] Logical decoding on standby

On 3 April 2017 at 15:27, Craig Ringer wrote: > On 3 April 2017 at 13:46, Craig Ringer wrote: > >> OK, updated catalog_xmin logging patch attached. > > Ahem, that should be v5. ... and here's v6, which returns to the separate

Re: [HACKERS] Logical decoding on standby

On 3 April 2017 at 13:46, Craig Ringer wrote: > OK, updated catalog_xmin logging patch attached. Ahem, that should be v5. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From

Re: [HACKERS] Logical decoding on standby

On 31 March 2017 at 12:49, Craig Ringer wrote: > On 31 March 2017 at 01:16, Andres Freund wrote: >>> @@ -9633,6 +9643,12 @@ xlog_redo(XLogReaderState *record) >>> SetTransactionIdLimit(checkPoint.oldestXid, >>> checkPoint.oldestXidDB);

Re: [HACKERS] Logical decoding on standby

On 31 March 2017 at 12:49, Craig Ringer wrote: > On 31 March 2017 at 01:16, Andres Freund wrote: >> The comment and code don't quite square to me - it's far from obvious >> that LogStandbySnapshot does something like that. I'd even say it's a >> bad

Re: [HACKERS] Logical decoding on standby

On 31 March 2017 at 01:16, Andres Freund wrote: >> @@ -9633,6 +9643,12 @@ xlog_redo(XLogReaderState *record) >> SetTransactionIdLimit(checkPoint.oldestXid, >> checkPoint.oldestXidDB); >> >> /* >> + * There can be no concurrent writers

Re: [HACKERS] Logical decoding on standby

On 31 March 2017 at 01:16, Andres Freund wrote: > On 2017-03-29 08:01:34 +0800, Craig Ringer wrote: >> On 28 March 2017 at 23:22, Andres Freund wrote: >> >> >> --- a/doc/src/sgml/protocol.sgml >> >> +++ b/doc/src/sgml/protocol.sgml >> >> @@ -2034,6 +2034,8

Re: [HACKERS] Logical decoding on standby

On 2017-03-30 19:40:08 +0100, Simon Riggs wrote: > On 30 March 2017 at 18:16, Andres Freund wrote: > > >> /* > >> * Each page of XLOG file has a header like this: > >> */ > >> -#define XLOG_PAGE_MAGIC 0xD097 /* can be used as WAL version > >> indicator */ > >>

Re: [HACKERS] Logical decoding on standby

On 30 March 2017 at 18:16, Andres Freund wrote: >> /* >> * Each page of XLOG file has a header like this: >> */ >> -#define XLOG_PAGE_MAGIC 0xD097 /* can be used as WAL version >> indicator */ >> +#define XLOG_PAGE_MAGIC 0xD100 /* can be used as WAL version

Re: [HACKERS] Logical decoding on standby

On 2017-03-29 08:01:34 +0800, Craig Ringer wrote: > On 28 March 2017 at 23:22, Andres Freund wrote: > > >> --- a/doc/src/sgml/protocol.sgml > >> +++ b/doc/src/sgml/protocol.sgml > >> @@ -2034,6 +2034,8 @@ The commands accepted in walsender mode are: > >> > >>

Re: [HACKERS] Logical decoding on standby

> @@ -9633,6 +9643,12 @@ xlog_redo(XLogReaderState *record) > SetTransactionIdLimit(checkPoint.oldestXid, > checkPoint.oldestXidDB); > > /* > + * There can be no concurrent writers to oldestCatalogXmin > during > + * recovery, so no need to

Re: [HACKERS] Logical decoding on standby

On 30 March 2017 at 15:27, Andres Freund wrote: > On 2017-03-30 15:26:02 +0100, Simon Riggs wrote: >> On 30 March 2017 at 09:07, Craig Ringer wrote: >> >> > Attached. >> >> * Cleaned up in 3 places >> * Added code for faked up RunningTransactions in

Re: [HACKERS] Logical decoding on standby

On 2017-03-30 15:26:02 +0100, Simon Riggs wrote: > On 30 March 2017 at 09:07, Craig Ringer wrote: > > > Attached. > > * Cleaned up in 3 places > * Added code for faked up RunningTransactions in xlog.c > * Ensure catalog_xmin doesn't go backwards > > All else looks good.

Re: [HACKERS] Logical decoding on standby

On 30 March 2017 at 09:07, Craig Ringer wrote: > Attached. * Cleaned up in 3 places * Added code for faked up RunningTransactions in xlog.c * Ensure catalog_xmin doesn't go backwards All else looks good. Comments before commit? -- Simon Riggs

Re: [HACKERS] Logical decoding on standby

On 30 March 2017 at 11:34, Craig Ringer wrote: > On 29 March 2017 at 23:13, Simon Riggs wrote: >> On 29 March 2017 at 10:17, Craig Ringer wrote: >>> On 29 March 2017 at 16:44, Craig Ringer wrote:

Re: [HACKERS] Logical decoding on standby

On 29 March 2017 at 23:13, Simon Riggs wrote: > On 29 March 2017 at 10:17, Craig Ringer wrote: >> On 29 March 2017 at 16:44, Craig Ringer wrote: >> >>> * Split oldestCatalogXmin tracking into separate patch >> >>

Re: [HACKERS] Logical decoding on standby

On 29 March 2017 at 10:17, Craig Ringer wrote: > On 29 March 2017 at 16:44, Craig Ringer wrote: > >> * Split oldestCatalogXmin tracking into separate patch > > Regarding this, Simon raised concerns about xlog volume here. > > It's pretty negligible.

Re: [HACKERS] Logical decoding on standby

On 29 March 2017 at 16:44, Craig Ringer wrote: > * Split oldestCatalogXmin tracking into separate patch Regarding this, Simon raised concerns about xlog volume here. It's pretty negligible. We only write a new record when a vacuum runs after catalog_xmin advances on the

Re: [HACKERS] Logical decoding on standby

On 29 March 2017 at 08:11, Craig Ringer wrote: > On 29 March 2017 at 08:01, Craig Ringer wrote: > >> I just notice that I failed to remove the docs changes regarding >> dropping slots becoming db-specific, so I'll post a follow-up for that >> in a

Re: [HACKERS] Logical decoding on standby

On 29 March 2017 at 08:01, Craig Ringer wrote: > I just notice that I failed to remove the docs changes regarding > dropping slots becoming db-specific, so I'll post a follow-up for that > in a sec. Attached. -- Craig Ringer

Re: [HACKERS] Logical decoding on standby

On 28 March 2017 at 23:22, Andres Freund wrote: >> --- a/doc/src/sgml/protocol.sgml >> +++ b/doc/src/sgml/protocol.sgml >> @@ -2034,6 +2034,8 @@ The commands accepted in walsender mode are: >> >>Drops a replication slot, freeing any reserved server-side

Re: [HACKERS] Logical decoding on standby

Hi, On 2017-03-27 16:03:48 +0800, Craig Ringer wrote: > On 27 March 2017 at 14:08, Craig Ringer wrote: > > > So this patch makes ReplicationSlotAcquire check that the slot > > database matches the current database and refuse to acquire the slot > > if it does not. > >

Re: [HACKERS] Logical decoding on standby

On 27 March 2017 at 16:20, Simon Riggs wrote: > On 27 March 2017 at 09:03, Craig Ringer wrote: > >> I think this one's ready to go. > > Looks like something I could commit. Full review by me while offline > today, aiming to commit tomorrow

Re: [HACKERS] Logical decoding on standby

On 27 March 2017 at 09:03, Craig Ringer wrote: > I think this one's ready to go. Looks like something I could commit. Full review by me while offline today, aiming to commit tomorrow barring issues raised. -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] Logical decoding on standby

On 27 March 2017 at 14:08, Craig Ringer wrote: > So this patch makes ReplicationSlotAcquire check that the slot > database matches the current database and refuse to acquire the slot > if it does not. New patch attached that drops above requirement, so slots can still be

Re: [HACKERS] Logical decoding on standby

Hi Here's the next patch in the split-up series, drop db-specific (logical) replication slots on DROP DATABASE. Current behaviour is to ERROR if logical slots exist on the DB, whether in-use or not. With this patch we can DROP a database if it has logical slots so long as they are not active. I

Re: [HACKERS] Logical decoding on standby

On 20 March 2017 at 17:33, Andres Freund wrote: > Have you checked how high the overhead of XLogReadDetermineTimeline is? > A non-local function call, especially into a different translation-unit > (no partial inlining), for every single page might end up being > noticeable.

Re: [HACKERS] Logical decoding on standby

On 24 March 2017 at 06:23, Craig Ringer wrote: > On 23 March 2017 at 17:44, Craig Ringer wrote: > > Minor update to catalog_xmin walsender patch to fix failure to > parenthesize definition of PROCARRAY_PROC_FLAGS_MASK . > > This one's ready to go.

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 17:44, Craig Ringer wrote: Minor update to catalog_xmin walsender patch to fix failure to parenthesize definition of PROCARRAY_PROC_FLAGS_MASK . This one's ready to go. Working on drop slots on DB drop now. -- Craig Ringer

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 16:07, Craig Ringer wrote: > If preferred I can instead add > > proc.h: > > #define PROC_RESERVED 0x20 > > procarray.h: > > #define PROCARRAY_REPLICATION_SLOTS 0x20 > > and then test for (flags & PROCARRAY_REPLICATION_SLOTS) Attached done that way. --

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 00:13, Simon Riggs wrote: > On 22 March 2017 at 08:53, Craig Ringer wrote: > >> I'm splitting up the rest of the decoding on standby patch set with >> the goal of getting minimal functionality for creating and managing >>

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 12:41, Andres Freund wrote: > On 2017-03-23 12:14:02 +0800, Craig Ringer wrote: >> On 23 March 2017 at 09:39, Andres Freund wrote: >> > I still think decoding-on-standby is simply not the right approach as >> > the basic/first HA

Re: [HACKERS] Logical decoding on standby

On 2017-03-23 12:14:02 +0800, Craig Ringer wrote: > On 23 March 2017 at 09:39, Andres Freund wrote: > > I still think decoding-on-standby is simply not the right approach as > > the basic/first HA approach for logical rep. It's a nice later-on > > feature. But that's an

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 00:13, Simon Riggs wrote: > On 22 March 2017 at 08:53, Craig Ringer wrote: > >> I'm splitting up the rest of the decoding on standby patch set with >> the goal of getting minimal functionality for creating and managing >>

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 09:39, Andres Freund wrote: > We can't just assume that snapbuild is going to work correctly when it's > prerequisites - pinned xmin horizon - isn't working. Makes sense. >> What do _you_ see as the minimum acceptable way to achieve the ability >> for a

Re: [HACKERS] Logical decoding on standby

On 2017-03-23 09:14:07 +0800, Craig Ringer wrote: > On 23 March 2017 at 07:31, Andres Freund wrote: > > On 2017-03-23 06:55:53 +0800, Craig Ringer wrote: > > >> I was thinking that by disallowing snapshot use and output plugin > >> invocation we'd avoid the need to support

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 07:31, Andres Freund wrote: > On 2017-03-23 06:55:53 +0800, Craig Ringer wrote: >> I was thinking that by disallowing snapshot use and output plugin >> invocation we'd avoid the need to support cancellation on recovery >> conflicts, etc, simplifying things

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 00:17, Andres Freund wrote: > On 2017-03-22 15:59:42 +, Simon Riggs wrote: >> On 22 March 2017 at 13:06, Andres Freund wrote: >> >> >> The parts I think are important for Pg10 are: >> > >> >> * Ability to create logical slots on

Re: [HACKERS] Logical decoding on standby

On 2017-03-23 06:55:53 +0800, Craig Ringer wrote: > On 22 March 2017 at 21:06, Andres Freund wrote: > > Hi, > > > > On 2017-03-21 09:05:26 +0800, Craig Ringer wrote: > >> > 0002 should be doable as a whole this release, I have severe doubts that > >> > 0003 as a whole has a

Re: [HACKERS] Logical decoding on standby

On 22 March 2017 at 21:06, Andres Freund wrote: > Hi, > > On 2017-03-21 09:05:26 +0800, Craig Ringer wrote: >> > 0002 should be doable as a whole this release, I have severe doubts that >> > 0003 as a whole has a chance for 10 - the code is in quite a raw shape, >> > there's a

Re: [HACKERS] Logical decoding on standby

On 2017-03-22 15:59:42 +, Simon Riggs wrote: > On 22 March 2017 at 13:06, Andres Freund wrote: > > >> The parts I think are important for Pg10 are: > > > >> * Ability to create logical slots on replicas > > > > Doesn't this also imply recovery conflicts on DROP DATABASE?

Re: [HACKERS] Logical decoding on standby

On 22 March 2017 at 08:53, Craig Ringer wrote: > I'm splitting up the rest of the decoding on standby patch set with > the goal of getting minimal functionality for creating and managing > slots on standbys in, so we can maintain slots on standbys and use > them when the

Re: [HACKERS] Logical decoding on standby

On 22 March 2017 at 13:06, Andres Freund wrote: >> The parts I think are important for Pg10 are: > >> * Ability to create logical slots on replicas > > Doesn't this also imply recovery conflicts on DROP DATABASE? Not needed until the slot is in use, which is a later patch.

Re: [HACKERS] Logical decoding on standby

On 2017-03-22 14:58:29 +, Simon Riggs wrote: > On 22 March 2017 at 13:06, Andres Freund wrote: > > > But I have to admit, I've *severe* doubts about getting the whole > > infrastructure for slot creation on replica into 10. The work is far > > from ready, and we're mere

Re: [HACKERS] Logical decoding on standby

On 22 March 2017 at 13:06, Andres Freund wrote: > But I have to admit, I've *severe* doubts about getting the whole > infrastructure for slot creation on replica into 10. The work is far > from ready, and we're mere days away from freeze. If Craig has to guess what would be

Re: [HACKERS] Logical decoding on standby

Hi, On 2017-03-21 09:05:26 +0800, Craig Ringer wrote: > > 0002 should be doable as a whole this release, I have severe doubts that > > 0003 as a whole has a chance for 10 - the code is in quite a raw shape, > > there's a significant number of open ends. I'd suggest breaking of bits > > that are

Re: [HACKERS] Logical decoding on standby

On 22 March 2017 at 10:51, Craig Ringer wrote: > Hi all > > Updated timeline following patch attached. > > There's a change in read_local_xlog_page to ensure we maintain > ThisTimeLineID properly, otherwise it's just comment changes. OK, so we're looking OK with the TL

Re: [HACKERS] Logical decoding on standby

Hi all Updated timeline following patch attached. There's a change in read_local_xlog_page to ensure we maintain ThisTimeLineID properly, otherwise it's just comment changes. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] Logical decoding on standby

On 21 March 2017 at 09:05, Craig Ringer wrote: > Thanks, that's a helpful point. The commit in question is 978b2f65. I > didn't notice that it introduced XLogReader use in twophase.c, though > I should've realised given the discussion about fetching recent 2pc > info from

Re: [HACKERS] Logical decoding on standby

On 21 March 2017 at 02:21, Craig Ringer wrote: > On 20 March 2017 at 17:33, Andres Freund wrote: > >>> Subject: [PATCH 2/3] Follow timeline switches in logical decoding >> >> FWIW, the title doesn't really seem accurate to me. > > Yeah, it's not really

Re: [HACKERS] Logical decoding on standby

On 20 March 2017 at 17:33, Andres Freund wrote: >> Subject: [PATCH 2/3] Follow timeline switches in logical decoding > > FWIW, the title doesn't really seem accurate to me. Yeah, it's not really at the logical decoding layer at all. "Teach xlogreader to follow timeline

Re: [HACKERS] Logical decoding on standby

.On 20 March 2017 at 17:33, Andres Freund wrote: > Hi, > > Have you checked how high the overhead of XLogReadDetermineTimeline is? > A non-local function call, especially into a different translation-unit > (no partial inlining), for every single page might end up being >

Re: [HACKERS] Logical decoding on standby

On 19 March 2017 at 22:12, Petr Jelinek wrote: > I am slightly worried about impact of the readTimeLineHistory() call but > I think it should be called so little that it should not matter. Pretty much my thinking too. > That brings us to the big patch 0003. > > I

Re: [HACKERS] Logical decoding on standby

Hi, Have you checked how high the overhead of XLogReadDetermineTimeline is? A non-local function call, especially into a different translation-unit (no partial inlining), for every single page might end up being noticeable. That's fine in the cases it actually adds functionality, but for a

Re: [HACKERS] Logical decoding on standby

On 20 March 2017 at 14:57, Simon Riggs wrote: > 2.1 Why does call to ReplicationSlotAcquire() move earlier in > pg_logical_slot_get_changes_guts()? That appears to be an oversight from an earlier version where it looped over timelines in

Re: [HACKERS] Logical decoding on standby

On 19 March 2017 at 21:12, Craig Ringer wrote: > Rebased attached. Patch1 looks good to go. I'll correct a spelling mistake in the tap test when I commit that later today. Patch2 has a couple of points 2.1 Why does call to ReplicationSlotAcquire() move earlier in

Re: [HACKERS] Logical decoding on standby

Hi, I don't know how well I can review the 0001 (the TAP infra patch) but it looks okay to me. I don't really have any complaints about 0002 either. I like that it's more or less one self-contained function and there are no weird ifdefs anymore like in 9.6 version (btw your commit message talks

Re: [HACKERS] Logical decoding on standby

On 13 March 2017 at 10:56, Craig Ringer wrote: > On 7 March 2017 at 21:08, Simon Riggs wrote: > >> Patch 4 committed. Few others need rebase. > > Since this patch series Patch 1 fails since feature has already been applied. If other reason,

Re: [HACKERS] Logical decoding on standby

On 13 March 2017 at 10:56, Craig Ringer wrote: > On 7 March 2017 at 21:08, Simon Riggs wrote: > >> Patch 4 committed. Few others need rebase. > > Since this patch series and initial data copy for logical replication > both add a facility for

Re: [HACKERS] Logical decoding on standby

On 24 January 2017 at 06:37, Craig Ringer wrote: > Rebased series attached, on top of current master (which includes > logical replicaiton). > > I'm inclined to think I should split out a few of the changes from > 0005, roughly along the lines of the bullet points in its

Re: [HACKERS] Logical decoding on standby

On Tue, Jan 24, 2017 at 7:37 AM, Craig Ringer wrote: > Rebased series attached, on top of current master (which includes > logical replicaiton). > > I'm inclined to think I should split out a few of the changes from > 0005, roughly along the lines of the bullet points in

Re: [HACKERS] Logical decoding on standby

On 5 January 2017 at 01:21, Craig Ringer wrote: > On 5 January 2017 at 09:19, Craig Ringer wrote: > >> so here's a rebased series on top of master. No other changes. > > Now with actual patches. Patch 5 no longer applies: patching file

Re: [HACKERS] Logical decoding on standby

On Fri, Jan 6, 2017 at 1:07 PM, Craig Ringer wrote: > On 5 January 2017 at 13:12, Michael Paquier wrote: >> On Thu, Jan 5, 2017 at 10:21 AM, Craig Ringer wrote: >>> On 5 January 2017 at 09:19, Craig Ringer

Re: [HACKERS] Logical decoding on standby

On 5 January 2017 at 13:12, Michael Paquier wrote: > On Thu, Jan 5, 2017 at 10:21 AM, Craig Ringer wrote: >> On 5 January 2017 at 09:19, Craig Ringer wrote: >> >>> so here's a rebased series on top of master. No other

Re: [HACKERS] Logical decoding on standby

On Thu, Jan 5, 2017 at 10:21 AM, Craig Ringer wrote: > On 5 January 2017 at 09:19, Craig Ringer wrote: > >> so here's a rebased series on top of master. No other changes. > > Now with actual patches. Looking at the PostgresNode code in 0001... +=pod

Re: [HACKERS] Logical decoding on standby

On 4 January 2017 at 16:19, Craig Ringer wrote: > On 4 January 2017 at 12:15, Craig Ringer wrote: > >> That's particularly relevant to you Simon as you expressed a wish to >> commit the new streaming rep tests. Simon committed 1, 2, 3 and 5: *

Re: [HACKERS] Logical decoding on standby

On 4 January 2017 at 12:15, Craig Ringer wrote: > That's particularly relevant to you Simon as you expressed a wish to > commit the new streaming rep tests. Patches 0001 and 0005 in this series also posted as

Re: [HACKERS] Logical decoding on standby

On 4 January 2017 at 12:08, Craig Ringer wrote: > > 0001 incorporates the changes requested by Michael Paquier. Simon > expressed his intention to commit this after updates, in the separate > thread [...] ... > 0005 (new streaming rep tests) is updated for the changes in

Re: [HACKERS] Logical decoding on standby

On 12/22/2016 01:21 AM, Craig Ringer wrote: + my @fields = ('plugin', 'slot_type', 'datoid', 'database', 'active', 'active_pid', 'xmin', 'catalog_xmin', 'restart_lsn'); + my $result = $self->safe_psql('postgres', 'SELECT ' . join(', ', @fields) . " FROM pg_catalog.pg_replication_slots

Re: [HACKERS] Logical decoding on standby

On 22 December 2016 at 14:21, Craig Ringer wrote: changes-in-0001-v2.diff shows the changes to PostgresNode.pm per Michael's comments, and applies on top of 0001. I also attach a patch to add a new CREATE_REPLICATION_SLOT option per Petr's suggestion, so you can request a

Re: [HACKERS] Logical decoding on standby

On 22 December 2016 at 13:43, Michael Paquier wrote: > So, for 0001: > --- a/src/test/perl/PostgresNode.pm > +++ b/src/test/perl/PostgresNode.pm > @@ -93,6 +93,7 @@ use RecursiveCopy; > use Socket; > use Test::More; > use TestLib (); > +use pg_lsn qw(parse_lsn); >

Re: [HACKERS] Logical decoding on standby

On Tue, Dec 20, 2016 at 4:03 PM, Petr Jelinek wrote: > That's about approach, but since there are prerequisite patches in the > patchset that don't really depend on the approach I will comment about > them as well. > > 0001 and 0002 add testing infrastructure and

Re: [HACKERS] Logical decoding on standby

On Tue, Dec 20, 2016 at 10:06 PM, Craig Ringer wrote: > On 20 December 2016 at 15:03, Petr Jelinek > wrote: >>> The biggest change in this patch, and the main intrusive part, is that >>> procArray->replication_slot_catalog_xmin is no longer

Re: [HACKERS] Logical decoding on standby

On 21/12/16 04:06, Craig Ringer wrote: > On 20 December 2016 at 15:03, Petr Jelinek > wrote: > >>> The biggest change in this patch, and the main intrusive part, is that >>> procArray->replication_slot_catalog_xmin is no longer directly used by >>> vacuum. Instead,

Re: [HACKERS] Logical decoding on standby

On 21/12/16 04:31, Craig Ringer wrote: > On 20 December 2016 at 15:03, Petr Jelinek > wrote: > >> But in 0003 I don't understand following code: >>> + if (endpos != InvalidXLogRecPtr && !do_start_slot) >>> + { >>> + fprintf(stderr, >>> +

Re: [HACKERS] Logical decoding on standby

On 20 December 2016 at 15:03, Petr Jelinek wrote: > But in 0003 I don't understand following code: >> + if (endpos != InvalidXLogRecPtr && !do_start_slot) >> + { >> + fprintf(stderr, >> + _("%s: cannot use

Re: [HACKERS] Logical decoding on standby

On 20 December 2016 at 15:03, Petr Jelinek wrote: >> The biggest change in this patch, and the main intrusive part, is that >> procArray->replication_slot_catalog_xmin is no longer directly used by >> vacuum. Instead, a new ShmemVariableCache->oldestCatalogXmin

Re: [HACKERS] Logical decoding on standby

On 07/12/16 07:05, Craig Ringer wrote: > On 21 November 2016 at 16:17, Craig Ringer wrote: >> Hi all >> >> I've prepared a working initial, somewhat raw implementation for >> logical decoding on physical standbys. > > Hi all > > I've attached a significantly revised

Re: [HACKERS] Logical decoding on standby

>> --- a/contrib/pgstattuple/pgstatapprox.c >> +++ b/contrib/pgstattuple/pgstatapprox.c >> @@ -70,7 +70,7 @@ statapprox_heap(Relation rel, output_type *stat) >> TransactionId OldestXmin; >> uint64 misc_count = 0; >> >> - OldestXmin = GetOldestXmin(rel, true); >> +

Re: [HACKERS] Logical decoding on standby

Hi, I did look at the code a bit. The first 6 patches seem reasonable. I don't understand why some patches are separate tbh (like 7-10, or 11). About the 0009: > diff --git a/contrib/pg_visibility/pg_visibility.c > b/contrib/pg_visibility/pg_visibility.c > index 9985e3e..4fa3ad4 100644 > ---

Re: [HACKERS] Logical decoding on standby

On 26 Nov. 2016 23:40, "Robert Haas" wrote: > > On Wed, Nov 23, 2016 at 8:37 AM, Craig Ringer wrote: > >>> The last checkpoint's oldestXid, and ShmemVariableCache's oldestXid, > >>> are already held down by ProcArray's catalog_xmin. But that doesn't

Re: [HACKERS] Logical decoding on standby

On Wed, Nov 23, 2016 at 8:37 AM, Craig Ringer wrote: >>> The last checkpoint's oldestXid, and ShmemVariableCache's oldestXid, >>> are already held down by ProcArray's catalog_xmin. But that doesn't >>> mean we haven't removed newer tuples from specific relations and >>>

Re: [HACKERS] Logical decoding on standby

On 23 November 2016 at 03:55, Robert Haas wrote: > On Tue, Nov 22, 2016 at 1:49 AM, Craig Ringer wrote: >> On 22 November 2016 at 10:20, Craig Ringer wrote: >>> I'm currently looking at making detection of replay conflict with

Re: [HACKERS] Logical decoding on standby

On Tue, Nov 22, 2016 at 1:49 AM, Craig Ringer wrote: > On 22 November 2016 at 10:20, Craig Ringer wrote: >> I'm currently looking at making detection of replay conflict with a >> slot work by separating the current catalog_xmin into two effective >>

Re: [HACKERS] Logical decoding on standby

On 22 November 2016 at 10:20, Craig Ringer wrote: > I'm currently looking at making detection of replay conflict with a > slot work by separating the current catalog_xmin into two effective > parts - the catalog_xmin currently needed by any known slots >

Re: [HACKERS] Logical decoding on standby

Hi, On 2016-11-21 16:17:58 +0800, Craig Ringer wrote: > I've prepared a working initial, somewhat raw implementation for > logical decoding on physical standbys. Please attach. Otherwise in a year or two it'll be impossible to look this up. Andres -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Logical decoding on standby

On 19 January 2016 at 23:30, Дмитрий Сарафанников wrote: When you plan to add logical decoding on standby? > > it is useful to have separate standby server for logical replication that > will not break the master if you make a mistake in plugin. > Indeed. It might PANIC

Re: [HACKERS] Logical decoding on standby

On 2016-01-20 15:11:06 +0800, Craig Ringer wrote: > Unfortunately it's not particularly simple and nobody seems to have time to > implement it. FWIW, I don't think it's *that* hard. > As Álvaro pointed out, sometimes you have to do the work if > you want the change to happen. Or find someone

Re: [HACKERS] Logical decoding on standby

Hi Dimitriy, Дмитрий Сарафанников wrote: > /* > * TODO: We got to change that someday soon... [ more code ] > if (RecoveryInProgress()) > ereport(ERROR, > (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), > errmsg("logical decoding cannot be used while in recovery"))); > When you plan to add logical

  1   2   >