Re: [HACKERS] logical decoding of two-phase transactions

On 28 October 2017 at 03:53, Sokolov Yura wrote: > On 2017-10-26 22:01, Sokolov Yura wrote: > Small improvement compared to v7: > - twophase_gid is written with alignment padding in the XactLogCommitRecord > and XactLogAbortRecord. I think Nikhils has done some

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-10-26 22:01, Sokolov Yura wrote: On 2017-09-27 14:46, Stas Kelvich wrote: On 7 Sep 2017, at 18:58, Nikhil Sontakke wrote: Hi, FYI all, wanted to mention that I am working on an updated version of the latest patch that I plan to submit to a later CF. Cool!

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-09-27 14:46, Stas Kelvich wrote: On 7 Sep 2017, at 18:58, Nikhil Sontakke wrote: Hi, FYI all, wanted to mention that I am working on an updated version of the latest patch that I plan to submit to a later CF. Cool! So what kind of architecture do you have

Re: [HACKERS] logical decoding of two-phase transactions

> On 7 Sep 2017, at 18:58, Nikhil Sontakke wrote: > > Hi, > > FYI all, wanted to mention that I am working on an updated version of > the latest patch that I plan to submit to a later CF. > Cool! So what kind of architecture do you have in mind? Same way as is it

Re: [HACKERS] logical decoding of two-phase transactions

Hi, FYI all, wanted to mention that I am working on an updated version of the latest patch that I plan to submit to a later CF. Regards, Nikhils On 14 May 2017 at 04:02, Dmitry Dolgov <9erthali...@gmail.com> wrote: > On 13 May 2017 at 22:22, Tom Lane wrote: >> >> Apparently

Re: [HACKERS] logical decoding of two-phase transactions

On 13 May 2017 at 22:22, Tom Lane wrote: > > Apparently you are not testing against current HEAD. That's been there > since d10c626de (a whole two days now ;-)) Indeed, I was working on a more than two-day old antiquity. Unfortunately, it's even more complicated to apply

Re: [HACKERS] logical decoding of two-phase transactions

Dmitry Dolgov <9erthali...@gmail.com> writes: > Just a note about this patch. Of course time flies by and it needs rebase, > but also there are few failing tests right now: > ERROR: function pg_wal_lsn_diff(pg_lsn, unknown) does not exist Apparently you are not testing against current HEAD.

Re: [HACKERS] logical decoding of two-phase transactions

Hi > On 4 April 2017 at 19:13, Masahiko Sawada wrote: > > Other than that issue current patch still could not pass 'make check' > test of contrib/test_decoding. Just a note about this patch. Of course time flies by and it needs rebase, but also there are few failing tests

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-04-04 13:06:13 +0300, Stas Kelvich wrote: > That is just argument against Andres concern that prepared transaction > is able to deadlock with decoding process — at least no such cases in > regression tests. There's few longer / adverse xacts, that doesn't say much. > And that concern is

Re: [HACKERS] logical decoding of two-phase transactions

On Tue, Apr 4, 2017 at 7:06 PM, Stas Kelvich wrote: > >> On 4 Apr 2017, at 04:23, Masahiko Sawada wrote: >> >> >> I reviewed this patch but when I tried to build contrib/test_decoding >> I got the following error. >> > > Thanks! > > Yes, seems

Re: [HACKERS] logical decoding of two-phase transactions

> On 4 Apr 2017, at 04:23, Masahiko Sawada wrote: > > > I reviewed this patch but when I tried to build contrib/test_decoding > I got the following error. > Thanks! Yes, seems that 18ce3a4a changed ProcessUtility_hook signature. Updated. > There are still some

Re: [HACKERS] logical decoding of two-phase transactions

On Thu, Mar 30, 2017 at 12:55 AM, Stas Kelvich wrote: > >> On 28 Mar 2017, at 18:08, Andres Freund wrote: >> >> On 2017-03-28 15:55:15 +0100, Simon Riggs wrote: >>> >>> >>> That assertion is obviously false... the plugin can resolve this in >>>

Re: [HACKERS] logical decoding of two-phase transactions

> On 28 Mar 2017, at 18:08, Andres Freund wrote: > > On 2017-03-28 15:55:15 +0100, Simon Riggs wrote: >> >> >> That assertion is obviously false... the plugin can resolve this in >> various ways, if we allow it. > > Handling it by breaking replication isn't handling it

Re: [HACKERS] logical decoding of two-phase transactions

On 28 Mar. 2017 23:08, "Andres Freund" wrote: > > > >> I don't think its for us to say what the plugin is allowed to do. We > > >> decided on a plugin architecture, so we have to trust that the plugin > > >> author resolves the issues. We can document them so those choices

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-03-28 15:55:15 +0100, Simon Riggs wrote: > On 28 March 2017 at 15:38, Andres Freund wrote: > > On 2017-03-28 15:32:49 +0100, Simon Riggs wrote: > >> On 28 March 2017 at 03:53, Craig Ringer wrote: > >> > On 28 March 2017 at 09:25, Andres Freund

Re: [HACKERS] logical decoding of two-phase transactions

On 28 March 2017 at 15:38, Andres Freund wrote: > On 2017-03-28 15:32:49 +0100, Simon Riggs wrote: >> On 28 March 2017 at 03:53, Craig Ringer wrote: >> > On 28 March 2017 at 09:25, Andres Freund wrote: >> > >> >> If you actually

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-03-28 15:32:49 +0100, Simon Riggs wrote: > On 28 March 2017 at 03:53, Craig Ringer wrote: > > On 28 March 2017 at 09:25, Andres Freund wrote: > > > >> If you actually need separate decoding of 2PC, then you want to wait for > >> the PREPARE to

Re: [HACKERS] logical decoding of two-phase transactions

On 28 March 2017 at 03:53, Craig Ringer wrote: > On 28 March 2017 at 09:25, Andres Freund wrote: > >> If you actually need separate decoding of 2PC, then you want to wait for >> the PREPARE to be replicated. If that replication has to wait for the >>

Re: [HACKERS] logical decoding of two-phase transactions

On 28 March 2017 at 10:53, Craig Ringer wrote: > However, even once I add an option to force decoding of 2pc xacts with > catalog changes to test_decoding, I cannot reproduce the expected > locking issues so far. See tests in attached updated version, in >

Re: [HACKERS] logical decoding of two-phase transactions

On 28 March 2017 at 09:25, Andres Freund wrote: > If you actually need separate decoding of 2PC, then you want to wait for > the PREPARE to be replicated. If that replication has to wait for the > to-be-replicated prepared transaction to commit prepared, and commit > prepare

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-03-28 03:30:28 +0100, Simon Riggs wrote: > On 28 March 2017 at 02:25, Andres Freund wrote: > > On 2017-03-28 04:12:41 +0300, Stas Kelvich wrote: > >> > >> > On 28 Mar 2017, at 00:25, Andres Freund wrote: > >> > > >> > Hi, > >> > > >> > On

Re: [HACKERS] logical decoding of two-phase transactions

On 28 March 2017 at 02:25, Andres Freund wrote: > On 2017-03-28 04:12:41 +0300, Stas Kelvich wrote: >> >> > On 28 Mar 2017, at 00:25, Andres Freund wrote: >> > >> > Hi, >> > >> > On 2017-03-28 00:19:29 +0300, Stas Kelvich wrote: >> >> Ok, here it is. >> >

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-03-28 04:12:41 +0300, Stas Kelvich wrote: > > > On 28 Mar 2017, at 00:25, Andres Freund wrote: > > > > Hi, > > > > On 2017-03-28 00:19:29 +0300, Stas Kelvich wrote: > >> Ok, here it is. > > > > On a very quick skim, this doesn't seem to solve the issues around > >

Re: [HACKERS] logical decoding of two-phase transactions

> On 28 Mar 2017, at 00:25, Andres Freund wrote: > > Hi, > > On 2017-03-28 00:19:29 +0300, Stas Kelvich wrote: >> Ok, here it is. > > On a very quick skim, this doesn't seem to solve the issues around > deadlocks of prepared transactions vs. catalog tables. What if the >

Re: [HACKERS] logical decoding of two-phase transactions

On 28 March 2017 at 08:50, Stas Kelvich wrote: > >> On 28 Mar 2017, at 00:19, Stas Kelvich wrote: >> >> * It is actually doesn’t pass one of mine regression tests. I’ve added >> expected output >> as it should be. I’ll try to send follow up

Re: [HACKERS] logical decoding of two-phase transactions

On 28 March 2017 at 05:25, Andres Freund wrote: > On a very quick skim, this doesn't seem to solve the issues around > deadlocks of prepared transactions vs. catalog tables. What if the > prepared transaction contains something like LOCK pg_class; (there's a > lot more

Re: [HACKERS] logical decoding of two-phase transactions

> On 28 Mar 2017, at 00:19, Stas Kelvich wrote: > > * It is actually doesn’t pass one of mine regression tests. I’ve added > expected output > as it should be. I’ll try to send follow up message with fix, but right now > sending it > as is, as you asked. > >

Re: [HACKERS] logical decoding of two-phase transactions

Hi, On 2017-03-28 00:19:29 +0300, Stas Kelvich wrote: > Ok, here it is. On a very quick skim, this doesn't seem to solve the issues around deadlocks of prepared transactions vs. catalog tables. What if the prepared transaction contains something like LOCK pg_class; (there's a lot more realistic

Re: [HACKERS] logical decoding of two-phase transactions

> On 27 Mar 2017, at 16:29, Craig Ringer wrote: > > On 27 March 2017 at 17:53, Stas Kelvich wrote: > >> I’m heavily underestimated amount of changes there, but almost finished >> and will send updated patch in several hours. > > Oh, brilliant!

Re: [HACKERS] logical decoding of two-phase transactions

On 27 March 2017 at 17:53, Stas Kelvich wrote: > I’m heavily underestimated amount of changes there, but almost finished > and will send updated patch in several hours. Oh, brilliant! Please post whatever you have before you knock off for the day anyway, even if it's

Re: [HACKERS] logical decoding of two-phase transactions

> On 27 Mar 2017, at 12:26, Craig Ringer wrote: > > On 27 March 2017 at 09:31, Craig Ringer wrote: > >> We're in the last week of the CF. If you have a patch that's nearly >> ready or getting there, now would be a good time to post it for help >>

Re: [HACKERS] logical decoding of two-phase transactions

On 27 March 2017 at 09:31, Craig Ringer wrote: > We're in the last week of the CF. If you have a patch that's nearly > ready or getting there, now would be a good time to post it for help > and input from others. > > I would really like to get this in, but we're running

Re: [HACKERS] logical decoding of two-phase transactions

On 20 March 2017 at 21:47, Stas Kelvich wrote: > >> On 20 Mar 2017, at 16:39, Craig Ringer wrote: >> >> On 20 March 2017 at 20:57, Stas Kelvich wrote: >>> On 20 Mar 2017, at 15:17, Craig Ringer

Re: [HACKERS] logical decoding of two-phase transactions

> On 20 Mar 2017, at 16:39, Craig Ringer wrote: > > On 20 March 2017 at 20:57, Stas Kelvich wrote: >> >>> On 20 Mar 2017, at 15:17, Craig Ringer wrote: >>> I thought about having special field (or reusing one of

Re: [HACKERS] logical decoding of two-phase transactions

On 20 March 2017 at 20:57, Stas Kelvich wrote: > >> On 20 Mar 2017, at 15:17, Craig Ringer wrote: >> >>> I thought about having special field (or reusing one of the existing fields) >>> in snapshot struct to force filtering xmax > snap->xmax or

Re: [HACKERS] logical decoding of two-phase transactions

> On 20 Mar 2017, at 15:17, Craig Ringer wrote: > >> I thought about having special field (or reusing one of the existing fields) >> in snapshot struct to force filtering xmax > snap->xmax or xmin = snap->xmin >> as Petr suggested. Then this logic can reside in

Re: [HACKERS] logical decoding of two-phase transactions

On 17 March 2017 at 23:59, Robert Haas wrote: > On Thu, Mar 16, 2017 at 10:34 PM, Craig Ringer wrote: >> On 17 March 2017 at 08:10, Stas Kelvich wrote: >>> While working on this i’ve spotted quite a nasty corner case with

Re: [HACKERS] logical decoding of two-phase transactions

> I thought about having special field (or reusing one of the existing fields) > in snapshot struct to force filtering xmax > snap->xmax or xmin = snap->xmin > as Petr suggested. Then this logic can reside in ReorderBufferCommit(). > However this is not solving problem with catcache, so I'm

Re: [HACKERS] logical decoding of two-phase transactions

> On 20 Mar 2017, at 11:32, Craig Ringer wrote: > > On 19 March 2017 at 21:26, Petr Jelinek wrote: > >> I think only genam would need changes to do two-phase scan for this as >> the catalog scans should ultimately go there. It's going to

Re: [HACKERS] logical decoding of two-phase transactions

On 20/03/17 09:32, Craig Ringer wrote: > On 19 March 2017 at 21:26, Petr Jelinek wrote: > >> I think only genam would need changes to do two-phase scan for this as >> the catalog scans should ultimately go there. It's going to slow down >> things but we could limit

Re: [HACKERS] logical decoding of two-phase transactions

On 19 March 2017 at 21:26, Petr Jelinek wrote: > I think only genam would need changes to do two-phase scan for this as > the catalog scans should ultimately go there. It's going to slow down > things but we could limit the impact by doing the two-phase scan only >

Re: [HACKERS] logical decoding of two-phase transactions

On 17 March 2017 at 23:59, Robert Haas wrote: > But that lock could need to be held for an unbounded period of time - > as long as decoding takes to complete - which seems pretty > undesirable. Yeah. We could use a recovery-conflict like mechanism to signal the decoding

Re: [HACKERS] logical decoding of two-phase transactions

On 17/03/17 03:34, Craig Ringer wrote: > On 17 March 2017 at 08:10, Stas Kelvich wrote: > >> While working on this i’ve spotted quite a nasty corner case with aborted >> prepared >> transaction. I have some not that great ideas how to fix it, but maybe i >> blurred my

Re: [HACKERS] logical decoding of two-phase transactions

On Thu, Mar 16, 2017 at 10:34 PM, Craig Ringer wrote: > On 17 March 2017 at 08:10, Stas Kelvich wrote: >> While working on this i’ve spotted quite a nasty corner case with aborted >> prepared >> transaction. I have some not that great ideas how

Re: [HACKERS] logical decoding of two-phase transactions

On 16 March 2017 at 19:52, Stas Kelvich wrote: > > I’m working right now on issue with building snapshots for decoding prepared > tx. > I hope I'll send updated patch later today. Great. What approach are you taking? It looks like the snapshot builder actually does

Re: [HACKERS] logical decoding of two-phase transactions

On 17 March 2017 at 08:10, Stas Kelvich wrote: > While working on this i’ve spotted quite a nasty corner case with aborted > prepared > transaction. I have some not that great ideas how to fix it, but maybe i > blurred my > view and missed something. So want to ask

Re: [HACKERS] logical decoding of two-phase transactions

>> On 2 Mar 2017, at 11:00, Craig Ringer wrote: >> >> BTW, I've been reviewing the patch in more detail. Other than a bunch >> of copy-and-paste that I'm cleaning up, the main issue I've found is >> that in DecodePrepare, you call: >> >>

Re: [HACKERS] logical decoding of two-phase transactions

> On 16 Mar 2017, at 14:44, Craig Ringer wrote: > > I'm going to try to pick this patch up and amend its interface per our > discussion earlier, see if I can get it committable. I’m working right now on issue with building snapshots for decoding prepared tx. I hope I'll

Re: [HACKERS] logical decoding of two-phase transactions

On 15 March 2017 at 15:42, Petr Jelinek wrote: > Thinking about this some more. Why can't we use the same mechanism > standby uses, ie, use xid to identify the 2PC? It pushes work onto the downstream, which has to keep an mapping in a crash-safe,

Re: [HACKERS] logical decoding of two-phase transactions

On 02/03/17 17:34, Petr Jelinek wrote: > On 02/03/17 13:23, Craig Ringer wrote: >> On 2 March 2017 at 16:20, Stas Kelvich wrote: >>> On 2 Mar 2017, at 11:00, Craig Ringer wrote: We already have it, because we just decoded the

Re: [HACKERS] logical decoding of two-phase transactions

On 3/2/17 11:34 AM, Petr Jelinek wrote: > On 02/03/17 13:23, Craig Ringer wrote: >> >> I particularly dislike calling a commit callback for an abort. So I'd >> like to look further into the interface side of things. I'm inclined >> to suggest adding new callbacks for 2pc prepare, commit and

Re: [HACKERS] logical decoding of two-phase transactions

On 02/03/17 13:23, Craig Ringer wrote: > On 2 March 2017 at 16:20, Stas Kelvich wrote: >> >>> On 2 Mar 2017, at 11:00, Craig Ringer wrote: >>> >>> We already have it, because we just decoded the PREPARE TRANSACTION. >>> I'm preparing a patch

Re: [HACKERS] logical decoding of two-phase transactions

On 2 March 2017 at 16:20, Stas Kelvich wrote: > >> On 2 Mar 2017, at 11:00, Craig Ringer wrote: >> >> We already have it, because we just decoded the PREPARE TRANSACTION. >> I'm preparing a patch revision to demonstrate this. > > Yes, we already

Re: [HACKERS] logical decoding of two-phase transactions

> On 2 Mar 2017, at 11:00, Craig Ringer wrote: > > We already have it, because we just decoded the PREPARE TRANSACTION. > I'm preparing a patch revision to demonstrate this. Yes, we already have it, but if server reboots between commit prepared (all prepared state is

Re: [HACKERS] logical decoding of two-phase transactions

On 2 March 2017 at 16:00, Craig Ringer wrote: > What about if we ROLLBACK PREPARED after > we made the snapshot visible? Yeah, I'm pretty sure that's going to be a problem actually. You're telling the snapshot builder that an xact committed at PREPARE TRANSACTION time.

Re: [HACKERS] logical decoding of two-phase transactions

On 2 March 2017 at 15:27, Stas Kelvich wrote: > >> On 2 Mar 2017, at 01:20, Petr Jelinek wrote: >> >> The info gets removed on COMMIT PREPARED but at that point >> there is no real difference between replicating it as 2pc or 1pc since >>

Re: [HACKERS] logical decoding of two-phase transactions

> On 2 Mar 2017, at 01:20, Petr Jelinek wrote: > > The info gets removed on COMMIT PREPARED but at that point > there is no real difference between replicating it as 2pc or 1pc since > the 2pc behavior is for all intents and purposes lost at that point. > If we

Re: [HACKERS] logical decoding of two-phase transactions

On 2 March 2017 at 06:20, Petr Jelinek wrote: > If I understand you correctly you are saying that if PREPARE is being > decoded, we can load the GID from the 2pc info in memory about the > specific 2pc. The info gets removed on COMMIT PREPARED but at that point >

Re: [HACKERS] logical decoding of two-phase transactions

On 01/03/17 10:24, Craig Ringer wrote: > On 9 February 2017 at 21:23, Stas Kelvich wrote: > >>> On 2 Feb 2017, at 00:35, Craig Ringer wrote: >>> >>> Stas was concerned about what happens in logical decoding if we crash >>> between PREPSRE

Re: [HACKERS] logical decoding of two-phase transactions

On 9 February 2017 at 21:23, Stas Kelvich wrote: >> On 2 Feb 2017, at 00:35, Craig Ringer wrote: >> >> Stas was concerned about what happens in logical decoding if we crash >> between PREPSRE TRANSACTION and COMMIT PREPARED. But we'll always go

Re: [HACKERS] logical decoding of two-phase transactions

> On 31 Jan 2017, at 12:22, Craig Ringer wrote: > > Personally I don't think lack of access to the GID justifies blocking 2PC > logical decoding. It can be added separately. But it'd be nice to have > especially if it's cheap. Agreed. > On 2 Feb 2017, at 00:35, Craig

Re: [HACKERS] logical decoding of two-phase transactions

On 02/04/2017 03:08 AM, Andres Freund wrote: > On 2017-02-03 18:47:23 -0500, Robert Haas wrote: >> On Fri, Feb 3, 2017 at 6:00 PM, Andres Freund wrote: >>> I still haven't seen a credible model for being able to apply a stream >>> of interleaved transactions that can roll back

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-02-03 19:09:43 -0500, Robert Haas wrote: > On Fri, Feb 3, 2017 at 7:08 PM, Andres Freund wrote: > > On 2017-02-03 18:47:23 -0500, Robert Haas wrote: > >> On Fri, Feb 3, 2017 at 6:00 PM, Andres Freund wrote: > >> > I still haven't seen a credible

Re: [HACKERS] logical decoding of two-phase transactions

On Fri, Feb 3, 2017 at 7:08 PM, Andres Freund wrote: > On 2017-02-03 18:47:23 -0500, Robert Haas wrote: >> On Fri, Feb 3, 2017 at 6:00 PM, Andres Freund wrote: >> > I still haven't seen a credible model for being able to apply a stream >> > of interleaved

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-02-03 18:47:23 -0500, Robert Haas wrote: > On Fri, Feb 3, 2017 at 6:00 PM, Andres Freund wrote: > > I still haven't seen a credible model for being able to apply a stream > > of interleaved transactions that can roll back individually; I think we > > really need the

Re: [HACKERS] logical decoding of two-phase transactions

On Fri, Feb 3, 2017 at 6:00 PM, Andres Freund wrote: > I still haven't seen a credible model for being able to apply a stream > of interleaved transactions that can roll back individually; I think we > really need the ability to have multiple transactions alive in one >

Re: [HACKERS] logical decoding of two-phase transactions

On 2017-02-03 17:47:50 -0500, Robert Haas wrote: > On Thu, Feb 2, 2017 at 7:14 PM, Craig Ringer wrote: > > We could make reorder buffers persistent and shared between decoding > > sessions but it'd totally change the logical decoding model and create > > some other

Re: [HACKERS] logical decoding of two-phase transactions

On Thu, Feb 2, 2017 at 7:14 PM, Craig Ringer wrote: > We could make reorder buffers persistent and shared between decoding > sessions but it'd totally change the logical decoding model and create > some other problems. It's certainly not a topic for this patch. So we > can

Re: [HACKERS] logical decoding of two-phase transactions

On 3 February 2017 at 03:34, Robert Haas wrote: > On Wed, Feb 1, 2017 at 4:35 PM, Craig Ringer wrote: >> Right. Per my comments uothread I don't see why we need to add anything more >> to WAL here. >> >> Stas was concerned about what happens in

Re: [HACKERS] logical decoding of two-phase transactions

On Wed, Feb 1, 2017 at 4:35 PM, Craig Ringer wrote: > Right. Per my comments uothread I don't see why we need to add anything more > to WAL here. > > Stas was concerned about what happens in logical decoding if we crash > between PREPSRE TRANSACTION and COMMIT PREPARED. But

Re: [HACKERS] logical decoding of two-phase transactions

On Wed, Feb 1, 2017 at 2:32 PM, Tom Lane wrote: > Robert Haas writes: >> Also, including the GID in the WAL for each COMMIT/ABORT PREPARED >> doesn't seem inordinately expensive to me. > > I'm confused ... isn't it there already? If not, how do we

Re: [HACKERS] logical decoding of two-phase transactions

On 2 Feb. 2017 08:32, "Tom Lane" wrote: Robert Haas writes: > Also, including the GID in the WAL for each COMMIT/ABORT PREPARED > doesn't seem inordinately expensive to me. I'm confused ... isn't it there already? If not, how do we handle

Re: [HACKERS] logical decoding of two-phase transactions

On 02/01/2017 10:32 PM, Tom Lane wrote: > Robert Haas writes: >> Also, including the GID in the WAL for each COMMIT/ABORT PREPARED >> doesn't seem inordinately expensive to me. > I'm confused ... isn't it there already? If not, how do we handle > reconstructing 2PC state

Re: [HACKERS] logical decoding of two-phase transactions

Robert Haas writes: > Also, including the GID in the WAL for each COMMIT/ABORT PREPARED > doesn't seem inordinately expensive to me. I'm confused ... isn't it there already? If not, how do we handle reconstructing 2PC state from WAL at all?

Re: [HACKERS] logical decoding of two-phase transactions

On Tue, Jan 31, 2017 at 9:05 PM, Michael Paquier wrote: >> Personally I don't think lack of access to the GID justifies blocking 2PC >> logical decoding. It can be added separately. But it'd be nice to have >> especially if it's cheap. > > I think it should be added

Re: [HACKERS] logical decoding of two-phase transactions

On Tue, Jan 31, 2017 at 6:22 PM, Craig Ringer wrote: > That's where you've misunderstood - it isn't committed yet. The point or > this change is to allow us to do logical decoding at the PREPARE TRANSACTION > point. The xact is not yet committed or rolled back. Yes, I got

Re: [HACKERS] logical decoding of two-phase transactions

On 31 Jan. 2017 22:43, "Konstantin Knizhnik" wrote: On 31.01.2017 09:29, Michael Paquier wrote: > On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer > wrote: > >> Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when >> wal_level

Re: [HACKERS] logical decoding of two-phase transactions

On 31.01.2017 09:29, Michael Paquier wrote: On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer wrote: Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when wal_level >= logical I don't think that's the end of the world. But since we already have almost

Re: [HACKERS] logical decoding of two-phase transactions

On 31 Jan. 2017 19:29, "Michael Paquier" wrote: On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer wrote: > Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when > wal_level >= logical I don't think that's the end of the world. But >

Re: [HACKERS] logical decoding of two-phase transactions

On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer wrote: > Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when > wal_level >= logical I don't think that's the end of the world. But > since we already have almost everything we need in memory, why not > just

Re: [HACKERS] logical decoding of two-phase transactions

On Tue, Jan 31, 2017 at 3:29 PM, Michael Paquier wrote: > On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer wrote: >> Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when >> wal_level >= logical I don't think that's the end of the

Re: [HACKERS] logical decoding of two-phase transactions

On 26 January 2017 at 19:34, Stas Kelvich wrote: > Imagine following scenario: > > 1. PREPARE happend > 2. PREPARE decoded and sent where it should be sent > 3. We got all responses from participating nodes and issuing COMMIT/ABORT > 4. COMMIT/ABORT decoded and sent > >

Re: [HACKERS] logical decoding of two-phase transactions

> On 26 Jan 2017, at 12:51, Craig Ringer wrote: > > * Tracking xid/gid map in memory also doesn’t help much — if server reboots > between prepare > and commit we’ll lose that mapping. > > Er what? That's why I suggested using the prepared xacts shmem state. It's >

Re: [HACKERS] logical decoding of two-phase transactions

On 26 Jan. 2017 18:43, "Stas Kelvich" wrote: >> >> Yes, that’s also possible but seems to be less flexible restricting us to some >> specific GID format. >> >> Anyway, I can measure WAL space overhead introduced by the GID’s inside commit records >> to know exactly what

Re: [HACKERS] logical decoding of two-phase transactions

>> >> Yes, that’s also possible but seems to be less flexible restricting us to >> some >> specific GID format. >> >> Anyway, I can measure WAL space overhead introduced by the GID’s inside >> commit records >> to know exactly what will be the cost of such approach. > > Stas, > > Have you

Re: [HACKERS] logical decoding of two-phase transactions

On 5 January 2017 at 20:43, Stas Kelvich wrote: > >> On 5 Jan 2017, at 13:49, Simon Riggs wrote: >> >> Surely in this case the master server is acting as the Transaction >> Manager, and it knows the mapping, so we are good? >> >> I guess if you

Re: [HACKERS] logical decoding of two-phase transactions

On 5 January 2017 at 12:43, Stas Kelvich wrote: > >> On 5 Jan 2017, at 13:49, Simon Riggs wrote: >> >> Surely in this case the master server is acting as the Transaction >> Manager, and it knows the mapping, so we are good? >> >> I guess if you

Re: [HACKERS] logical decoding of two-phase transactions

On 5 January 2017 at 20:43, Stas Kelvich wrote: > Anyway, I can measure WAL space overhead introduced by the GID’s inside > commit records > to know exactly what will be the cost of such approach. Sounds like a good idea, especially if you remove any attempt to work

Re: [HACKERS] logical decoding of two-phase transactions

> On 5 Jan 2017, at 13:49, Simon Riggs wrote: > > Surely in this case the master server is acting as the Transaction > Manager, and it knows the mapping, so we are good? > > I guess if you are using >2 nodes then you need to use full 2PC on each node. > > Please explain

Re: [HACKERS] logical decoding of two-phase transactions

On 5 January 2017 at 10:21, Stas Kelvich wrote: > Thank you for looking into this. > >> On 5 Jan 2017, at 09:43, Simon Riggs wrote: >>> >>> GID is now variable sized. You seem to have added this to every >>> commit, not just 2PC >> > > Hm, didn’t

Re: [HACKERS] logical decoding of two-phase transactions

Thank you for looking into this. > On 5 Jan 2017, at 09:43, Simon Riggs wrote: >> >> GID is now variable sized. You seem to have added this to every >> commit, not just 2PC > Hm, didn’t realise that, i’ll fix. > I've just realised that you're adding GID because it

Re: [HACKERS] logical decoding of two-phase transactions

On 4 January 2017 at 21:20, Simon Riggs wrote: > On 31 December 2016 at 08:36, Stas Kelvich wrote: >> Here is resubmission of patch to implement logical decoding of two-phase >> transactions (instead of treating them >> as usual transaction when

Re: [HACKERS] logical decoding of two-phase transactions

On 31 December 2016 at 08:36, Stas Kelvich wrote: > Here is resubmission of patch to implement logical decoding of two-phase > transactions (instead of treating them > as usual transaction when commit) [1] I’ve slightly polished things and used > test_decoding output