Re: out-of-order XID insertion in KnownAssignedXids

2018-10-11 Thread Michael Paquier
On Thu, Oct 11, 2018 at 08:04:11PM +0300, Konstantin Knizhnik wrote: > Proposed patch is attached. The problem I have with this patch doing the duplication removal and qsort work in LogCurrentRunningXacts is that it would still lock ProcArrayLock until the WAL record has been written with wal_leve

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-11 Thread Konstantin Knizhnik
On 11.10.2018 12:06, Michael Paquier wrote: On Wed, Oct 10, 2018 at 11:22:45AM +0900, Michael Paquier wrote: I am not sure if the performance argument is actually this much sensible, it could be as you say, but another thing that we could argue about is that the presence of duplicate entries i

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-11 Thread Michael Paquier
On Wed, Oct 10, 2018 at 11:22:45AM +0900, Michael Paquier wrote: > I am not sure if the performance argument is actually this much > sensible, it could be as you say, but another thing that we could argue > about is that the presence of duplicate entries in > GetRunningTransactionData() can be used

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-09 Thread Michael Paquier
On Tue, Oct 09, 2018 at 05:26:49PM +0300, Konstantin Knizhnik wrote: > But GetRunningTransactionData may be used in some other cases... For > example I have used it in my snapfs Postgres extensions (fast > file-level snapshots) to check if there are no more active > transactions in the system. In m

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-09 Thread Konstantin Knizhnik
On 09.10.2018 10:52, Michael Paquier wrote: On Tue, Oct 09, 2018 at 02:59:00PM +0900, Michael Paquier wrote: +1. I am looking at how to make that possible. And so... Going through a bit of investigation the problem is that each 2PC transaction preparing finishes by putting the procarray in

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-09 Thread Michael Paquier
On Tue, Oct 09, 2018 at 02:59:00PM +0900, Michael Paquier wrote: > +1. I am looking at how to make that possible. And so... Going through a bit of investigation the problem is that each 2PC transaction preparing finishes by putting the procarray in a state where there are two entries referring t

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Michael Paquier
On Mon, Oct 08, 2018 at 09:30:49AM -0700, Andres Freund wrote: > Sounds less terrible, but still pretty bad. I think we should fix the > underlying data inconsistency, not paper over it a couple hundred meters > away. +1. I am looking at how to make that possible. -- Michael signature.asc Desc

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Andres Freund
gt; >     if (head > tail && > > > > >         TransactionIdFollowsOrEquals(KnownAssignedXids[head - 1], > > > from_xid)) > > > > >     { > > > > >         KnownAssignedXidsDisplay(LOG); > > > > >         elog(ERROR,

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Konstantin Knizhnik
error in KnownAssignedXidsAdd:     if (head > tail &&         TransactionIdFollowsOrEquals(KnownAssignedXids[head - 1], from_xid))     {         KnownAssignedXidsDisplay(LOG);         elog(ERROR, "out-of-order XID insertion in KnownAssignedXids");     } Th

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Andres Freund
in >>> KnownAssignedXidsAdd: >>> >>>     if (head > tail && >>>         TransactionIdFollowsOrEquals(KnownAssignedXids[head - 1], >from_xid)) >>>     { >>>         KnownAssignedXidsDisplay(LOG); >>>         elog(ERROR,

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Konstantin Knizhnik
On 08.10.2018 12:14, Michael Paquier wrote: On Mon, Oct 08, 2018 at 12:04:28PM +0300, Konstantin Knizhnik wrote: The simplest way to fix the problem is to ignore duplicates before adding them to KnownAssignedXids. We in any case perform sort i this place... I may of course be missing somethi

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Michael Paquier
On Mon, Oct 08, 2018 at 12:04:28PM +0300, Konstantin Knizhnik wrote: > The simplest way to fix the problem is to ignore duplicates before adding > them to KnownAssignedXids. > We in any case perform sort i this place... I may of course be missing something, but shouldn't we not have duplicates in

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Konstantin Knizhnik
ssignedXids[head - 1], from_xid))     {         KnownAssignedXidsDisplay(LOG);         elog(ERROR, "out-of-order XID insertion in KnownAssignedXids");     } The probability of this error is very small but it can quite easily reproduced: you should just set breakpoint in debugger after c

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-05 Thread Konstantin Knizhnik
ssignedXids[head - 1], from_xid))     {         KnownAssignedXidsDisplay(LOG);         elog(ERROR, "out-of-order XID insertion in KnownAssignedXids");     } The probability of this error is very small but it can quite easily reproduced: you should just set breakpoint in d

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-05 Thread Michael Paquier
edXids[head - 1], from_xid)) >     { >         KnownAssignedXidsDisplay(LOG); >         elog(ERROR, "out-of-order XID insertion in KnownAssignedXids"); >     } > > The probability of this error is very small but it can quite easily > reproduced: you should just set breakpoint in debugger

out-of-order XID insertion in KnownAssignedXids

2018-10-05 Thread Konstantin Knizhnik
Hi hackers, Looks like there is a bug with logging running transactions XIDs and prepared transactions. One of our customers get error "FATAL: out-of-order XID insertion in KnownAssignedXids" trying to apply backup. WAL contains the following record: rmgr: Standby len (rec/to