Re: out-of-order XID insertion in KnownAssignedXids

2018-10-11 Thread Michael Paquier
On Thu, Oct 11, 2018 at 08:04:11PM +0300, Konstantin Knizhnik wrote: > Proposed patch is attached. The problem I have with this patch doing the duplication removal and qsort work in LogCurrentRunningXacts is that it would still lock ProcArrayLock until the WAL record has been written with wal_leve

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-11 Thread Konstantin Knizhnik
On 11.10.2018 12:06, Michael Paquier wrote: On Wed, Oct 10, 2018 at 11:22:45AM +0900, Michael Paquier wrote: I am not sure if the performance argument is actually this much sensible, it could be as you say, but another thing that we could argue about is that the presence of duplicate entries i

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-11 Thread Michael Paquier
On Wed, Oct 10, 2018 at 11:22:45AM +0900, Michael Paquier wrote: > I am not sure if the performance argument is actually this much > sensible, it could be as you say, but another thing that we could argue > about is that the presence of duplicate entries in > GetRunningTransactionData() can be used

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-09 Thread Michael Paquier
On Tue, Oct 09, 2018 at 05:26:49PM +0300, Konstantin Knizhnik wrote: > But GetRunningTransactionData may be used in some other cases... For > example I have used it in my snapfs Postgres extensions (fast > file-level snapshots) to check if there are no more active > transactions in the system. In m

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-09 Thread Konstantin Knizhnik
On 09.10.2018 10:52, Michael Paquier wrote: On Tue, Oct 09, 2018 at 02:59:00PM +0900, Michael Paquier wrote: +1. I am looking at how to make that possible. And so... Going through a bit of investigation the problem is that each 2PC transaction preparing finishes by putting the procarray in

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-09 Thread Michael Paquier
On Tue, Oct 09, 2018 at 02:59:00PM +0900, Michael Paquier wrote: > +1. I am looking at how to make that possible. And so... Going through a bit of investigation the problem is that each 2PC transaction preparing finishes by putting the procarray in a state where there are two entries referring t

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Michael Paquier
On Mon, Oct 08, 2018 at 09:30:49AM -0700, Andres Freund wrote: > Sounds less terrible, but still pretty bad. I think we should fix the > underlying data inconsistency, not paper over it a couple hundred meters > away. +1. I am looking at how to make that possible. -- Michael signature.asc Desc

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Andres Freund
On 2018-10-08 18:28:52 +0300, Konstantin Knizhnik wrote: > > > On 08.10.2018 18:24, Andres Freund wrote: > > > > On October 8, 2018 2:04:28 AM PDT, Konstantin Knizhnik > > wrote: > > > > > > On 05.10.2018 11:04, Michael Paquier wrote: > > > > On Fri, Oct 05, 2018 at 10:06:45AM +0300, Konstant

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Konstantin Knizhnik
On 08.10.2018 18:24, Andres Freund wrote: On October 8, 2018 2:04:28 AM PDT, Konstantin Knizhnik wrote: On 05.10.2018 11:04, Michael Paquier wrote: On Fri, Oct 05, 2018 at 10:06:45AM +0300, Konstantin Knizhnik wrote: As you can notice, XID 2004495308 is encountered twice which cause er

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Andres Freund
On October 8, 2018 2:04:28 AM PDT, Konstantin Knizhnik wrote: > > >On 05.10.2018 11:04, Michael Paquier wrote: >> On Fri, Oct 05, 2018 at 10:06:45AM +0300, Konstantin Knizhnik wrote: >>> As you can notice, XID 2004495308 is encountered twice which cause >error in >>> KnownAssignedXidsAdd: >>>

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Konstantin Knizhnik
On 08.10.2018 12:14, Michael Paquier wrote: On Mon, Oct 08, 2018 at 12:04:28PM +0300, Konstantin Knizhnik wrote: The simplest way to fix the problem is to ignore duplicates before adding them to KnownAssignedXids. We in any case perform sort i this place... I may of course be missing somethi

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Michael Paquier
On Mon, Oct 08, 2018 at 12:04:28PM +0300, Konstantin Knizhnik wrote: > The simplest way to fix the problem is to ignore duplicates before adding > them to KnownAssignedXids. > We in any case perform sort i this place... I may of course be missing something, but shouldn't we not have duplicates in

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-08 Thread Konstantin Knizhnik
On 05.10.2018 11:04, Michael Paquier wrote: On Fri, Oct 05, 2018 at 10:06:45AM +0300, Konstantin Knizhnik wrote: As you can notice, XID 2004495308 is encountered twice which cause error in KnownAssignedXidsAdd:     if (head > tail &&         TransactionIdFollowsOrEquals(KnownAssignedXids[he

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-05 Thread Konstantin Knizhnik
On 05.10.2018 11:04, Michael Paquier wrote: On Fri, Oct 05, 2018 at 10:06:45AM +0300, Konstantin Knizhnik wrote: As you can notice, XID 2004495308 is encountered twice which cause error in KnownAssignedXidsAdd:     if (head > tail &&         TransactionIdFollowsOrEquals(KnownAssignedXids[h

Re: out-of-order XID insertion in KnownAssignedXids

2018-10-05 Thread Michael Paquier
On Fri, Oct 05, 2018 at 10:06:45AM +0300, Konstantin Knizhnik wrote: > As you can notice, XID 2004495308 is encountered twice which cause error in > KnownAssignedXidsAdd: > >     if (head > tail && >         TransactionIdFollowsOrEquals(KnownAssignedXids[head - 1], from_xid)) >     { >         Kno