Re: Duplicated LSN in ReorderBuffer

2020-01-31 Thread Amit Kapila
On Fri, Jan 31, 2020 at 12:35 AM Alvaro Herrera wrote: > > On 2019-Jul-26, Andres Freund wrote: > > > Petr, Simon, see the potential issue related to fast forward at the > > bottom. > > I think we neglected this bit. I looked at the patch Simon submitted > downthread, and while I vaguely

Re: Duplicated LSN in ReorderBuffer

2020-01-30 Thread Alvaro Herrera
On 2019-Jul-26, Andres Freund wrote: > Petr, Simon, see the potential issue related to fast forward at the > bottom. I think we neglected this bit. I looked at the patch Simon submitted downthread, and while I vaguely understand that we need to process NEW_CID records during fast-forwarding, I

Re: Duplicated LSN in ReorderBuffer

2019-09-13 Thread Andres Freund
Hi, On 2019-09-13 16:42:47 -0300, Alvaro Herrera wrote: > On 2019-Sep-10, Alvaro Herrera from 2ndQuadrant wrote: > > > Here's a couple of patches. > > > > always_decode_assignment.patch is Masahiko Sawada's patch, which has > > been confirmed to fix the assertion failure. > > I pushed this one

Re: Duplicated LSN in ReorderBuffer

2019-09-13 Thread Alvaro Herrera
On 2019-Sep-10, Alvaro Herrera from 2ndQuadrant wrote: > Here's a couple of patches. > > always_decode_assignment.patch is Masahiko Sawada's patch, which has > been confirmed to fix the assertion failure. I pushed this one to all branches. Thanks Ildar for reporting and Sawada-san for fixing,

Re: Duplicated LSN in ReorderBuffer

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
Hello, On 2019-Sep-06, Andres Freund wrote: > On 2019-08-19 08:51:43 -0700, Andres Freund wrote: > > On August 19, 2019 7:43:28 AM PDT, Alvaro Herrera > > wrote: > > >Never mind. I was able to reproduce it later, and verify that Andres' > > >proposed strategy doesn't seem to fix the problem.

Re: Duplicated LSN in ReorderBuffer

2019-09-06 Thread Andres Freund
Hi, On 2019-08-19 08:51:43 -0700, Andres Freund wrote: > On August 19, 2019 7:43:28 AM PDT, Alvaro Herrera > wrote: > >On 2019-Aug-19, Masahiko Sawada wrote: > > > >> On Tue, Aug 13, 2019 at 6:36 AM Alvaro Herrera > > wrote: > > > >> > BTW I wrote the code as suggested and it passes all the

Re: Duplicated LSN in ReorderBuffer

2019-08-19 Thread Andres Freund
Hi, On August 19, 2019 7:43:28 AM PDT, Alvaro Herrera wrote: >On 2019-Aug-19, Masahiko Sawada wrote: > >> On Tue, Aug 13, 2019 at 6:36 AM Alvaro Herrera > wrote: > >> > BTW I wrote the code as suggested and it passes all the tests ... >but I >> > then noticed that the unpatched code doesn't

Re: Duplicated LSN in ReorderBuffer

2019-08-19 Thread Alvaro Herrera
On 2019-Aug-19, Masahiko Sawada wrote: > On Tue, Aug 13, 2019 at 6:36 AM Alvaro Herrera > wrote: > > BTW I wrote the code as suggested and it passes all the tests ... but I > > then noticed that the unpatched code doesn't fail Ildar's original > > pgbench-based test for me, either. So maybe

Re: Duplicated LSN in ReorderBuffer

2019-08-19 Thread Masahiko Sawada
On Tue, Aug 13, 2019 at 6:36 AM Alvaro Herrera wrote: > > On 2019-Aug-07, Andres Freund wrote: > > > I think we would need to do this for all values of > > SnapBuildCurrentState() - after all the problem occurs because we > > *previously* didn't assign subxids to the toplevel xid. Compared to

Re: Duplicated LSN in ReorderBuffer

2019-08-12 Thread Alvaro Herrera
On 2019-Aug-07, Andres Freund wrote: > I think we would need to do this for all values of > SnapBuildCurrentState() - after all the problem occurs because we > *previously* didn't assign subxids to the toplevel xid. Compared to the > cost of catalog changes, ReorderBufferAssignChild() is really

Re: Duplicated LSN in ReorderBuffer

2019-08-07 Thread Andres Freund
Hi, On 2019-08-07 16:19:13 -0400, Alvaro Herrera wrote: > On 2019-Jul-26, Andres Freund wrote: > > > 2) We could simply assign the subtransaction to the parent using > >ReorderBufferAssignChild() in SnapBuildProcessNewCid() or it's > >caller. That ought to also fix the bug > > > >I

Re: Duplicated LSN in ReorderBuffer

2019-08-07 Thread Alvaro Herrera
On 2019-Jul-26, Andres Freund wrote: > 2) We could simply assign the subtransaction to the parent using >ReorderBufferAssignChild() in SnapBuildProcessNewCid() or it's >caller. That ought to also fix the bug > >I also has the advantage that we can save some memory in transactions >

Re: Duplicated LSN in ReorderBuffer

2019-07-28 Thread Simon Riggs
On Mon, 29 Jul 2019 at 00:09, Petr Jelinek wrote: > Given that we don't process any other records in this function besides > XLOG_HEAP2_MULTI_INSERT and XLOG_HEAP2_NEW_CID, it seems like simplest > fix is to just remove the first check for fast forward and keep the one > in

Re: Duplicated LSN in ReorderBuffer

2019-07-28 Thread Petr Jelinek
Hi, On 27. 07. 19 3:15, Andres Freund wrote: Hi, Petr, Simon, see the potential issue related to fast forward at the bottom. [..snip..] This actually made me look at the nearby changes due to commit 9c7d06d60680c7f00d931233873dee81fdb311c6 Author: Simon Riggs Date: 2018-01-17 11:38:34

Re: Duplicated LSN in ReorderBuffer

2019-07-26 Thread Andres Freund
Hi, Petr, Simon, see the potential issue related to fast forward at the bottom. On 2019-07-26 18:46:35 -0400, Alvaro Herrera wrote: > On 2019-Jul-09, Masahiko Sawada wrote: > > > I think the cause of this bug would be that a ReorderBufferTXN entry > > of sub transaction is created as top-level

Re: Duplicated LSN in ReorderBuffer

2019-07-26 Thread Alvaro Herrera
On 2019-Jul-09, Masahiko Sawada wrote: > I think the cause of this bug would be that a ReorderBufferTXN entry > of sub transaction is created as top-level transaction. And this > happens because we skip to decode ASSIGNMENT during the state of > snapshot builder < SNAPBUILD_FULL. That

Re: Duplicated LSN in ReorderBuffer

2019-07-09 Thread Masahiko Sawada
On Mon, Jul 8, 2019 at 11:46 PM Masahiko Sawada wrote: > > On Mon, Jul 8, 2019 at 9:00 AM Thomas Munro wrote: > > > > On Wed, Jun 26, 2019 at 2:46 AM Ildar Musin wrote: > > > Attached is a simple patch that uses subxid instead of top-level xid > > > in ReorderBufferAddNewTupleCids() call. It

Re: Duplicated LSN in ReorderBuffer

2019-07-08 Thread Masahiko Sawada
On Mon, Jul 8, 2019 at 9:00 AM Thomas Munro wrote: > > On Wed, Jun 26, 2019 at 2:46 AM Ildar Musin wrote: > > Attached is a simple patch that uses subxid instead of top-level xid > > in ReorderBufferAddNewTupleCids() call. It seems to fix the bug, but > > i'm not sure that this is a valid

Re: Duplicated LSN in ReorderBuffer

2019-07-07 Thread Thomas Munro
On Wed, Jun 26, 2019 at 2:46 AM Ildar Musin wrote: > Attached is a simple patch that uses subxid instead of top-level xid > in ReorderBufferAddNewTupleCids() call. It seems to fix the bug, but > i'm not sure that this is a valid change. Can someone please verify it > or maybe suggest a better

Duplicated LSN in ReorderBuffer

2019-06-25 Thread Ildar Musin
Hi hackers, I believe we found a bug in logical decoding. It only occures with casserts enabled. It was originally discovered and reproduced by Murat Kabilov and Ildus Kurbangaliev. Here is the stacktrace we've got: #0 0x7facc66ef82f in raise () from /usr/lib/libc.so.6 #1