Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 02:06:19PM -0400, Tom Lane wrote: > Sooner or later we are going to need to go to 8-byte TOAST object > identifiers. Maybe we should think about doing that sooner not > later rather than trying to invent some anti-wraparound solution > here. Yay! Is there any lift in

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 5/10/16 4:12 PM, Andres Freund wrote: The catalog representation (as in pg_class.reltoastrelid) isn't entirely clear to me. One way would be to invert pg_class.reltoastrelid's meaning and have the toast table point to the table it stores values for. That'd also open the potential of having

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Mon, May 16, 2016 at 8:28 AM, Andres Freund wrote: > On 2016-05-10 17:58:33 -0700, Andres Freund wrote: >> FWIW, I've commented out the relevant sections from xlog_redo and since >> then I've not been able to reproduce the issue. > > A couple days of running later, and it

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 17:58:33 -0700, Andres Freund wrote: > FWIW, I've commented out the relevant sections from xlog_redo and since > then I've not been able to reproduce the issue. A couple days of running later, and it hasn't yet re-triggered. So I assume that's indeed the fix. -- Sent via

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On May 11, 2016 7:06:05 AM PDT, Teodor Sigaev wrote: >>> Allow Pin/UnpinBuffer to operate in a lockfree manner. >>> I get the errors: >>> >>> ERROR: attempted to delete invisible tuple >>> ERROR: unexpected chunk number 1 (expected 2) for toast value > >Just reminder,

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Allow Pin/UnpinBuffer to operate in a lockfree manner. I get the errors: ERROR: attempted to delete invisible tuple ERROR: unexpected chunk number 1 (expected 2) for toast value Just reminder, you investigate "unexpected chunk number" problem, but, seems, we have another bug (first

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Jeff Janes writes: > But, another perhaps stupid question, why do we care what the value of > nextOid was at the start of the last successfully completed > checkpoint? The intended use of that field is to restore nextOid before replaying WAL. So it should correspond to the

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 17:36:06 -0700, Jeff Janes wrote: > OK, got it. I don' t know how I missed the bigger picture of that > function in the first place. > > But, another perhaps stupid question, why do we care what the value of > nextOid was at the start of the last successfully completed >

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 4:02 PM, Andres Freund wrote: > On 2016-05-10 15:53:38 -0700, Jeff Janes wrote: >> >> But isn't CreateCheckPoint called at the end of the checkpoint, not >> the start of it? > > No, CreateCheckPoint() does it all. > > > CreateCheckPoint(int flags) > { >

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 15:53:38 -0700, Jeff Janes wrote: > On Tue, May 10, 2016 at 2:00 PM, Andres Freund wrote: > > I think that's to blame here. Looking at the relevant WAL record shows: > > > > pg_xlogdump -p /data/freund/jj -s 2/12004018 -e 2/1327EA28|grep -E > >

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Andres Freund writes: > On 2016-05-10 18:29:59 -0400, Tom Lane wrote: >> Having said that, I still say that changing HeapTupleSatisfiesToast >> is the wrong thing. It can't go deciding not to return toast values >> because they're committed dead --- the parent tuple could

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 2:00 PM, Andres Freund wrote: > On 2016-05-10 09:19:16 -0700, Andres Freund wrote: >> On 2016-05-10 08:09:02 -0400, Robert Haas wrote: >> > On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: >> > > The easy way to trigger this

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 18:29:59 -0400, Tom Lane wrote: > Now having said that, you don't actually need *rapid* advancement > of the OID counter to have a potential problem. Imagine that some > transaction inserts a TOAST value and later fails, so that you have > a dead-but-unhinted toast tuple sitting

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Andres Freund writes: > On 2016-05-10 16:14:50 -0400, Tom Lane wrote: >> I think the problem is pretty hypothetical until you get to consuming a >> substantial part of the OID space within any one toast table, at which >> point you're going to need 8-byte toast OIDs.

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 13:17:52 -0700, Jeff Janes wrote: > On Tue, May 10, 2016 at 9:19 AM, Andres Freund wrote: > > On 2016-05-10 08:09:02 -0400, Robert Haas wrote: > >> On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: > >> > The easy way to trigger this

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 16:14:50 -0400, Tom Lane wrote: > I think the problem is pretty hypothetical until you get to consuming a > substantial part of the OID space within any one toast table, at which > point you're going to need 8-byte toast OIDs. Improving that situation > seems like something we can

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 15:20:39 -0300, Alvaro Herrera wrote: > Tom Lane wrote: > > Robert Haas writes: > > > On Tue, May 10, 2016 at 12:19 PM, Andres Freund > > > wrote: > > >> It's not super likely, yea. But you don't really need to "use" 4 billion > > >> oids

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 09:19:16 -0700, Andres Freund wrote: > On 2016-05-10 08:09:02 -0400, Robert Haas wrote: > > On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: > > > The easy way to trigger this problem would be to have an oid wraparound > > > - but the WAL shows that that's

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 9:19 AM, Andres Freund wrote: > On 2016-05-10 08:09:02 -0400, Robert Haas wrote: >> On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: >> > The easy way to trigger this problem would be to have an oid wraparound >> > - but the

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Alvaro Herrera writes: > Tom Lane wrote: >> Sooner or later we are going to need to go to 8-byte TOAST object >> identifiers. Maybe we should think about doing that sooner not later >> rather than trying to invent some anti-wraparound solution here. > Umm, it seems to

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 10 May 2016 at 17:20, Andres Freund wrote: > On 2016-05-10 12:28:57 +0200, Simon Riggs wrote: > > On 10 May 2016 at 09:05, Andres Freund wrote: > > > > > > > Is anybody ready with a good defense for SatisfiesToast not doing any > > > actual liveliness

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Tom Lane wrote: > Robert Haas writes: > > On Tue, May 10, 2016 at 12:19 PM, Andres Freund wrote: > >> It's not super likely, yea. But you don't really need to "use" 4 billion > >> oids to get a wraparound. Once you have a significant number of values >

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Robert Haas writes: > On Tue, May 10, 2016 at 12:19 PM, Andres Freund wrote: >> It's not super likely, yea. But you don't really need to "use" 4 billion >> oids to get a wraparound. Once you have a significant number of values >> in various toast

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 12:19 PM, Andres Freund wrote: >> I assume that this was installed as a performance optimization, and I >> don't really see why it shouldn't be or be able to be made safe. I >> assume that the wraparound case was deemed safe because at that time >> the

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 10:09:07 -0400, Tom Lane wrote: > Andres Freund writes: > > Is anybody ready with a good defense for SatisfiesToast not doing any > > actual liveliness checks? > > As long as we do not update toast values after creation, there is no > need; the liveness check on

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 12:28:57 +0200, Simon Riggs wrote: > On 10 May 2016 at 09:05, Andres Freund wrote: > > > > Is anybody ready with a good defense for SatisfiesToast not doing any > > actual liveliness checks? > > > > I provided a patch earlier that rechecks the OID fetched from

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 2016-05-10 08:09:02 -0400, Robert Haas wrote: > On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: > > The easy way to trigger this problem would be to have an oid wraparound > > - but the WAL shows that that's not the case here. I've not figured > > that one out

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Andres Freund writes: > Is anybody ready with a good defense for SatisfiesToast not doing any > actual liveliness checks? As long as we do not update toast values after creation, there is no need; the liveness check on the parent tuple is what's important. Adding a liveness

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On Tue, May 10, 2016 at 3:05 AM, Andres Freund wrote: > The easy way to trigger this problem would be to have an oid wraparound > - but the WAL shows that that's not the case here. I've not figured > that one out entirely (and won't tonight). But I do see WAL records > like:

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

On 10 May 2016 at 09:05, Andres Freund wrote: > Is anybody ready with a good defense for SatisfiesToast not doing any > actual liveliness checks? > I provided a patch earlier that rechecks the OID fetched from a toast chunk matches the OID requested. I didn't commit it, I

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

Hi, On 2016-04-29 10:38:55 -0700, Jeff Janes wrote: > I've bisected the errors I was seeing, discussed in > http://www.postgresql.org/message-id/CAMkU=1xqehc0ok4d+tkjfq1nvuho37pyrkhjp6q8oxifmx7...@mail.gmail.com > > It look like they first appear in: > > commit