Re: [HACKERS] SSI freezing bug

2013-10-08 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: A comment somewhere would be nice to explain why we're no longer worried about confusing an old tuple version with a new tuple in the same slot. Not sure where. For now I counted on the commit message.  Perhaps we also want to add something to

Re: [HACKERS] SSI freezing bug

2013-10-08 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: On 07.10.2013 23:45, Heikki Linnakangas wrote: To fix the bug that Hannu pointed out, we also need to apply these fixes: http://www.postgresql.org/message-id/52440266.5040...@vmware.com Per a chat with Bruce, I'm going to apply that patch

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 06.10.2013 20:34, Kevin Grittner wrote: Note this comment, which I think was written by Heikki back when there was a lot more benchmarking and profiling of SSI going on: * Because a particular target might become obsolete, due to update to a new * version, before the reading

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: So I don't think you can ever get a false conflict because of slot reuse. I spent some time looking at this, and I now agree. And if there's a hole in that thinking I can't see right now, the worst that will happen is some unnecessary

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Patch attached.  Any objections to applying that Real Soon Now? Oh, without the new line in predicate.h. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Andres Freund
On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached.  Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly careful, but I personally slightly vote for applying it after the backbranch releases.

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 16:58, Andres Freund wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached. Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly careful, but I personally slightly vote for

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 16:44, Kevin Grittner wrote: Heikki Linnakangashlinnakan...@vmware.com wrote: So I don't think you can ever get a false conflict because of slot reuse. I spent some time looking at this, and I now agree. And if there's a hole in that thinking I can't see right now, the worst

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Andres Freund
On 2013-10-07 17:07:16 +0300, Heikki Linnakangas wrote: On 07.10.2013 16:58, Andres Freund wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached. Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: Well, it's fairly harsh if the feature isn't working as advertised. Right -- there are people counting on serializable transaction to avoid data corruption (creation of data which violates the business rules which they believe are being

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached.  Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly careful, but I personally slightly vote

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached.  Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending.  Still investigating the impact and possible fixes. I found a one-line fix for this.  It tested without problem. Pushed. -- Kevin Grittner EDB:

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Dan Ports
On Mon, Oct 07, 2013 at 12:26:37PM +0300, Heikki Linnakangas wrote: When updating a tuple, CheckTargetForConflictsIn() only marks a conflict if the transaction holding the predicate lock overlapped with the updating transaction. Ah, this is the bit I was forgetting. (I really ought to have

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 22:35, Kevin Grittner wrote: Kevin Grittnerkgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending. Still investigating the impact and possible fixes. I found a one-line fix for this. It tested without

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 23:45, Heikki Linnakangas wrote: On 07.10.2013 22:35, Kevin Grittner wrote: Kevin Grittnerkgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending. Still investigating the impact and possible fixes. I found a

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Peter Eisentraut
On Mon, 2013-10-07 at 23:49 +0300, Heikki Linnakangas wrote: On 07.10.2013 23:45, Heikki Linnakangas wrote: On 07.10.2013 22:35, Kevin Grittner wrote: Kevin Grittnerkgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending.

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 08.10.2013 03:25, Peter Eisentraut wrote: On Mon, 2013-10-07 at 23:49 +0300, Heikki Linnakangas wrote: To fix the bug that Hannu pointed out, we also need to apply these fixes: http://www.postgresql.org/message-id/52440266.5040...@vmware.com Per a chat with Bruce, I'm going to apply that

Re: [HACKERS] SSI freezing bug

2013-10-06 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Kevin Grittner kgri...@ymail.com wrote: I'm strongly leaning toward the idea that a slightly tweaked version of the proposed patch is appropriate for the back-branches, because the fix Heikki is now suggesting seems too invasive to back-patch.  I think

Re: [HACKERS] SSI freezing bug

2013-10-04 Thread Andres Freund
On 2013-10-03 21:14:17 -0700, Dan Ports wrote: On Thu, Oct 03, 2013 at 06:19:49AM -0700, Kevin Grittner wrote: Heikki Linnakangas hlinnakan...@vmware.com wrote: IMHO it would be better to remove xmin from the lock key, and vacuum away the old predicate locks when the corresponding tuple

Re: [HACKERS] SSI freezing bug

2013-10-04 Thread Heikki Linnakangas
On 04.10.2013 13:23, Andres Freund wrote: On 2013-10-03 21:14:17 -0700, Dan Ports wrote: On Thu, Oct 03, 2013 at 06:19:49AM -0700, Kevin Grittner wrote: Heikki Linnakangashlinnakan...@vmware.com wrote: IMHO it would be better to remove xmin from the lock key, and vacuum away the old

Re: [HACKERS] SSI freezing bug

2013-10-04 Thread Andres Freund
On 2013-10-04 13:51:00 +0300, Heikki Linnakangas wrote: On 04.10.2013 13:23, Andres Freund wrote: On 2013-10-03 21:14:17 -0700, Dan Ports wrote: On Thu, Oct 03, 2013 at 06:19:49AM -0700, Kevin Grittner wrote: Heikki Linnakangashlinnakan...@vmware.com wrote: IMHO it would be better to remove

Re: [HACKERS] SSI freezing bug

2013-10-04 Thread Heikki Linnakangas
On 04.10.2013 14:02, Andres Freund wrote: But locks on those still can have meaning, right? From my very limited understanding of predicate.c/SSI I don't see why we cannot have meaningful conflicts on tuples that are already vacuumable. You'd need some curiously interleaved transactions, sure,

Re: [HACKERS] SSI freezing bug

2013-10-04 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-10-03 21:14:17 -0700, Dan Ports wrote: Heikki Linnakangashlinnakan...@vmware.com  wrote: IMHO it would be better to remove xmin from the lock key, and vacuum away the old predicate locks when the corresponding tuple is vacuumed. The xmin

Re: [HACKERS] SSI freezing bug

2013-10-03 Thread Heikki Linnakangas
On 03.10.2013 01:05, Kevin Grittner wrote: Andres Freundand...@2ndquadrant.com wrote: On 2013-10-01 07:41:46 -0700, Kevin Grittner wrote: Andres Freundand...@2ndquadrant.com wrote: A better solution probably is to promote tuple-level locks if they exist to a relation level one upon

Re: [HACKERS] SSI freezing bug

2013-10-03 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: IMHO it would be better to remove xmin from the lock key, and vacuum away the old predicate locks when the corresponding tuple is vacuumed. The xmin field is only required to handle the case that a tuple is vacuumed, and a new unrelated tuple

Re: [HACKERS] SSI freezing bug

2013-10-03 Thread Dan Ports
On Thu, Oct 03, 2013 at 06:19:49AM -0700, Kevin Grittner wrote: Heikki Linnakangas hlinnakan...@vmware.com wrote: IMHO it would be better to remove xmin from the lock key, and vacuum away the old predicate locks when the corresponding tuple is vacuumed. The xmin field is only required to

Re: [HACKERS] SSI freezing bug

2013-10-01 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: A better solution probably is to promote tuple-level locks if they exist to a relation level one upon freezing I guess? It would be sufficient to promote the tuple lock to a page lock. It would be pretty easy to add a function to predicate.c which

Re: [HACKERS] SSI freezing bug

2013-10-01 Thread Andres Freund
On 2013-10-01 07:41:46 -0700, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: A better solution probably is to promote tuple-level locks if they exist to a relation level one upon freezing I guess? It would be sufficient to promote the tuple lock to a page lock. It

Re: [HACKERS] SSI freezing bug

2013-09-26 Thread Heikki Linnakangas
On 23.09.2013 01:07, Hannu Krosing wrote: On 09/20/2013 12:55 PM, Heikki Linnakangas wrote: Hi, Prompted by Andres Freund's comments on my Freezing without Write I/O patch, I realized that there's there's an existing bug in the way predicate locking handles freezing (or rather, it doesn't

Re: [HACKERS] SSI freezing bug

2013-09-25 Thread Heikki Linnakangas
On 22.09.2013 00:12, Hannu Krosing wrote: On 09/21/2013 10:46 PM, Andres Freund wrote: Heikki Linnakangashlinnakan...@vmware.com schrieb: Kevin Grittnerkgri...@ymail.com wrote: Andres Freundand...@2ndquadrant.com wrote: On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a

Re: [HACKERS] SSI freezing bug

2013-09-25 Thread Heikki Linnakangas
On 21.09.2013 23:46, Andres Freund wrote: Heikki Linnakangashlinnakan...@vmware.com schrieb: Kevin Grittnerkgri...@ymail.com wrote: Andres Freundand...@2ndquadrant.com wrote: On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a tuple is predicate-locked, the key of the lock is

Re: [HACKERS] SSI freezing bug

2013-09-22 Thread Hannu Krosing
On 09/20/2013 12:55 PM, Heikki Linnakangas wrote: Hi, Prompted by Andres Freund's comments on my Freezing without Write I/O patch, I realized that there's there's an existing bug in the way predicate locking handles freezing (or rather, it doesn't handle it). When a tuple is

Re: [HACKERS] SSI freezing bug

2013-09-21 Thread Heikki Linnakangas
Kevin Grittner kgri...@ymail.com wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a tuple is predicate-locked, the key of the lock is ctid+xmin. However, when a tuple is frozen, its xmin is changed to FrozenXid. That effectively

Re: [HACKERS] SSI freezing bug

2013-09-21 Thread Andres Freund
Heikki Linnakangas hlinnakan...@vmware.com schrieb: Kevin Grittner kgri...@ymail.com wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a tuple is predicate-locked, the key of the lock is ctid+xmin. However, when a tuple is frozen,

Re: [HACKERS] SSI freezing bug

2013-09-21 Thread Hannu Krosing
On 09/21/2013 10:46 PM, Andres Freund wrote: Heikki Linnakangas hlinnakan...@vmware.com schrieb: Kevin Grittner kgri...@ymail.com wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a tuple is predicate-locked, the key of the lock

[HACKERS] SSI freezing bug

2013-09-20 Thread Heikki Linnakangas
Hi, Prompted by Andres Freund's comments on my Freezing without Write I/O patch, I realized that there's there's an existing bug in the way predicate locking handles freezing (or rather, it doesn't handle it). When a tuple is predicate-locked, the key of the lock is ctid+xmin. However, when

Re: [HACKERS] SSI freezing bug

2013-09-20 Thread Andres Freund
Hi, On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a tuple is predicate-locked, the key of the lock is ctid+xmin. However, when a tuple is frozen, its xmin is changed to FrozenXid. That effectively invalidates any predicate lock on the tuple, as checking for a lock on the same

Re: [HACKERS] SSI freezing bug

2013-09-20 Thread Andres Freund
On 2013-09-20 13:53:04 +0200, Andres Freund wrote: Hi, On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a tuple is predicate-locked, the key of the lock is ctid+xmin. However, when a tuple is frozen, its xmin is changed to FrozenXid. That effectively invalidates any

Re: [HACKERS] SSI freezing bug

2013-09-20 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-09-20 13:55:36 +0300, Heikki Linnakangas wrote: When a tuple is predicate-locked, the key of the lock is ctid+xmin. However, when a tuple is frozen, its xmin is changed to FrozenXid. That effectively invalidates any predicate lock on the