Re: [HACKERS] foreign key locks

2013-01-23 Thread Alvaro Herrera
I just pushed this patch to the master branch. There was a corresponding catversion bump and pg_control version bump. I have verified that "make check-world" passes on my machine, as well as isolation tests and pg_upgrade. Tom Lane said at one point "this is too complex to maintain". Several ti

Re: [HACKERS] foreign key locks

2013-01-19 Thread Simon Riggs
On 18 January 2013 21:01, Tom Lane wrote: > Andres Freund writes: >> On 2013-01-18 15:37:47 -0500, Tom Lane wrote: >>> I doubt it ever came up before. What use is logging only the content of >>> a buffer page? Surely you'd need to know, for example, which relation >>> and page number it is from

Re: [HACKERS] foreign key locks

2013-01-18 Thread Andres Freund
On 2013-01-18 16:01:18 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-01-18 15:37:47 -0500, Tom Lane wrote: > >> I doubt it ever came up before. What use is logging only the content of > >> a buffer page? Surely you'd need to know, for example, which relation > >> and page number it

Re: [HACKERS] foreign key locks

2013-01-18 Thread Tom Lane
Andres Freund writes: > On 2013-01-18 15:37:47 -0500, Tom Lane wrote: >> I doubt it ever came up before. What use is logging only the content of >> a buffer page? Surely you'd need to know, for example, which relation >> and page number it is from. > It only got to be a length of 0 because the

Re: [HACKERS] foreign key locks

2013-01-18 Thread Andres Freund
On 2013-01-18 15:37:47 -0500, Tom Lane wrote: > Alvaro Herrera writes: > > The reason is that there is an (unknown to me) rule that there must be > > some data not associated with a buffer: > > > /* > > * NOTE: We disallow len == 0 because it provides a useful bit of extra > > * err

Re: [HACKERS] foreign key locks

2013-01-18 Thread Tom Lane
Alvaro Herrera writes: > The reason is that there is an (unknown to me) rule that there must be > some data not associated with a buffer: > /* >* NOTE: We disallow len == 0 because it provides a useful bit of extra >* error checking in ReadRecord. This means that all caller

Re: [HACKERS] foreign key locks

2013-01-18 Thread Simon Riggs
On 18 January 2013 20:02, Alvaro Herrera wrote: > XLOG_HEAP2_LOCK_UPDATED Every xlog record needs to know where to put the block. Compare with XLOG_HEAP_LOCK xlrec.target.node = relation->rd_node; -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Supp

Re: [HACKERS] foreign key locks

2013-01-18 Thread Alvaro Herrera
Alvaro Herrera wrote: > Here's version 28 of this patch. The only substantive change here from > v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive > or LockTupleNoKeyExclusive, depending on whether the key columns are > being modified by the update. This needs to go through

Re: [HACKERS] foreign key locks

2013-01-11 Thread Andres Freund
On 2013-01-11 13:10:49 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > No, I was thinking about an update without triggers present. > > > > T0: CREATE TABLE tbl(id serial pk, name text unique, data text); > > T1: BEGIN; -- read committed > > T1: UPDATE tbl SET name = 'foo' WHERE name = 'b

Re: [HACKERS] foreign key locks

2013-01-11 Thread Alvaro Herrera
Andres Freund wrote: > No, I was thinking about an update without triggers present. > > T0: CREATE TABLE tbl(id serial pk, name text unique, data text); > T1: BEGIN; -- read committed > T1: UPDATE tbl SET name = 'foo' WHERE name = 'blarg'; /* key update of row id > = 1 */ > T2: BEGIN; -- read co

Re: [HACKERS] foreign key locks

2013-01-11 Thread Andres Freund
On 2013-01-11 12:11:47 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2013-01-10 18:00:40 -0300, Alvaro Herrera wrote: > > > Here's version 28 of this patch. The only substantive change here from > > > v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive > > > or Lo

Re: [HACKERS] foreign key locks

2013-01-11 Thread Alvaro Herrera
Andres Freund wrote: > On 2013-01-10 18:00:40 -0300, Alvaro Herrera wrote: > > Here's version 28 of this patch. The only substantive change here from > > v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive > > or LockTupleNoKeyExclusive, depending on whether the key columns ar

Re: [HACKERS] foreign key locks

2013-01-10 Thread Andres Freund
On 2013-01-10 18:00:40 -0300, Alvaro Herrera wrote: > Here's version 28 of this patch. The only substantive change here from > v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive > or LockTupleNoKeyExclusive, depending on whether the key columns are > being modified by the upd

Re: [HACKERS] foreign key locks

2012-12-22 Thread Andres Freund
On 2012-12-22 10:53:47 +0100, Erik Rijkers wrote: > On Thu, November 29, 2012 17:16, Alvaro Herrera wrote: > > Here it is. > > > > fklocks-26.patch.gz > > This applies today after removing, not only the infamous catversion.h chunk, > but also a file_fdw > chunk. (It's not really a problem.) > >

Re: [HACKERS] foreign key locks

2012-12-22 Thread Erik Rijkers
On Thu, November 29, 2012 17:16, Alvaro Herrera wrote: > Here it is. > > fklocks-26.patch.gz This applies today after removing, not only the infamous catversion.h chunk, but also a file_fdw chunk. (It's not really a problem.) I also wondered if anyone has any perl/python/bash programs handy th

Re: [HACKERS] foreign key locks - EvalPlanQual interactions

2012-11-29 Thread Andres Freund
On 2012-11-27 12:07:34 -0300, Alvaro Herrera wrote: > Alvaro Herrera wrote: > > Here's version 24. > > Old review emails still contains some items that didn't lead to any > changes. I tried to keep close track of those. To that list I add a > couple of things of my own. Here they are, for those

Re: [HACKERS] foreign key locks

2012-11-29 Thread Alvaro Herrera
Kevin Grittner wrote: > Kevin Grittner wrote: > > Alvaro Herrera wrote: > > > >> Here's version 24. > > > > This no longer applies after the rmgr rm_desc patch. > > I took a shot at merging those so I could review the patch against > HEAD; attached in hopes that it will be useful. > > Hopefully

Re: [HACKERS] foreign key locks

2012-11-28 Thread Kevin Grittner
Alvaro Herrera wrote: > Here's version 24. This no longer applies after the rmgr rm_desc patch. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] foreign key locks

2012-11-27 Thread Alvaro Herrera
Alvaro Herrera wrote: > Here's version 24. Old review emails still contains some items that didn't lead to any changes. I tried to keep close track of those. To that list I add a couple of things of my own. Here they are, for those following along. I welcome suggestions. - Fix the multixact c

Re: [HACKERS] foreign key locks

2012-11-20 Thread Andres Freund
On 2012-11-20 17:36:14 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > > - I find using a default: clause in switches with enum types where > > everything > > is expected to be handled like the following a bad idea, this way the > > compiler won't warn you if youve missed case's whic

Re: [HACKERS] foreign key locks

2012-11-20 Thread Alvaro Herrera
Andres Freund wrote: > - I find using a default: clause in switches with enum types where everything > is expected to be handled like the following a bad idea, this way the > compiler won't warn you if youve missed case's which makes > changing/extending code harder: > switch (

Re: [HACKERS] foreign key locks

2012-11-19 Thread Andres Freund
On 2012-11-14 13:27:26 -0300, Alvaro Herrera wrote: > Alvaro Herrera wrote: > > > > * In heap_lock_tuple's XMAX_IS_MULTI case > > > > > > [snip] > > > > > > why is it membermode > mode and not membermode >= mode? > > > > Uh, that's a bug. Fixed. As noticed in the comment above that snippet, > >

Re: [HACKERS] foreign key locks

2012-11-19 Thread Andres Freund
On 2012-11-14 13:27:26 -0300, Alvaro Herrera wrote: > Alvaro Herrera wrote: > > > > * In heap_lock_tuple's XMAX_IS_MULTI case > > > > > > [snip] > > > > > > why is it membermode > mode and not membermode >= mode? > > > > Uh, that's a bug. Fixed. As noticed in the comment above that snippet, > >

Re: [HACKERS] foreign key locks

2012-11-17 Thread Noah Misch
On Sat, Nov 17, 2012 at 05:07:18PM +0100, Andres Freund wrote: > > > I agree that tripling FOR SHARE cost is risky. Where is the added cost > > > concentrated? Perchance that multiple belies optimization opportunities. > > > > Good question, let me play a bit. > > Ok, I benchmarked around and fr

Re: [HACKERS] foreign key locks

2012-11-17 Thread Andres Freund
> > I agree that tripling FOR SHARE cost is risky. Where is the added cost > > concentrated? Perchance that multiple belies optimization opportunities. > > Good question, let me play a bit. Ok, I benchmarked around and from what I see there is no single easy target. The biggest culprits I could

Re: [HACKERS] foreign key locks

2012-11-17 Thread Noah Misch
On Sat, Nov 17, 2012 at 03:20:20PM +0100, Andres Freund wrote: > On 2012-11-16 22:31:51 -0500, Noah Misch wrote: > > On Fri, Nov 16, 2012 at 05:31:12PM +0100, Andres Freund wrote: > > > On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: > > > > Andres is on the verge of convincing me that we need

Re: [HACKERS] foreign key locks

2012-11-17 Thread Andres Freund
On 2012-11-16 22:31:51 -0500, Noah Misch wrote: > On Fri, Nov 16, 2012 at 05:31:12PM +0100, Andres Freund wrote: > > On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: > > > Andres is on the verge of convincing me that we need to support > > > singleton FOR SHARE without multixacts due to performa

Re: [HACKERS] foreign key locks

2012-11-16 Thread Noah Misch
On Fri, Nov 16, 2012 at 05:31:12PM +0100, Andres Freund wrote: > On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: > > Andres is on the verge of convincing me that we need to support > > singleton FOR SHARE without multixacts due to performance concerns. > > I don't really see any arguments agai

Re: [HACKERS] foreign key locks

2012-11-16 Thread Andres Freund
On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: > Noah Misch wrote: > > On Wed, Nov 14, 2012 at 01:27:26PM -0300, Alvaro Herrera wrote: > > > https://github.com/alvherre/postgres/commit/df2847e38198e99f57e52490e1e9391ebb70d770 > > > > > > (I don't think this is worth a v24 submission). > > > >

Re: [HACKERS] foreign key locks

2012-11-16 Thread Alvaro Herrera
Noah Misch wrote: > On Wed, Nov 14, 2012 at 01:27:26PM -0300, Alvaro Herrera wrote: > > https://github.com/alvherre/postgres/commit/df2847e38198e99f57e52490e1e9391ebb70d770 > > > > (I don't think this is worth a v24 submission). > > Are you aware of any defects in or unanswered questions of this

Re: [HACKERS] foreign key locks

2012-11-16 Thread Andres Freund
On 2012-11-14 13:27:26 -0300, Alvaro Herrera wrote: > Alvaro Herrera wrote: > > > > * In heap_lock_tuple's XMAX_IS_MULTI case > > > > > > [snip] > > > > > > why is it membermode > mode and not membermode >= mode? > > > > Uh, that's a bug. Fixed. As noticed in the comment above that snippet, > >

Re: [HACKERS] foreign key locks

2012-11-15 Thread Noah Misch
On Wed, Nov 14, 2012 at 01:27:26PM -0300, Alvaro Herrera wrote: > https://github.com/alvherre/postgres/commit/df2847e38198e99f57e52490e1e9391ebb70d770 > > (I don't think this is worth a v24 submission). Are you aware of any defects in or unanswered questions of this version that would stall your

Re: [HACKERS] foreign key locks

2012-11-14 Thread Alvaro Herrera
Alvaro Herrera wrote: > > * In heap_lock_tuple's XMAX_IS_MULTI case > > > > [snip] > > > > why is it membermode > mode and not membermode >= mode? > > Uh, that's a bug. Fixed. As noticed in the comment above that snippet, > there was a deadlock possible here. Maybe I should add a test to en

Re: [HACKERS] foreign key locks

2012-11-13 Thread Alvaro Herrera
Noah Misch wrote: > On Wed, Oct 31, 2012 at 05:22:10PM -0300, Alvaro Herrera wrote: > > Not really sure about the proposed syntax, but yes clearly we need some > > other syntax to mean "FOR NON KEY UPDATE". I would rather keep FOR > > UPDATE to mean what I currently call FOR KEY UPDATE. More pro

Re: [HACKERS] foreign key locks

2012-11-05 Thread Dimitri Fontaine
Alvaro Herrera writes: >> FOR NON KEY UPDATE >> FOR KEY UPDATE >> >> KEY is the default, so FOR UPDATE is a synonym of FOR KEY UPDATE > > Not really sure about the proposed syntax, but yes clearly we need some > other syntax to mean "FOR NON KEY UPDATE". I would rather keep FOR > UPDATE to mean

Re: [HACKERS] foreign key locks

2012-11-05 Thread Andres Freund
On Monday, November 05, 2012 02:37:15 PM Alvaro Herrera wrote: > FWIW I have gotten a lot of feedback about this patch, and since I don't > have time right now to produce an updated version, that I'm going to > close this as Returned with Feedback, and submit an updated version to > the upcoming co

Re: [HACKERS] foreign key locks

2012-11-05 Thread Alvaro Herrera
FWIW I have gotten a lot of feedback about this patch, and since I don't have time right now to produce an updated version, that I'm going to close this as Returned with Feedback, and submit an updated version to the upcoming commitfest. This patch still needs much more review -- for example, as f

Re: [HACKERS] foreign key locks

2012-10-31 Thread Noah Misch
On Wed, Oct 31, 2012 at 05:22:10PM -0300, Alvaro Herrera wrote: > Simon Riggs wrote: > > On 31 October 2012 19:35, Noah Misch wrote: > > > +1. I had not considered this angle during previous reviews, but I agree > > > with > > > Andres's position. Since this patch does not strengthen the strong

Re: [HACKERS] foreign key locks

2012-10-31 Thread Alvaro Herrera
Simon Riggs wrote: > On 31 October 2012 19:35, Noah Misch wrote: > > On Mon, Oct 29, 2012 at 08:18:33AM -0400, Kevin Grittner wrote: > >> Andres Freund wrote: > >> > The point is the introduction of a weaker lock level which can be > >> > used by the ri triggers. I don't see any imperative that t

Re: [HACKERS] foreign key locks

2012-10-31 Thread Simon Riggs
On 31 October 2012 19:35, Noah Misch wrote: > On Mon, Oct 29, 2012 at 08:18:33AM -0400, Kevin Grittner wrote: >> Andres Freund wrote: >> > On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: >> >> Andres Freund wrote: >> >> >>> * Is it ok to make FOR UPDATE somewhat weaker than before? In

Re: [HACKERS] foreign key locks

2012-10-31 Thread Noah Misch
On Mon, Oct 29, 2012 at 08:18:33AM -0400, Kevin Grittner wrote: > Andres Freund wrote: > > On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: > >> Andres Freund wrote: > > >>> * Is it ok to make FOR UPDATE somewhat weaker than before? In 9.2 > >>> and earlier you could be sure that if you

Re: [HACKERS] foreign key locks

2012-10-29 Thread Kevin Grittner
Andres Freund wrote: > On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: >> Andres Freund wrote: >>> * Is it ok to make FOR UPDATE somewhat weaker than before? In 9.2 >>> and earlier you could be sure that if you FOR UPDATE'ed a row you >>> could delete it. Unless I miss something now th

Re: [HACKERS] foreign key locks

2012-10-29 Thread Simon Riggs
On 29 October 2012 12:27, Andres Freund wrote: >> This sounds like we need a GUC or table-level default to control >> whether FOR UPDATE means FOR UPDATE or FOR DELETE > > I don't like adding a new guc for something that should be solveable with some > creative naming. If a new user doesn't get a

Re: [HACKERS] foreign key locks

2012-10-29 Thread Andres Freund
On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: > On 27 October 2012 00:06, Andres Freund wrote: > > On Thursday, October 18, 2012 09:58:20 PM Alvaro Herrera wrote: > >> Here is version 22 of this patch. This version contains fixes to issues > >> reported by Andres, as well as a rebase

Re: [HACKERS] foreign key locks

2012-10-28 Thread Simon Riggs
On 27 October 2012 00:06, Andres Freund wrote: > On Thursday, October 18, 2012 09:58:20 PM Alvaro Herrera wrote: >> Here is version 22 of this patch. This version contains fixes to issues >> reported by Andres, as well as a rebase to latest master. > > Ok, I now that pgconf.eu has ended I am star

Re: [HACKERS] foreign key locks

2012-10-26 Thread Andres Freund
On Thursday, October 18, 2012 09:58:20 PM Alvaro Herrera wrote: > Here is version 22 of this patch. This version contains fixes to issues > reported by Andres, as well as a rebase to latest master. Ok, I now that pgconf.eu has ended I am starting to do a real review: * Is it ok to make FOR UPDAT

Re: [HACKERS] foreign key locks

2012-10-19 Thread Noah Misch
On Thu, Oct 18, 2012 at 04:58:20PM -0300, Alvaro Herrera wrote: > Here is version 22 of this patch. This version contains fixes to issues > reported by Andres, as well as a rebase to latest master. I scanned this for obvious signs of work left to do. It contains numerous XXX and FIXME comments.

Re: [HACKERS] foreign key locks

2012-10-12 Thread Alvaro Herrera
Alvaro Herrera wrote: > Andres Freund wrote: > > * heap_lock_tuple with mode == LockTupleKeyShare && nowait looks like it > > would > > wait anyway in heap_lock_updated_tuple_rec > > Oops. I took a stab at fixing this. However, it is not easy. First you need a way to reproduce the problem, w

Re: [HACKERS] foreign key locks

2012-10-11 Thread Alvaro Herrera
Andres Freund wrote: > On Friday, August 31, 2012 06:59:51 AM Alvaro Herrera wrote: > > v21 is merged to latest master. > Ok, I am starting to look at this. > > (working with a git merge of alvherre/fklocks into todays master) > > In a very first pass as somebody who hasn't followed the discussio

Re: [HACKERS] foreign key locks

2012-10-11 Thread Andres Freund
On Friday, August 31, 2012 06:59:51 AM Alvaro Herrera wrote: > v21 is merged to latest master. Ok, I am starting to look at this. (working with a git merge of alvherre/fklocks into todays master) In a very first pass as somebody who hasn't followed the discussions in the past I took notice of th

Re: [HACKERS] foreign key locks

2012-08-24 Thread Robert Haas
On Wed, Aug 22, 2012 at 5:31 PM, Alvaro Herrera wrote: > Patch v19 contains some tweaks. Most notably, > > 1. if an Xid requests a lock A, and then a lock B which is stronger than > A, then record only lock B and forget lock A. This is important for > performance, because EvalPlanQual obtains Fo

Re: [HACKERS] foreign key locks

2012-07-05 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of vie jun 29 19:17:02 -0400 2012: > I was only testing migrating from an old version into patched, not > same-version upgrades. > > I think I see what's happening here. Okay, I have pushed the fix to github -- as I suspected, code-wise the fix was simple.

Re: [HACKERS] foreign key locks

2012-06-29 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of mié jun 27 08:40:58 -0400 2012: > Alvaro Herrera wrote: > > > here's fklocks v14, which also merges to new master as there were > > several other conflicts. It passes make installcheck-world now. > > Recent commits broke it again, so here's a rebased

Re: [HACKERS] foreign key locks

2012-06-27 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of mié jun 27 08:40:58 -0400 2012: > Alvaro Herrera wrote: > > > here's fklocks v14, which also merges to new master as there were > > several other conflicts. It passes make installcheck-world now. > > Recent commits broke it again, so here's a rebased

Re: [HACKERS] foreign key locks

2012-06-24 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of sáb jun 23 18:38:10 -0400 2012: > Alvaro Herrera wrote: > > > So here's the rebased version. > > I found a couple problems on `make check-world`. Attached is a patch > to fix one of them. The other is on pg_upgrade, pasted below. Ah, I mismerged pg

Re: [HACKERS] foreign key locks

2012-06-23 Thread Kevin Grittner
Alvaro Herrera wrote: > So here's the rebased version. I found a couple problems on `make check-world`. Attached is a patch to fix one of them. The other is on pg_upgrade, pasted below. + pg_upgrade -d /home/kevin/pg/master/contrib/pg_upgrade/tmp_check/data.old -D /home/kevin/pg/master/con

Re: [HACKERS] foreign key locks

2012-06-20 Thread Tom Lane
Jaime Casanova writes: > On Thu, Jun 14, 2012 at 11:41 AM, Alvaro Herrera > wrote: >> This is v12 of the foreign key locks patch. > Just noticed that this patch needs a rebase because of the refactoring > Tom did in ri_triggers.c Hold on a bit before you work on that code --- I've got one more

Re: [HACKERS] foreign key locks

2012-06-20 Thread Jaime Casanova
On Thu, Jun 14, 2012 at 11:41 AM, Alvaro Herrera wrote: > Hi, > > This is v12 of the foreign key locks patch. > Hi Álvaro, Just noticed that this patch needs a rebase because of the refactoring Tom did in ri_triggers.c -- Jaime Casanova         www.2ndQuadrant.com Professional PostgreSQL: Sopo

Re: [HACKERS] foreign key locks, 2nd attempt

2012-04-05 Thread Peter Geoghegan
On 25 March 2012 09:17, Simon Riggs wrote: > The main thing we're waiting on are the performance tests to confirm > the lack of regression. I have extensively benchmarked the latest revision of the patch (tpc-b.sql), which I pulled from Alvaro's github account. The benchmark was of the feature br

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-25 Thread Simon Riggs
On Sat, Mar 17, 2012 at 10:45 PM, Alvaro Herrera wrote: > Here is v11.  This version is mainly updated to add pg_upgrade support, > as discussed.  It also contains the README file that was posted earlier > (plus wording fixes per Bruce), a couple of bug fixes, and some comment > updates. The mai

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-17 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of mar mar 06 17:28:12 -0300 2012: > On Tue, Mar 6, 2012 at 7:39 PM, Alvaro Herrera > wrote: > > > We provide four levels of tuple locking strength: SELECT FOR KEY UPDATE is > > super-exclusive locking (used to delete tuples and more generally to update > > tu

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-17 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of mar mar 06 18:33:13 -0300 2012: > The lock modes are correct, appropriate and IMHO have meaningful > names. No redesign required here. > > Not sure about the naming of some of the flag bits however. Feel free to suggest improvements ... I've probably seen

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-17 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of lun mar 05 15:28:59 -0300 2012: > > On Mon, Feb 27, 2012 at 2:47 AM, Robert Haas wrote: > >> Regarding performance, the good thing about this patch is that if you > >> have an operation that used to block, it might now not block.  So maybe > >> multixact-r

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of vie mar 16 15:22:05 -0300 2012: > > On Fri, Mar 16, 2012 at 10:08:07AM -0400, Robert Haas wrote: > > On Thu, Mar 15, 2012 at 11:09 PM, Bruce Momjian wrote: > > > On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: > > >> On Mon, Mar 12, 2012 at 3

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Bruce Momjian
On Fri, Mar 16, 2012 at 10:08:07AM -0400, Robert Haas wrote: > On Thu, Mar 15, 2012 at 11:09 PM, Bruce Momjian wrote: > > On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: > >> On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs wrote: > >> > I agree with you that some worst case performance

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Bruce Momjian
On Fri, Mar 16, 2012 at 10:40:01AM -0300, Alvaro Herrera wrote: > > Excerpts from Alvaro Herrera's message of vie mar 16 10:36:11 -0300 2012: > > > > Now I am confused. Where do you see the word "hint" used by > > > HEAP_XMAX_EXCL_LOCK and HEAP_XMAX_SHARED_LOCK. These are tuple infomask > > > b

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Robert Haas
On Thu, Mar 15, 2012 at 11:09 PM, Bruce Momjian wrote: > On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: >> On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs wrote: >> > I agree with you that some worst case performance tests should be >> > done. Could you please say what you think the wo

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of vie mar 16 10:36:11 -0300 2012: > > Now I am confused. Where do you see the word "hint" used by > > HEAP_XMAX_EXCL_LOCK and HEAP_XMAX_SHARED_LOCK. These are tuple infomask > > bits, not hints, meaning they are not optional or there just for > > performa

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of vie mar 16 00:04:06 -0300 2012: > > On Tue, Mar 13, 2012 at 02:35:02PM -0300, Alvaro Herrera wrote: > > > > Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: > > > > > > On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrot

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Bruce Momjian
On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: > On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs wrote: > > I agree with you that some worst case performance tests should be > > done. Could you please say what you think the worst cases would be, so > > those can be tested? That would av

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Bruce Momjian
On Thu, Mar 15, 2012 at 11:04:06PM -0400, Bruce Momjian wrote: > On Tue, Mar 13, 2012 at 02:35:02PM -0300, Alvaro Herrera wrote: > > > > Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: > > > > > > On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: > > > > >

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Bruce Momjian
On Tue, Mar 13, 2012 at 02:35:02PM -0300, Alvaro Herrera wrote: > > Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: > > > > On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: > > > > When there is a single locker in a tuple, we can just store the locking >

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Noah Misch
On Thu, Mar 15, 2012 at 08:37:36PM -0400, Robert Haas wrote: > On Thu, Mar 15, 2012 at 5:07 PM, Simon Riggs wrote: > > On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas wrote: > >>> You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on > >>> mxid > >>> lookups, so I think something m

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue mar 15 21:37:36 -0300 2012: > > On Thu, Mar 15, 2012 at 5:07 PM, Simon Riggs wrote: > > On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas wrote: > >>> You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on > >>> mxid > >>> lookups, so I thi

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Robert Haas
On Thu, Mar 15, 2012 at 5:07 PM, Simon Riggs wrote: > On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas wrote: >>> You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on mxid >>> lookups, so I think something more sophisticated is needed to exercise that >>> cost.  Not sure what. >> >>

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 10:13 PM, Alvaro Herrera wrote: > > Excerpts from Simon Riggs's message of jue mar 15 19:04:41 -0300 2012: >> >> On Thu, Mar 15, 2012 at 9:54 PM, Alvaro Herrera >> wrote: >> > >> > Excerpts from Simon Riggs's message of jue mar 15 18:38:53 -0300 2012: >> >> On Thu, Mar 15,

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of jue mar 15 19:04:41 -0300 2012: > > On Thu, Mar 15, 2012 at 9:54 PM, Alvaro Herrera > wrote: > > > > Excerpts from Simon Riggs's message of jue mar 15 18:38:53 -0300 2012: > >> On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas wrote: > > > >> > But that would o

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 9:54 PM, Alvaro Herrera wrote: > > Excerpts from Simon Riggs's message of jue mar 15 18:38:53 -0300 2012: >> On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas wrote: > >> > But that would only make sense if >> > we thought that getting rid of the fsyncs would be more valuable t

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of jue mar 15 18:46:44 -0300 2012: > > On Thu, Mar 15, 2012 at 1:17 AM, Alvaro Herrera > wrote: > > > As things stand today > > Can I confirm where we are now? Is there another version of the patch > coming out soon? Yes, another version is coming soon. --

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of jue mar 15 18:38:53 -0300 2012: > On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas wrote: > > But that would only make sense if > > we thought that getting rid of the fsyncs would be more valuable than > > avoiding the blocking here, and I don't. > > You're ri

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 1:17 AM, Alvaro Herrera wrote: > As things stand today Can I confirm where we are now? Is there another version of the patch coming out soon? --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Support, Training & Services -- Se

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas wrote: > On Wed, Mar 14, 2012 at 9:17 PM, Alvaro Herrera > wrote: >>> > Agreed.  But speaking of that, why exactly do we fsync the multixact SLRU >>> > today? >>> >>> Good question.  So far, I can't think of a reason.  "nextMulti" is critical, >>> but

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 2:15 AM, Robert Haas wrote: > On Wed, Mar 14, 2012 at 6:10 PM, Noah Misch wrote: >>> Well, post-release, the cat is out of the bag: we'll be stuck with >>> this whether the performance characteristics are acceptable or not. >>> That's why we'd better be as sure as possible

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas wrote: >> You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on mxid >> lookups, so I think something more sophisticated is needed to exercise that >> cost.  Not sure what. > > I don't think HEAP_XMAX_COMMITTED is much help, because co

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Robert Haas
On Wed, Mar 14, 2012 at 9:17 PM, Alvaro Herrera wrote: >> > Agreed.  But speaking of that, why exactly do we fsync the multixact SLRU >> > today? >> >> Good question.  So far, I can't think of a reason.  "nextMulti" is critical, >> but we already fsync it with pg_control.  We could delete the oth

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Robert Haas
On Wed, Mar 14, 2012 at 6:10 PM, Noah Misch wrote: >> Well, post-release, the cat is out of the bag: we'll be stuck with >> this whether the performance characteristics are acceptable or not. >> That's why we'd better be as sure as possible before committing to >> this implementation that there's

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Alvaro Herrera
Excerpts from Noah Misch's message of mié mar 14 19:10:00 -0300 2012: > > On Wed, Mar 14, 2012 at 01:23:14PM -0400, Robert Haas wrote: > > On Tue, Mar 13, 2012 at 11:42 PM, Noah Misch wrote: > > > More often than that; each 2-member mxid takes 4 bytes in an offsets file > > > and > > > 10 bytes

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Noah Misch
On Wed, Mar 14, 2012 at 01:23:14PM -0400, Robert Haas wrote: > On Tue, Mar 13, 2012 at 11:42 PM, Noah Misch wrote: > > More often than that; each 2-member mxid takes 4 bytes in an offsets file > > and > > 10 bytes in a members file. ?So, more like one fsync per ~580 mxids. ?Note > > that we alrea

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Robert Haas
On Tue, Mar 13, 2012 at 11:42 PM, Noah Misch wrote: > More often than that; each 2-member mxid takes 4 bytes in an offsets file and > 10 bytes in a members file.  So, more like one fsync per ~580 mxids.  Note > that we already fsync the multixact SLRUs today, so any increase will arise > from the

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Noah Misch
On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: > On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs wrote: > > I agree with you that some worst case performance tests should be > > done. Could you please say what you think the worst cases would be, so > > those can be tested? That would av

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Robert Haas
On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs wrote: > I agree with you that some worst case performance tests should be > done. Could you please say what you think the worst cases would be, so > those can be tested? That would avoid wasting time or getting anything > backwards. I've thought about

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: > > On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: > > When there is a single locker in a tuple, we can just store the locking info > > in the tuple itself. We do this by storing the locker's Xid in XMAX,

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Robert Haas
On Mon, Mar 12, 2012 at 9:24 PM, Noah Misch wrote: > When we lock an update-in-progress row, we walk the t_ctid chain and lock all > descendant tuples.  They may all have uncommitted xmins.  This is essential to > ensure that the final outcome of the updating transaction does not affect > whether

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Bruce Momjian
On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: > Here's a first attempt at a README illustrating this. I intend this to > be placed in src/backend/access/heap/README.tuplock; the first three > paragraphs are stolen from the comment in heap_lock_tuple, so I'd remove > those from th

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Noah Misch
On Mon, Mar 12, 2012 at 01:28:11PM -0400, Robert Haas wrote: > I spent some time thinking about this over the weekend, and I have an > observation, and an idea. Here's the observation: I believe that > locking a tuple whose xmin is uncommitted is always a noop, because if > it's ever possible for

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Simon Riggs
On Mon, Mar 12, 2012 at 6:14 PM, Robert Haas wrote: > Considering that nobody's done any work to resolve the uncertainty > about whether the worst-case performance characteristics of this patch > are acceptable, and considering further that it was undergoing massive > code churn for more than a m

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Robert Haas
On Mon, Mar 12, 2012 at 1:50 PM, Simon Riggs wrote: > On Mon, Mar 12, 2012 at 5:28 PM, Robert Haas wrote: >> In other words, we'd entirely avoid needing to make mxacts crash-safe, >> and we'd avoid most of the extra SLRU lookups that the current >> implementation requires; they'd only be needed w

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Simon Riggs
On Mon, Mar 12, 2012 at 5:28 PM, Robert Haas wrote: > In other words, we'd entirely avoid needing to make mxacts crash-safe, > and we'd avoid most of the extra SLRU lookups that the current > implementation requires; they'd only be needed when (and for as long > as) the locked tuple was not yet a

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Robert Haas
On Sun, Feb 26, 2012 at 9:47 PM, Robert Haas wrote: >> Regarding performance, the good thing about this patch is that if you >> have an operation that used to block, it might now not block.  So maybe >> multixact-related operation is a bit slower than before, but if it >> allows you to continue op

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-07 Thread Simon Riggs
n Wed, Mar 7, 2012 at 11:37 AM, Gokulakannan Somasundaram wrote: >> >> Please explain in detail your idea of how it will work. > So we will take some kind of lock, which will stop such a happening. ... > May be someone can come up with better ideas than this. With respect, I don't call this a de

  1   2   >