Re: [HACKERS] FlexLocks

2011-12-02 Thread Kevin Grittner
Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: The extraWaits code still looks like black magic to me [explanation of the extraWaits behavior] Thanks. I'll spend some time reviewing this part.

Re: [HACKERS] FlexLocks

2011-12-01 Thread Robert Haas
On Wed, Nov 30, 2011 at 7:01 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: OK.  There are a few things I found in this pass which missed in the last.  One contrib module was missed, I found another typo in a comment, and I think we can

Re: [HACKERS] FlexLocks

2011-11-30 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: Why is it OK to drop these lines from the else condition in ProcArrayEndTransaction()?: /* must be cleared with xid/xmin: */ proc-vacuumFlags = ~PROC_VACUUM_STATE_MASK; It's probably

Re: [HACKERS] FlexLocks

2011-11-30 Thread Kevin Grittner
Kevin Grittner kevin.gritt...@wicourts.gov wrote: OK. There are a few things I found in this pass which missed in the last. One contrib module was missed, I found another typo in a comment, and I think we can reduce the include files a bit. Rather than describe it, I'm attaching a patch

Re: [HACKERS] FlexLocks

2011-11-23 Thread Kevin Grittner
Kevin Grittner wrote: Robert Haas wrote: Updated patches attached. I have to admit I don't have my head around the extraWaits issue, so I can't personally vouch for that code, although I have no reason to doubt it, either. Everything else was something that I at least *think* I

Re: [HACKERS] FlexLocks

2011-11-22 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Updated patches attached. I've gotten through several days of performance tests for this pair of related patches, with results posted on a separate thread. I'll link those in to the CF application shortly. To summarize the other (fairly long) thread

Re: [HACKERS] FlexLocks

2011-11-21 Thread Bruce Momjian
Robert Haas wrote: On Wed, Nov 16, 2011 at 12:25 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: We could alternatively change one or the other of them to be a struct with one member, but I think the cure might be worse than the disease. ?By my count, we are talking about saving

Re: [HACKERS] FlexLocks

2011-11-18 Thread Pavan Deolasee
On Thu, Nov 17, 2011 at 10:19 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Thu, Nov 17, 2011 at 10:01 AM, Robert Haas robertmh...@gmail.com wrote: I am not convinced that that's a better API.  I mean, consider something like this:    /*     * OK, let's do it.  First let other

Re: [HACKERS] FlexLocks

2011-11-18 Thread Robert Haas
On Fri, Nov 18, 2011 at 6:26 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: My apologies for hijacking the thread, but the work seems quite related, so I thought I should post here instead of starting a new thread. Here is a WIP patch based on the idea of having a shared Q. A process

Re: [HACKERS] FlexLocks

2011-11-18 Thread Pavan Deolasee
On Fri, Nov 18, 2011 at 10:29 PM, Robert Haas robertmh...@gmail.com wrote: So the upside and downside of this approach is that it modifies the existing LWLock implementation rather than allowing multiple lock implementations to exist side-by-side.  That means every LWLock in the system has

Re: [HACKERS] FlexLocks

2011-11-16 Thread Simon Riggs
On Tue, Nov 15, 2011 at 8:33 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: I'm not really enthused by the idea of completely rewriting lwlocks for this. Seems like specialised code is likely to be best, as well as having less collateral damage. Well, the problem that I have with

Re: [HACKERS] FlexLocks

2011-11-16 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: I just don't see the reason to do a global search and replace on the lwlock name I was going to review further before commenting on that, but since it has now come up -- it seems odd that a source file which uses only LW locks needs to change so much

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 10:26 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: For example, if these two macros were defined, predicate.c wouldn't have needed any modifications, and I suspect that is true of many other files (although possibly needing a few other macros): #define

Re: [HACKERS] FlexLocks

2011-11-16 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Simon Riggs si...@2ndquadrant.com wrote: I just don't see the reason to do a global search and replace on the lwlock name I was going to review further before commenting on that, but since it has now come up -- it seems odd that a source

Re: [HACKERS] FlexLocks

2011-11-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, it would certainly be easy enough to add those macros, and I'm not necessarily opposed to it, but I fear it could end up being a bit confusing in the long run. If we adopt this infrastructure, then I expect knowledge of different types of

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 10:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, it would certainly be easy enough to add those macros, and I'm not necessarily opposed to it, but I fear it could end up being a bit confusing in the long run.  If we adopt this

Re: [HACKERS] FlexLocks

2011-11-16 Thread Greg Stark
On Wed, Nov 16, 2011 at 3:41 PM, Robert Haas robertmh...@gmail.com wrote:  Now, you're always going to use LWLockAcquire() and LWLockRelease() to acquire and release an LWLock, but a FlexLockId isn't guaranteed to be an LWLockId - any given value might also refer to a FlexLock of some other

Re: [HACKERS] FlexLocks

2011-11-16 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Now maybe there is some better way to do this, but at the moment, I'm not seeing it. If we call them all LWLocks, but only some of them support LWLockAcquire(), then that's going to be pretty weird. Is there any way to typedef our way out of it,

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 11:14 AM, Greg Stark st...@mit.edu wrote: On Wed, Nov 16, 2011 at 3:41 PM, Robert Haas robertmh...@gmail.com wrote:  Now, you're always going to use LWLockAcquire() and LWLockRelease() to acquire and release an LWLock, but a FlexLockId isn't guaranteed to be an LWLockId

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 11:17 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: Now maybe there is some better way to do this, but at the moment, I'm not seeing it.  If we call them all LWLocks, but only some of them support LWLockAcquire(), then

Re: [HACKERS] FlexLocks

2011-11-16 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: Is there any way to typedef our way out of it [?] Well, if we just say: typedef FlexLockId LWLockId; ...that's about equivalent to the #define from the compiler's point of view. Bummer -- I

Re: [HACKERS] FlexLocks

2011-11-16 Thread Pavan Deolasee
On Tue, Nov 15, 2011 at 7:20 PM, Robert Haas robertmh...@gmail.com wrote:  The lower layer I called FlexLocks, and it's designed to allow a variety of locking implementations to be built on top of it and reuse as much of the basic infrastructure as I could figure out how to make reusable

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 11:16 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Tue, Nov 15, 2011 at 7:20 PM, Robert Haas robertmh...@gmail.com wrote:  The lower layer I called FlexLocks, and it's designed to allow a variety of locking implementations to be built on top of it and reuse as

Re: [HACKERS] FlexLocks

2011-11-16 Thread Pavan Deolasee
On Thu, Nov 17, 2011 at 10:01 AM, Robert Haas robertmh...@gmail.com wrote: I am not convinced that that's a better API.  I mean, consider something like this:    /*     * OK, let's do it.  First let other backends know I'm in ANALYZE.     */    LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);

Re: [HACKERS] FlexLocks

2011-11-15 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: I'm not necessarily saying that any of these particular things are what we want to do, just throwing out the idea that we may want a variety of lock types that are similar to lightweight locks but with subtly different behavior, yet with common

Re: [HACKERS] FlexLocks

2011-11-15 Thread Simon Riggs
On Tue, Nov 15, 2011 at 1:50 PM, Robert Haas robertmh...@gmail.com wrote: It basically works like a regular LWLock, except that it has a special operation to optimize ProcArrayEndTransaction().  In the uncontended case, instead of acquiring and releasing the lock, it just grabs the lock,

Re: [HACKERS] FlexLocks

2011-11-15 Thread Robert Haas
On Tue, Nov 15, 2011 at 1:40 PM, Simon Riggs si...@2ndquadrant.com wrote: Which is the same locking avoidance technique we already use for sync rep and for the new group commit patch. Yep... I've been saying for some time that we should use the same technique for ProcArray and clog also, so

Re: [HACKERS] FlexLocks

2011-11-15 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar nov 15 17:16:31 -0300 2011: On Tue, Nov 15, 2011 at 1:40 PM, Simon Riggs si...@2ndquadrant.com wrote: Which is the same locking avoidance technique we already use for sync rep and for the new group commit patch. Yep... I've been saying for

Re: [HACKERS] FlexLocks

2011-11-15 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: As Kevin says nearby it's likely that we could find some way to rewrite the SLRU (clog and such) locking protocol using these new things too. Yeah, I really meant all SLRU, not just clog. And having seen what Robert has done here, I'm kinda

Re: [HACKERS] FlexLocks

2011-11-15 Thread Robert Haas
On Tue, Nov 15, 2011 at 3:47 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Alvaro Herrera alvhe...@commandprompt.com wrote: As Kevin says nearby it's likely that we could find some way to rewrite the SLRU (clog and such) locking protocol using these new things too. Yeah, I really

Re: [HACKERS] FlexLocks

2011-11-15 Thread Dan Ports
On Tue, Nov 15, 2011 at 10:55:49AM -0600, Kevin Grittner wrote: And I would be surprised if some creative thinking didn't yield a far better FL scheme for SSI than we can manage with existing LW locks. One place I could see it being useful is for SerializableFinishedListLock, which protects