Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-12-07 Thread Michael Paquier
On Wed, Dec 3, 2014 at 4:03 PM, Michael Paquier michael.paqu...@gmail.com wrote: Ping? This patch is in a stale state for a couple of weeks and still marked as waiting on author for this CF. Marked as returned with feedback. -- Michael -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-12-02 Thread Michael Paquier
On Tue, Nov 18, 2014 at 12:33 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Nov 17, 2014 at 10:31 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-17 10:21:04 -0500, Robert Haas wrote: Andres, where are we with this patch? 1. You're going to commit it, but haven't gotten

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-11-17 Thread Robert Haas
On Sat, Oct 25, 2014 at 1:50 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Oct 24, 2014 at 4:05 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-24 15:59:30 +0530, Amit Kapila wrote: and w.r.t performance it can lead extra function call, few checks and I think in

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-11-17 Thread Andres Freund
On 2014-11-17 10:21:04 -0500, Robert Haas wrote: Andres, where are we with this patch? 1. You're going to commit it, but haven't gotten around to it yet. 2. You're going to modify it some more and repost, but haven't gotten around to it yet. 3. You're willing to see it modified if

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-11-17 Thread Robert Haas
On Mon, Nov 17, 2014 at 10:31 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-17 10:21:04 -0500, Robert Haas wrote: Andres, where are we with this patch? 1. You're going to commit it, but haven't gotten around to it yet. 2. You're going to modify it some more and repost, but

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-24 Thread Amit Kapila
On Wed, Oct 22, 2014 at 7:12 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-22 13:32:07 +0530, Amit Kapila wrote: On Tue, Oct 21, 2014 at 7:56 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund and...@2ndquadrant.com wrote:

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-24 Thread Amit Kapila
On Wed, Oct 22, 2014 at 8:04 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-21 19:56:05 +0530, Amit Kapila wrote: On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund and...@2ndquadrant.com wrote: spin_delay_count gives how much delay has happened to acquire spinlock which when

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-24 Thread Andres Freund
On 2014-10-24 15:59:30 +0530, Amit Kapila wrote: and w.r.t performance it can lead extra function call, few checks and I think in some cases even can acquire/release spinlock. I fail to see how that could be the case. Won't it happen incase first backend sets releaseOK to true and

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-22 Thread Amit Kapila
On Tue, Oct 21, 2014 at 7:56 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: Today, I have verified all previous comments raised by me and looked at new code and below

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-22 Thread Andres Freund
On 2014-10-22 13:32:07 +0530, Amit Kapila wrote: On Tue, Oct 21, 2014 at 7:56 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: Today, I have verified all

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-22 Thread Andres Freund
On 2014-10-21 19:56:05 +0530, Amit Kapila wrote: On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: 2. LWLockWakeup() { .. #ifdef LWLOCK_STATS lwstats-spin_delay_count += SpinLockAcquire(lock-mutex);

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-21 Thread Amit Kapila
On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: 2. LWLockWakeup() { .. #ifdef LWLOCK_STATS lwstats-spin_delay_count += SpinLockAcquire(lock-mutex); #else SpinLockAcquire(lock-mutex); #endif .. }

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-10 Thread Andres Freund
On 2014-10-08 20:07:35 -0400, Robert Haas wrote: On Wed, Oct 8, 2014 at 2:04 PM, Andres Freund and...@2ndquadrant.com wrote: So, what makes it work for me (among other unrelated stuff) seems to be the following in .gdbinit, defineing away some things that gdb doesn't handle: macro define

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: 2. LWLockWakeup() { .. #ifdef LWLOCK_STATS lwstats-spin_delay_count += SpinLockAcquire(lock-mutex); #else SpinLockAcquire(lock-mutex); #endif .. } Earlier while releasing lock, we don't count it towards LWLock stats spin_delay_count.

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-10-08 14:47:44 +0200, Andres Freund wrote: On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: 5. LWLockWakeup() { .. dlist_foreach_modify(iter, (dlist_head *) wakeup) { PGPROC *waiter = dlist_container(PGPROC, lwWaitLink, iter.cur); LOG_LWDEBUG(LWLockRelease, l, mode,

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Robert Haas
On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund and...@2ndquadrant.com wrote: I don't see that as being relevant. The difference is an instruction or two - in the slow path we'll enter the kernel and sleep. This doesn't matter in comparison. And the code is *so* much more readable. I find the

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-10-08 13:13:33 -0400, Robert Haas wrote: On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund and...@2ndquadrant.com wrote: I don't see that as being relevant. The difference is an instruction or two - in the slow path we'll enter the kernel and sleep. This doesn't matter in comparison.

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Alvaro Herrera
Robert Haas wrote: On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund and...@2ndquadrant.com wrote: I don't see that as being relevant. The difference is an instruction or two - in the slow path we'll enter the kernel and sleep. This doesn't matter in comparison. And the code is *so* much more

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-10-08 14:23:44 -0300, Alvaro Herrera wrote: Robert Haas wrote: On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund and...@2ndquadrant.com wrote: I don't see that as being relevant. The difference is an instruction or two - in the slow path we'll enter the kernel and sleep. This

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Robert Haas
On Wed, Oct 8, 2014 at 2:04 PM, Andres Freund and...@2ndquadrant.com wrote: So, what makes it work for me (among other unrelated stuff) seems to be the following in .gdbinit, defineing away some things that gdb doesn't handle: macro define __builtin_offsetof(T, F) ((int) (((T *) 0)-F)) macro

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-25 Thread Amit Kapila
On Tue, Jun 24, 2014 at 9:33 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: 7. LWLockWaitForVar() { .. /* * Add myself to wait queue. Note that this

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Tue, Jun 17, 2014 at 8:56 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-17 20:47:51 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund and...@2ndquadrant.com wrote: You have followed it pretty well as far as I can understand from your replies, as

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Andres Freund
On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 8:56 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-17 20:47:51 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund and...@2ndquadrant.com wrote: You have followed it pretty

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 8:56 PM, Andres Freund and...@2ndquadrant.com wrote: 2. LWLockAcquireCommon() { .. if (!LWLockDequeueSelf(l)) { /* * Somebody

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: 12. #ifdef LWLOCK_DEBUG lock-owner = MyProc; #endif Shouldn't it be reset in LWLockRelease? That's actually intentional. It's quite useful to know the last

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Amit Kapila
On Fri, May 23, 2014 at 10:01 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund and...@2ndquadrant.com wrote: I've pushed a rebased version of the patchset to http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git branch rwlock

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Andres Freund
On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: On Fri, May 23, 2014 at 10:01 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund and...@2ndquadrant.com wrote: I've pushed a rebased version of the patchset to

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Amit Kapila
On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: On Fri, May 23, 2014 at 10:01 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund and...@2ndquadrant.com wrote:

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Andres Freund
On 2014-06-17 18:01:58 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund and...@2ndquadrant.com On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: 2. Handling of potentialy_spurious case seems to be pending in LWLock functions like LWLockAcquireCommon().

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Amit Kapila
On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-17 18:01:58 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund and...@2ndquadrant.com On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: 2. Handling of potentialy_spurious

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Andres Freund
On 2014-06-17 20:47:51 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-17 18:01:58 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund and...@2ndquadrant.com On 2014-06-17 12:41:26 +0530, Amit Kapila

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-05-23 Thread Amit Kapila
On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund and...@2ndquadrant.com wrote: I've pushed a rebased version of the patchset to http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git branch rwlock contention. 220b34331f77effdb46798ddd7cca0cffc1b2858 actually was the small problem,

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-10 Thread Heikki Linnakangas
On 01/31/2014 11:54 AM, Andres Freund wrote: Hi, On 2014-01-28 21:27:29 -0800, Peter Geoghegan wrote: On Fri, Nov 15, 2013 at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: 1) I've added an abstracted atomic ops implementation. Needs a fair amount of work, also submitted as a

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-04 Thread Andres Freund
On 2014-02-03 17:51:20 -0800, Peter Geoghegan wrote: On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: Here are the results of a benchmark on Nathan Boley's 64-core, 4 socket server:

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-03 Thread Jeff Janes
On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund and...@2ndquadrant.comwrote: Some background: The setups that triggered me into working on the patchset didn't really have a pgbench like workload, the individual queries were/are more complicated even though it's still an high throughput OLTP

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-03 Thread Peter Geoghegan
On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund and...@2ndquadrant.com wrote: The changed algorithm for lwlock imo is an *algorithmic* improvement, not one for a particular architecture. The advantage being that locking a lwlock which is primarily taken in shared mode will never need need to

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-03 Thread Peter Geoghegan
On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: Here are the results of a benchmark on Nathan Boley's 64-core, 4 socket server: http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/amd-4-socket-rwlocks/

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-02 Thread Andres Freund
Hi, On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: Here are the results of a benchmark on Nathan Boley's 64-core, 4 socket server: http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/amd-4-socket-rwlocks/ That's interesting. The maximum number of what you see here (~293125)

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Peter Geoghegan
I thought I'd try out what I was in an immediate position to do without having access to dedicated multi-socket hardware: A benchmark on AWS. This was a c3.8xlarge instance, which are reportedly backed by Intel Xeon E5-2680 processors. Since the Intel ARK website reports that these CPUs have 16

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Andres Freund
On 2014-01-31 17:52:58 -0800, Peter Geoghegan wrote: On Fri, Jan 31, 2014 at 1:54 AM, Andres Freund and...@2ndquadrant.com wrote: I plan to split the atomics patch into smaller chunks before reposting. Imo the Convert the PGPROC-lwWaitLink list into a dlist instead of open coding it. is

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Peter Geoghegan
On Sat, Feb 1, 2014 at 4:57 AM, Andres Freund and...@2ndquadrant.com wrote: I'm looking at alternative options, because this is not terribly helpful. With those big caveats in mind, consider the results of the benchmark, which show the patch performing somewhat worse than the master baseline

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Andres Freund
On 2014-02-01 13:40:20 -0800, Peter Geoghegan wrote: On Sat, Feb 1, 2014 at 4:57 AM, Andres Freund and...@2ndquadrant.com wrote: I'm looking at alternative options, because this is not terribly helpful. With those big caveats in mind, consider the results of the benchmark, which show the

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Peter Geoghegan
On Sat, Feb 1, 2014 at 1:41 PM, Andres Freund and...@2ndquadrant.com wrote: However, I tested the most recent revision from your git remote on the AWS instance. But that was before my fix, right. Except you managed to timetravel :) Heh, okay. So Nathan Boley has generously made available a

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-01-31 Thread Andres Freund
Hi, On 2014-01-28 21:27:29 -0800, Peter Geoghegan wrote: On Fri, Nov 15, 2013 at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: 1) I've added an abstracted atomic ops implementation. Needs a fair amount of work, also submitted as a separate CF entry. (Patch 1 2) Commit

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-01-31 Thread Peter Geoghegan
On Fri, Jan 31, 2014 at 1:54 AM, Andres Freund and...@2ndquadrant.com wrote: I plan to split the atomics patch into smaller chunks before reposting. Imo the Convert the PGPROC-lwWaitLink list into a dlist instead of open coding it. is worth being applied independently from the rest of the

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-01-28 Thread Peter Geoghegan
On Fri, Nov 15, 2013 at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: 1) I've added an abstracted atomic ops implementation. Needs a fair amount of work, also submitted as a separate CF entry. (Patch 1 2) Commit 220b34331f77effdb46798ddd7cca0cffc1b2858 caused bitrot when applying

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2013-12-21 Thread Peter Eisentraut
This patch didn't make it out of the 2013-11 commit fest. You should move it to the next commit fest (probably with an updated patch) before January 15th. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2013-11-16 Thread Peter Eisentraut
On Fri, 2013-11-15 at 20:47 +0100, Andres Freund wrote: So, here's the next version of this patchset: The 0002 patch contains non-ASCII, non-UTF8 characters: 0002-Very-basic-atomic-ops-implementation.patch: line 609, char 1, byte offset 43: invalid UTF-8 code Please change that to ASCII, or

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2013-11-15 Thread Andres Freund
On 2013-11-15 20:47:26 +0100, Andres Freund wrote: Hi, On 2013-09-27 00:55:45 +0200, Andres Freund wrote: So what's todo? The file header tells us: * - revive pure-spinlock implementation * - abstract away atomic ops, we really only need a few. * - CAS * - LOCK XADD * -