Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-12 Thread Maxime Henrion
Mark Kirkwood wrote: Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-12 Thread Maxime Henrion
Mark Kirkwood wrote: Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Mark Kirkwood
Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim.

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Mark Kirkwood [EMAIL PROTECTED] writes: Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway [EMAIL PROTECTED] writes: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim. Correct. The

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 10:41:04PM +1200, Mark Kirkwood wrote: Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 10:23:42AM -0400, Tom Lane wrote: Mark Kirkwood [EMAIL PROTECTED] writes: Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway [EMAIL PROTECTED] writes: On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is that Postgres never has more than one process waiting on any particular SysV semaphore, and so the problem doesn't really

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway [EMAIL PROTECTED] writes: I have not studied the exact code path, but there are indeed multiple wakeups happening from the semaphore code (as many as the number of active postgresql processes). It is easy to instrument sleepq_broadcast() and log them when they happen. There are

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Andrew - Supernews
On 2007-04-10, Tom Lane [EMAIL PROTECTED] wrote: Kris Kennaway [EMAIL PROTECTED] writes: I have not studied the exact code path, but there are indeed multiple wakeups happening from the semaphore code (as many as the number of active postgresql processes). It is easy to instrument

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway [EMAIL PROTECTED] writes: On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: Anyway I'd be interested to know what the test case is, and which PG version you were testing. I used 8.2 (and some older version when I first noticed it a year ago) and either sysbench or

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 03:52:00PM -0400, Tom Lane wrote: Kris Kennaway [EMAIL PROTECTED] writes: On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is that Postgres never has more than one process waiting on any

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 06:26:37PM -0400, Tom Lane wrote: Kris Kennaway [EMAIL PROTECTED] writes: On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: Anyway I'd be interested to know what the test case is, and which PG version you were testing. I used 8.2 (and some older version

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: Kris Kennaway [EMAIL PROTECTED] writes: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: Kris Kennaway [EMAIL PROTECTED] writes: I have not studied the exact code path, but there are indeed multiple wakeups happening from the semaphore code (as many as the number of active postgresql processes). It is easy to instrument