Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Tom Lane
Simon Riggs writes: > OK, I agree. I still think this is about solving a non-problem. > The implementation I would prefer would be to put the check in > START_CRIT_SECTION(); so we actually fail before we enter the section. ... and this is proposing adding significant overhead to solve a non-pr

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Simon Riggs
On 11 May 2012 17:48, Heikki Linnakangas wrote: > On 11.05.2012 18:18, Simon Riggs wrote: >> >> On 11 May 2012 15:14, Heikki Linnakangas >>  wrote: >>> >>> On 11.05.2012 16:56, Simon Riggs wrote: On 11 May 2012 11:07, Heikki Linnakangas    wrote: > I wonder if we sho

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Heikki Linnakangas
On 11.05.2012 18:18, Simon Riggs wrote: On 11 May 2012 15:14, Heikki Linnakangas wrote: On 11.05.2012 16:56, Simon Riggs wrote: On 11 May 2012 11:07, Heikki Linnakangas wrote: I wonder if we should reserve a few of the lwlock "slots" for critical sections, to make this less likely to h

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Simon Riggs
On 11 May 2012 15:14, Heikki Linnakangas wrote: > On 11.05.2012 16:56, Simon Riggs wrote: >> >> On 11 May 2012 11:07, Heikki Linnakangas >>  wrote: >> >>> I wonder if we should reserve a few of the lwlock "slots" for critical >>> sections, to make this less likely to happen. Not only in this case

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Robert Haas
On Fri, May 11, 2012 at 10:19 AM, Tom Lane wrote: > Heikki Linnakangas writes: >> On 11.05.2012 16:52, Tom Lane wrote: >>> IMO, no part of the system should ever get within an order of magnitude >>> of holding 100 LWLocks concurrently. > >> I agree we should never get anywhere near that limit. Bu

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Tom Lane
Heikki Linnakangas writes: > On 11.05.2012 16:52, Tom Lane wrote: >> IMO, no part of the system should ever get within an order of magnitude >> of holding 100 LWLocks concurrently. > I agree we should never get anywhere near that limit. But if we do - > because of another bug like this one - it

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Heikki Linnakangas
On 11.05.2012 16:56, Simon Riggs wrote: On 11 May 2012 11:07, Heikki Linnakangas wrote: I wonder if we should reserve a few of the lwlock "slots" for critical sections, to make this less likely to happen. Not only in this case, but in general. We haven't seen this problem often, but it would

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Heikki Linnakangas
On 11.05.2012 16:52, Tom Lane wrote: Heikki Linnakangas writes: I wonder if we should reserve a few of the lwlock "slots" for critical sections, to make this less likely to happen. Not only in this case, but in general. We haven't seen this problem often, but it would be quite trivial to reserv

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Simon Riggs
On 11 May 2012 11:07, Heikki Linnakangas wrote: > I wonder if we should reserve a few of the lwlock "slots" for critical > sections, to make this less likely to happen. Not only in this case, but in > general. We haven't seen this problem often, but it would be quite trivial > to reserve a few sl

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Tom Lane
Heikki Linnakangas writes: > I wonder if we should reserve a few of the lwlock "slots" for critical > sections, to make this less likely to happen. Not only in this case, but > in general. We haven't seen this problem often, but it would be quite > trivial to reserve a few slots. I'm against t

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Heikki Linnakangas
On 08.05.2012 04:15, Ryan Kelly wrote: On Mon, May 07, 2012 at 05:31:40PM +0100, tom Tom wrote: Nope, this was just a benchmark script that caused this, any sane person would use an intbig index instead I guess. A better error message would be nice though, I was pretty confused when this happene

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-11 Thread Heikki Linnakangas
On 07.05.2012 18:51, Tom Lane wrote: Heikki Linnakangas writes: We could rearrange the page splitting algorithm to release locks earlier, before traversing to the next parent level. That seems like a good idea just on concurrency grounds; I'm worried about both the performance implications an

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-08 Thread Heikki Linnakangas
On 07.05.2012 18:51, Tom Lane wrote: Heikki Linnakangas writes: I wrote a quick patch to do that, and with the patch the index build finished - but it took hours. And the index was 10GB in size, where the heap is just 12 MB, and searches using the index take ages. Hm, is the example exploitin

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-07 Thread Ryan Kelly
On Mon, May 07, 2012 at 05:31:40PM +0100, tom Tom wrote: > Nope, this was just a benchmark script that caused this, any sane person > would use an intbig index instead I guess. A better error message would be > nice though, I was pretty confused when this happened. > > This can also bring down pos

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-07 Thread tom Tom
Nope, this was just a benchmark script that caused this, any sane person would use an intbig index instead I guess. A better error message would be nice though, I was pretty confused when this happened. This can also bring down postgresql - it happens occasionally and causes the server to terminat

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-07 Thread Tom Lane
Heikki Linnakangas writes: > We could rearrange the page splitting algorithm to release locks > earlier, before traversing to the next parent level. That seems like a good idea just on concurrency grounds; I'm worried about both the performance implications and the risk of deadlock. > I wrote a

Re: [BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-07 Thread Heikki Linnakangas
On 05.05.2012 22:49, t...@tomforb.es wrote: The following bug has been logged on the website: Bug reference: 6629 Logged by: Tom Forbes Email address: t...@tomforb.es PostgreSQL version: 9.1.3 Operating system: Windows 7 64bit Description: On a test database with 10,000 row

[BUGS] BUG #6629: Creating a gist index fails with "too many LWLocks taken"

2012-05-06 Thread tom
The following bug has been logged on the website: Bug reference: 6629 Logged by: Tom Forbes Email address: t...@tomforb.es PostgreSQL version: 9.1.3 Operating system: Windows 7 64bit Description: On a test database with 10,000 rows, each containing a array of 50 uniqu