Re: [PERFORM] Allow sorts to use more available memory

2011-10-05 Thread Bruce Momjian
Stephen Frost wrote: -- Start of PGP signed section. > * Robert Schnabel (schnab...@missouri.edu) wrote: > > And getting back to the to-do list entry and reading the related > > posts, it appears that even if you could set work_mem that high it > > would only use 2GB anyway. I guess that was the s

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Stephen Frost
* Robert Schnabel (schnab...@missouri.edu) wrote: > And getting back to the to-do list entry and reading the related > posts, it appears that even if you could set work_mem that high it > would only use 2GB anyway. I guess that was the second part of my > question. Is that true? Errr, and to get

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Stephen Frost
* Robert Schnabel (schnab...@missouri.edu) wrote: > And getting back to the to-do list entry and reading the related > posts, it appears that even if you could set work_mem that high it > would only use 2GB anyway. I guess that was the second part of my > question. Is that true? Yes and no. wor

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Tom Lane
Robert Schnabel writes: > On 9/12/2011 3:58 PM, Scott Marlowe wrote: >> On Mon, Sep 12, 2011 at 11:33 AM, Robert Schnabel >> wrote: >>> The recent "data warehouse" thread made me think about how I use work_mem >>> for some of my big queries. So I tried SET work_mem = '4GB' for a session >>> and

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Robert Schnabel
On 9/12/2011 1:57 PM, Andy Colson wrote: On 9/12/2011 1:22 PM, Robert Schnabel wrote: On 9/12/2011 12:57 PM, Shaun Thomas wrote: On 09/12/2011 12:47 PM, Andy Colson wrote: work_mem is not the total a query can use. I believe each step can use that much, and each backend can use it for multip

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Robert Schnabel
On 9/12/2011 3:58 PM, Scott Marlowe wrote: On Mon, Sep 12, 2011 at 11:33 AM, Robert Schnabel wrote: The recent "data warehouse" thread made me think about how I use work_mem for some of my big queries. So I tried SET work_mem = '4GB' for a session and got ERROR: 4194304 is outside the valid

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Scott Marlowe
On Mon, Sep 12, 2011 at 11:33 AM, Robert Schnabel wrote: > The recent "data warehouse" thread made me think about how I use work_mem > for some of my big queries.  So I tried SET work_mem = '4GB' for a session > and got > > ERROR: 4194304 is outside the valid range for parameter "work_mem" (64 ..

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread pasman pasmański
I think , you may add a ramdisk as tablespace for temporary tables. This should work similar to bigger work_mem. 2011/9/12, Robert Schnabel : > > On 9/12/2011 12:57 PM, Shaun Thomas wrote: >> On 09/12/2011 12:47 PM, Andy Colson wrote: >> >>> work_mem is not the total a query can use. I believe eac

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Andy Colson
On 9/12/2011 1:22 PM, Robert Schnabel wrote: On 9/12/2011 12:57 PM, Shaun Thomas wrote: On 09/12/2011 12:47 PM, Andy Colson wrote: work_mem is not the total a query can use. I believe each step can use that much, and each backend can use it for multiple bits. So if you had two backends, each

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Andy Colson
On 9/12/2011 1:22 PM, Robert Schnabel wrote: On 9/12/2011 12:57 PM, Shaun Thomas wrote: On 09/12/2011 12:47 PM, Andy Colson wrote: work_mem is not the total a query can use. I believe each step can use that much, and each backend can use it for multiple bits. So if you had two backends, each

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Robert Schnabel
On 9/12/2011 12:57 PM, Shaun Thomas wrote: On 09/12/2011 12:47 PM, Andy Colson wrote: work_mem is not the total a query can use. I believe each step can use that much, and each backend can use it for multiple bits. So if you had two backends, each doing 2 sorts, you'd use 2*2 = 4 * 2GB = 8GB.

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Andy Colson
On 9/12/2011 12:57 PM, Shaun Thomas wrote: On 09/12/2011 12:47 PM, Andy Colson wrote: work_mem is not the total a query can use. I believe each step can use that much, and each backend can use it for multiple bits. So if you had two backends, each doing 2 sorts, you'd use 2*2 = 4 * 2GB = 8GB.

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Shaun Thomas
On 09/12/2011 12:47 PM, Andy Colson wrote: work_mem is not the total a query can use. I believe each step can use that much, and each backend can use it for multiple bits. So if you had two backends, each doing 2 sorts, you'd use 2*2 = 4 * 2GB = 8GB. Exactly. Find a big query somewhere in your

Re: [PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Andy Colson
On 9/12/2011 12:33 PM, Robert Schnabel wrote: The recent "data warehouse" thread made me think about how I use work_mem for some of my big queries. So I tried SET work_mem = '4GB' for a session and got ERROR: 4194304 is outside the valid range for parameter "work_mem" (64 .. 2097151) A bit of s

[PERFORM] Allow sorts to use more available memory

2011-09-12 Thread Robert Schnabel
The recent "data warehouse" thread made me think about how I use work_mem for some of my big queries. So I tried SET work_mem = '4GB' for a session and got ERROR: 4194304 is outside the valid range for parameter "work_mem" (64 .. 2097151) A bit of searching turned up the "Allow sorts to use