Re: [PERFORM] Configuring for maximum memory usage

2008-10-31 Thread Scott Carey
On Thu, Oct 30, 2008 at 2:06 PM, Christiaan Willemsen < [EMAIL PROTECTED]> wrote: > Thanks guys, > Lots of info here that I didn't know about! Since I have one of the latest > Opensolaris builds, I guess the write throttle feature is already in there. > Sadly, the blog doesn't say what build has i

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Christiaan Willemsen
Thanks guys, Lots of info here that I didn't know about! Since I have one of the latest Opensolaris builds, I guess the write throttle feature is already in there. Sadly, the blog doesn't say what build has it included. For writes, I do everything synchronized because we really need a c

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Scott Carey
On Thu, Oct 30, 2008 at 9:55 AM, Joshua D. Drake <[EMAIL PROTECTED]>wrote: > On Thu, 2008-10-30 at 09:46 -0700, Scott Carey wrote: > > > > > Remember that PostgreSQL doesn't cache anything on its own so > > if you do > > want to hit disk it has to be in file cache. > > > >

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Scott Carey
> > If you do very large aggregates, you may need even 1GB on work_mem. > However, a setting that high would require very careful tuning and reduction > of space used by shared_buffers and the ZFS ARC. Its dangerous since each > connection with a large aggregate or sort may consume a lot of mem

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Joshua D. Drake
On Thu, 2008-10-30 at 14:00 -0300, Alvaro Herrera wrote: > Joshua D. Drake wrote: > > > However that isn't the exact same thing as a "cache" at least as I was > > trying to describe it. shared buffers are used to keep track of pages > > (as well as some other stuff) and their current status. That

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Alvaro Herrera
Joshua D. Drake wrote: > However that isn't the exact same thing as a "cache" at least as I was > trying to describe it. shared buffers are used to keep track of pages > (as well as some other stuff) and their current status. That is not the > same as caching a relation. Um, having a page in shar

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Christiaan Willemsen
Hi Scott, Thaks for the clear answers! Scott Carey wrote: You must either increase the memory that ZFS uses, or increase Postgresql shard_mem and work_mem to get the aggregate of the two to use more RAM. I believe, that you have not told ZFS to reserve 8GB, but rather told it to limit itsel

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Joshua D. Drake
On Thu, 2008-10-30 at 09:46 -0700, Scott Carey wrote: > > Remember that PostgreSQL doesn't cache anything on its own so > if you do > want to hit disk it has to be in file cache. > > By my understanding, this is absolutely false. Postgres caches pages > from tabl

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Scott Carey
On Thu, Oct 30, 2008 at 9:05 AM, Joshua D. Drake <[EMAIL PROTECTED]>wrote: > On Thu, 2008-10-30 at 16:58 +0100, Christiaan Willemsen wrote: > > Joshua D. Drake wrote: > > > > > > PostgreSQL is only going to use what it needs. It relies on the OS for > > > much of the caching etc... > > > > > So th

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Scott Carey
You must either increase the memory that ZFS uses, or increase Postgresql shard_mem and work_mem to get the aggregate of the two to use more RAM. I believe, that you have not told ZFS to reserve 8GB, but rather told it to limit itself to 8GB. Some comments below: On Thu, Oct 30, 2008 at 8:15 AM,

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Joshua D. Drake
On Thu, 2008-10-30 at 16:58 +0100, Christiaan Willemsen wrote: > Joshua D. Drake wrote: > > > > PostgreSQL is only going to use what it needs. It relies on the OS for > > much of the caching etc... > > > So that would actually mean that I could raise the setting of the ARC > cache to far more tha

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Christiaan Willemsen
Joshua D. Drake wrote: PostgreSQL is only going to use what it needs. It relies on the OS for much of the caching etc... So that would actually mean that I could raise the setting of the ARC cache to far more than 8 GB? As I said, our database is 250 GB, So I would expect that postgres nee

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Joshua D. Drake
Christiaan Willemsen wrote: Hi there, The problem is getting it to use that much. At the moment, it only uses almost 9 GB, so by far not enough. The problem is getting it to use more... I hope you can help me with working config. PostgreSQL is only going to use what it needs. It relies on t

Re: [PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Ulrich
Hi, you could set effective_cache_size to a high value (free memory on your server that is used for caching). Christiaan Willemsen wrote: Hi there, I configured OpenSolaris on our OpenSolaris Machine. Specs: 2x Quad 2.6 Ghz Xeon 64 GB of memory 16x 15k5 SAS The filesystem is configured using

[PERFORM] Configuring for maximum memory usage

2008-10-30 Thread Christiaan Willemsen
Hi there, I configured OpenSolaris on our OpenSolaris Machine. Specs: 2x Quad 2.6 Ghz Xeon 64 GB of memory 16x 15k5 SAS The filesystem is configured using ZFS, and I think I have found a configuration that performs fairly well. I installed the standard PostgreSQL that came with the OpenSolar