Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-14 Thread k...@rice.edu
On Wed, Sep 14, 2011 at 03:40:07PM -0400, Igor Neyman wrote: From: Carlo Stonebanks [mailto:stonec.regis...@sympatico.ca] Sent: Tuesday, September 13, 2011 9:27 PM To: Performance support Postgresql Subject: Re: Migrated from 8.3 to 9.0 - need to update config (re-post)  

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-14 Thread Carlo Stonebanks
from 8.3 to 9.0 - need to update config (re-post) On 09/14/2011 02:56 AM, Carlo Stonebanks wrote: Even for 300 stateful applications that can remain connected for up to a week, continuously distilling data (imports)? If they're all doing active work all that time you can still benefit

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-13 Thread Carlo Stonebanks
be in this case? Thanks again for your time, Carlo Date: Fri, 9 Sep 2011 13:16:28 -0500 From: kevin.gritt...@wicourts.gov To: pgsql-performance@postgresql.org; stonec.regis...@sympatico.ca Subject: Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post) Carlo Stonebanks

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-13 Thread Kevin Grittner
Carlo Stonebanks wrote: max_connections = 300 Too high. Both throughput and latency should improve with correct use of a connection pooler. Even for 300 stateful applications that can remain connected for up to a week, continuously distilling data (imports)? Absolutely. A good

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-13 Thread Carlo Stonebanks
? Carlo Date: Tue, 13 Sep 2011 16:13:00 -0500 From: kevin.gritt...@wicourts.gov To: pgsql-performance@postgresql.org; stonec.regis...@sympatico.ca Subject: RE: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post) Carlo Stonebanks wrote: max_connections = 300

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-13 Thread Craig Ringer
On 09/14/2011 02:56 AM, Carlo Stonebanks wrote: Even for 300 stateful applications that can remain connected for up to a week, continuously distilling data (imports)? If they're all doing active work all that time you can still benefit from a pooler. Say your server can service 50

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-13 Thread Kevin Grittner
Craig Ringer wrote: I agreed with almost your entire post, but there is one sentence with which I take issue. However, it will also increase latency for service for those workers because they may have to wait a while before their transaction runs, even though their transaction will

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-10 Thread Carlo Stonebanks
, 2011 2:16 PM To: pgsql-performance@postgresql.org; Carlo Stonebanks Subject: Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post) Carlo Stonebanks stonec.regis...@sympatico.ca wrote: this is a full-time ETL system, with only a handful of actual *users* and automated processes

[PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-09 Thread Carlo Stonebanks
Hello performance wizards! (Sorry for the re-post if this appears twice - I see no evidence e-mailing to pgsql-perfomrance is working yet.) My client has migrated his 8.3 hosted DB to new machines running PG 9.0. It’s time to look at the config settings. Immediately below are the

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-09 Thread Kevin Grittner
Carlo Stonebanks stonec.regis...@sympatico.ca wrote: this is a full-time ETL system, with only a handful of actual *users* and automated processes over 300 connections running *import* programs 24/7 Intel* Xeon* Processor X5560 (8M Cache, 2.80 GHz, 6.40 GT/s Intel* QPI) x 2, dual quad

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-09 Thread Claudio Freire
On Fri, Sep 9, 2011 at 5:38 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: This is getting back to that issue of using only enough processes at one time to keep all the bottleneck resources fully utilized.  Some people tend to assuem that if they throw a few more concurrent processes

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-09 Thread Claudio Freire
On Fri, Sep 9, 2011 at 3:16 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Add together the shared_buffers setting and whatever the OS tells you is used for cache under your normal load.  It's usually 75% of RM or higher.  (NOTE: This doesn't cause any allocation of RAM; it's a hint to

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-09 Thread Kevin Grittner
Claudio Freire klaussfre...@gmail.com wrote: On Fri, Sep 9, 2011 at 3:16 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Add together the shared_buffers setting and whatever the OS tells you is used for cache under your normal load. It's usually 75% of RM or higher. (NOTE: This

Re: [PERFORM] Migrated from 8.3 to 9.0 - need to update config (re-post)

2011-09-09 Thread Kevin Grittner
Claudio Freire klaussfre...@gmail.com wrote: On the other hand, in order to benefit from synchro scans and stuff like that, one has to increase concurrency beyond what is normally considered optimal. That's a good example of why any general configuration advice should just be used as a