Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-26 Thread Greg Smith
Carlo Stonebanks wrote: effective_cache_size should be set to how much memory is leftover for disk caching after taking into account what's used by the operating system, dedicated PostgreSQL memory, and other applications. If it's set too low, indexes may not be used for executing queries

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-22 Thread Carlo Stonebanks
Hi Greg, As a follow up to this suggestion: I don't see effective_cache_size listed there. If that's at the default, I wouldn't be surprised that you're seeing sequential scans instead of indexed ones far too often. I found an article written by you

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Alvaro Herrera
Scott Marlowe escribió: On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: 4) Is this the right PG version for our needs? 8.3 is very stable. Update to the latest. 8.4 seems good, but I've had, and still am having, problems with it crashing in

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Scott Marlowe
On Thu, Jan 21, 2010 at 8:51 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Scott Marlowe escribió: On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: 4) Is this the right PG version for our needs? 8.3 is very stable.  Update to the latest.  8.4

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Alvaro Herrera
Scott Marlowe escribió: On Thu, Jan 21, 2010 at 8:51 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Scott Marlowe escribió: On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: 4) Is this the right PG version for our needs? 8.3 is very

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Scott Marlowe
On Thu, Jan 21, 2010 at 9:44 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Scott Marlowe escribió: On Thu, Jan 21, 2010 at 8:51 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Scott Marlowe escribió: On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-21 Thread Devrim GÜNDÜZ
On Thu, 2010-01-21 at 13:44 -0300, Alvaro Herrera wrote: I think Devrim publishes debuginfo packages which you need to install separately. Right. -- Devrim GÜNDÜZ, RHCE Command Prompt - http://www.CommandPrompt.com devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-20 Thread Carlo Stonebanks
yeah, the values are at the end. Sounds like your vacuum settings are too non-aggresive. Generally this is the vacuum cost delay being too high. Of course, I have to ask: what's the down side? Yes! You can run vacuum verbose against the regular old postgres database (or just create one for

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-20 Thread Kevin Grittner
Carlo Stonebanks stonec.regis...@sympatico.ca wrote: yeah, the values are at the end. Sounds like your vacuum settings are too non-aggresive. Generally this is the vacuum cost delay being too high. Of course, I have to ask: what's the down side? If you make it too aggressive, it could

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-20 Thread Robert Haas
On Wed, Jan 20, 2010 at 3:03 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: Yes!  You can run vacuum verbose against the regular old postgres database (or just create one for testing with nothing in it) and you'll still get the fsm usage numbers from that!  So, no need to run it

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-19 Thread Carlo Stonebanks
Hi Scott, Sorry for the very late reply on this post, but I'd like to follow up. The reason that I took so long to reply was due to this suggestion: Run vacuum verbose to see if you're overrunning the max_fsm_pages settings or the max_fsm_relations. My first thought was, does he mean

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-19 Thread Scott Marlowe
On Tue, Jan 19, 2010 at 2:09 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: Hi Scott, Sorry for the very late reply on this post, but I'd like to follow up. The reason that I took so long to reply was due to this suggestion: Run vacuum verbose to see if you're overrunning the

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Tony McC
On Thu, 14 Jan 2010 16:35:53 -0600 Dave Crooke dcro...@gmail.com wrote: For any given database engine, regardless of the marketing and support stance, there is only one true primary enterprise OS platform that most big mission critical sites use, and is the best supported and most stable

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Richard Broersma
On Fri, Jan 15, 2010 at 8:10 AM, Tony McC af...@btinternet.com wrote: most stable platform for that RDBMS. For Oracle, that's HP-UX (but 10 years ago, it was Solaris). For PostgreSQL, it's Linux. I am interested in this response and am wondering if this is just Dave's opinion or some sort of

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Robert Haas
On Fri, Jan 15, 2010 at 11:10 AM, Tony McC af...@btinternet.com wrote: what is it about Linux specifically (as contrasted with other Unix-like OSes, especially Open Source ones) that makes it particularly suitable for running PostgreSQL? Nothing that I know of. ...Robert -- Sent via

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-15 Thread marcin mank
On Thu, Jan 14, 2010 at 8:17 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: . 48 GB RAM 2) Which Windows OS would you recommend? (currently 2008 x64 Server) There is not a 64-bit windows build now - You would be limited to shared_buffers at about a gigabyte. Choose Linux Greetings

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Tom Lane
Richard Broersma richard.broer...@gmail.com writes: On Fri, Jan 15, 2010 at 8:10 AM, Tony McC af...@btinternet.com wrote: most stable platform for that RDBMS. For Oracle, that's HP-UX (but 10 years ago, it was Solaris). For PostgreSQL, it's Linux. I am interested in this response and am

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Greg Smith
Tom Lane wrote: Given the Linux kernel hackers' apparent disinterest in fixing their OOM kill policy or making write barriers work well (or at all, with LVM), I think arguing that Linux is the best database platform requires a certain amount of suspension of disbelief. Don't forget the

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Scott Marlowe
On Fri, Jan 15, 2010 at 11:28 AM, Greg Smith g...@2ndquadrant.com wrote: Tom Lane wrote: Given the Linux kernel hackers' apparent disinterest in fixing their OOM kill policy or making write barriers work well (or at all, with LVM), I think arguing that Linux is the best database platform

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Kevin Grittner
Scott Marlowe scott.marl...@gmail.com wrote: I will say that XFS seems to be a very stable file system, and we use it for some of our databases with no problems at all. But most of our stuff sits on ext3 because it's stable and reliable and fast enough. Our PostgreSQL data directories are

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Dave Crooke
This is the second time I've heard that PG shared buffer on Windows doesn't matter ... I'd like to understand the reasoning behind that claim, and why it differs from other DB servers. though that's much less important for Pg than for most other things, as Pg uses a

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Greg Smith
Dave Crooke wrote: My reasoning goes like this: a. there is a significant performance benefit to using a large proportion of memory as in-process DB server cache instead of OS level block / filesystem cache b. the only way to do so on modern hardware (i.e. 4GB) is with a 64-bit binary c.

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-15 Thread Tom Lane
Dave Crooke dcro...@gmail.com writes: This is the second time I've heard that PG shared buffer on Windows doesn't matter ... I'd like to understand the reasoning behind that claim, and why it differs from other DB servers. AFAIK we don't really understand why, but the experimental evidence is

[PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Carlo Stonebanks
My client just informed me that new hardware is available for our DB server. . Intel Core 2 Quads Quad . 48 GB RAM . 4 Disk RAID drive (RAID level TBD) I have put the ugly details of what we do with our DB below, as well as the postgres.conf settings. But, to summarize: we have a PostgreSQL

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Joshua D. Drake
On Thu, 14 Jan 2010 14:17:13 -0500, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: My client just informed me that new hardware is available for our DB server. . Intel Core 2 Quads Quad . 48 GB RAM . 4 Disk RAID drive (RAID level TBD) I have put the ugly details of what we do with

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Scott Marlowe
On Thu, Jan 14, 2010 at 12:17 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: My client just informed me that new hardware is available for our DB server. . Intel Core 2 Quads Quad . 48 GB RAM . 4 Disk RAID drive (RAID level TBD) I have put the ugly details of what we do with our

[PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-14 Thread Carlo Stonebanks
My client just informed me that new hardware is available for our DB server. . Intel Core 2 Quads Quad . 48 GB RAM . 4 Disk RAID drive (RAID level TBD) I have put the ugly details of what we do with our DB below, as well as the postgres.conf settings. But, to summarize: we have a PostgreSQL

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-14 Thread Dave Crooke
I'll bite 1. In general, RAID-10 is the only suitable RAID configuration for a database. The decision making comes in how many drives, and splitting stuff up into LUNs (like putting pg_xlog on its own LUN). 2. None of the above - you're asking the wrong question really. PostgreSQL is open

Re: [PERFORM] New server to improve performance on our large and busy DB - advice? (v2)

2010-01-14 Thread Craig Ringer
On 15/01/2010 6:35 AM, Dave Crooke wrote: I'll bite 1. In general, RAID-10 is the only suitable RAID configuration for a database. The decision making comes in how many drives, and splitting stuff up into LUNs (like putting pg_xlog on its own LUN). The biggest problem with Postgres

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Greg Smith
Carlo Stonebanks wrote: 1) Which RAID level would you recommend It looks like you stepped over a critical step, which is will the server have a good performing RAID card?. Your whole upgrade could underperform if you make a bad mistake on that part. It's really important to nail that

Re: [PERFORM] New server to improve performance on our large and busy DB - advice?

2010-01-14 Thread Carlo Stonebanks
Pretty much everyone thinks their requirements are exceptional. It's funny how infrequently that's actually true. The techniques that favor index-use aren't that unique: collect better stats, set basic parameters correctly, adjust random_page_cost, investigate plans that don't do what you