Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-08 Thread Simon Riggs
On Tue, 2005-06-07 at 23:50 -0400, Tom Lane wrote: Regarding 2GB memory allocation, though, we *could* really use support for work_mem and maintenance_mem of 2GB. Again, let's see some evidence that it's worth putting effort into that. (Offhand it seems this is probably an easier fix

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-08 Thread Michael Stone
On Tue, Jun 07, 2005 at 11:50:33PM -0400, Tom Lane wrote: Again, let's see some evidence that it's worth putting effort into that. (Offhand it seems this is probably an easier fix than changing the shared-memory allocation code; but conventional wisdom is that really large values of work_mem are

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-08 Thread Tom Arthurs
I just puhsd 8.0.3 to production on Sunday, and haven't had a time to really monitor it under load, so I can't tell if it's helped the context switch problem yet or not. Neil Conway wrote: Tom Arthurs wrote: Yes, shared buffers in postgres are not used for caching Shared buffers in

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-08 Thread Bjoern Metzdorf
Hi, I just puhsd 8.0.3 to production on Sunday, and haven't had a time to really monitor it under load, so I can't tell if it's helped the context switch problem yet or not. Attached is a vmstat 5 output from one of our machines. This is a dual Xeon 3,2 Ghz with EM64T and 8 GB RAM, running

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-08 Thread Sam Vilain
Joshua D. Drake wrote: Yes - we have seen with oracle 64 bit that there can be as much as a 10% hit moving from 32 - but we make it up big time with large db-buffer sizes that drastically Well for Opteron you should also gain from the very high memory bandwidth and the fact that it has I

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-08 Thread Neil Conway
Tom Arthurs wrote: I just puhsd 8.0.3 to production on Sunday, and haven't had a time to really monitor it under load, so I can't tell if it's helped the context switch problem yet or not. 8.0 is unlikely to make a significant difference -- by current sources I meant the current CVS HEAD

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Donald Courtney
Get FATAL when starting up (64 bit) with large shared_buffers setting I built a 64 bit for Sparc/Solaris easily but I found that the startup of postmaster generates a FATAL diagnostic due to going over the 2GB limit (3.7 GB). When building for 64 bit is there some other things that must change

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Tom Arthurs
According to my research, you only need a 64 bit image if you are going to be doing intensive floating point operations (which most db servers don't do). Some benchmarking results I've found on the internet indicate that 64 bit executables can be slower than 32 bit versions. I've been running

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Joshua D. Drake
The system has 8 CPUs w/ 32 GB - I'm hoping to see some benefit to large caches - Am I missing something key with postgreSQL? Yes - we have seen with oracle 64 bit that there can be as much as a 10% hit moving from 32 - but we make it up big time with large db-buffer sizes that drastically

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Donald Courtney
Tom Arthurs wrote: According to my research, you only need a 64 bit image if you are going to be doing intensive floating point operations (which most db servers don't do). Some benchmarking results I've found on the internet indicate that 64 bit executables can be slower than 32 bit

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Tom Arthurs
Yes, shared buffers in postgres are not used for caching -- unlike Oracle. Every time we hire an Oracle dba, I have to break them of the notion (which I shared when I started with postgres -- Josh Berkus and Josh Drake helped burst that bubble for me) :) You should gain i/o reduction through

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Alvaro Herrera
On Tue, Jun 07, 2005 at 04:19:24PM -0400, Donald Courtney wrote: The system has 8 CPUs w/ 32 GB - I'm hoping to see some benefit to large caches - Am I missing something key with postgreSQL? Yeah. Postgres makes extensive use of the kernel's cache (or, more precisely, assumes that the

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Michael Stone
On Tue, Jun 07, 2005 at 01:39:04PM -0700, Tom Arthurs wrote: Yes, shared buffers in postgres are not used for caching That begs the question of what they are used for. :) Mike Stone ---(end of broadcast)--- TIP 9: the planner will ignore your

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: This was the standard wisdom with releases previous to 8.0; I'm not sure if anyone confirmed to still hold after the buffer manager changes in 8.0 and later in 8.1 -- we saw extensive redesign of the bufmgr on both, so the behavior may have changed. If

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Neil Conway
Tom Arthurs wrote: Yes, shared buffers in postgres are not used for caching Shared buffers in Postgres _are_ used for caching, they just form a secondary cache on top of the kernel's IO cache. Postgres does IO through the filesystem, which is then cached by the kernel. Increasing

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread John A Meinel
Neil Conway wrote: Tom Arthurs wrote: Yes, shared buffers in postgres are not used for caching Shared buffers in Postgres _are_ used for caching, they just form a secondary cache on top of the kernel's IO cache. Postgres does IO through the filesystem, which is then cached by the

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Josh Berkus
Tom, Obviously we'd be willing to do this work if there were convincing evidence it'd be worth the time. A benchmark showing performance continuing to climb with increasing shared_buffers right up to the 2Gb limit would be reasonably convincing. I think there is 0 chance of drawing such a

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-07 Thread Tom Lane
Josh Berkus josh@agliodbs.com writes: Not that I've seen in testing so far. Your improvements have, fortunately, eliminated the penalty for allocating too much shared buffers as far as I can tell (at least, allocating 70,000 when gains stopped at 15,000 doesn't seem to carry a penalty),

[PERFORM] Postgresql on an AMD64 machine

2005-06-06 Thread Mark Rinaudo
I'm not sure if this is the appropriate list to post this question to but i'm starting with this one because it is related to the performance of Postgresql server. I have a Penguin Computing dual AMD 64 bit opteron machine with 8 Gigs of memory. In my attempt to increase the number of

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-06 Thread Vivek Khera
On Jun 6, 2005, at 1:53 PM, Mark Rinaudo wrote: I'm questioning the number of semaphores being used. In order for postgresql to start I had to set the maximum number of semaphores system wide to 600. This seems to be an abnormal amount of semaphores. I'm curious if this is a bug in

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-06 Thread Mike Rylander
On 06 Jun 2005 12:53:40 -0500, Mark Rinaudo [EMAIL PROTECTED] wrote: I'm not sure if this is the appropriate list to post this question to but i'm starting with this one because it is related to the performance of Postgresql server. I have a Penguin Computing dual AMD 64 bit opteron machine

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-06 Thread Tom Lane
Mike Rylander [EMAIL PROTECTED] writes: On 06 Jun 2005 12:53:40 -0500, Mark Rinaudo [EMAIL PROTECTED] wrote: I'm not sure if this is the appropriate list to post this question to but i'm starting with this one because it is related to the performance of Postgresql server. I have a Penguin

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-06 Thread Mark Rinaudo
I'm running the Redhat Version of Postgresql which came pre-installed with Redhat ES. It's version number is 7.3.10-1. I'm not sure what options it was compiled with. Is there a way for me to tell? Should i just compile my own postgresql for this platform? Thanks Mark On Mon, 2005-06-06 at

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-06 Thread Neil Conway
Mark Rinaudo wrote: I'm running the Redhat Version of Postgresql which came pre-installed with Redhat ES. It's version number is 7.3.10-1. I'm not sure what options it was compiled with. Is there a way for me to tell? `pg_config --configure` in recent releases. Should i just compile my own

Re: [PERFORM] Postgresql on an AMD64 machine

2005-06-06 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: (From a Redhat POV, it would probably be a good idea to patch 7.3 to include the relatively trivial changes needed for decent AMD64 performance, How embarrassing :-( Will see about fixing it. However, this certainly won't ship before the next RHEL3