Re: [PERFORM] Hardware suggestions for Linux/PGSQL server

2003-12-13 Thread Jeff Bohmer
Just one more piece of advice, you might want to look into a good battery
backed cache hardware RAID controller.  They work quite well for heavily
updated databases.  The more drives you throw at the RAID array the faster
it will be.
I've seen this list often recommended such a setup.  We'll probably 
get battery-backed write cache and start out with a 4 disk RAID 10 
array.  Then add more disks and change RAID 5 if more read 
performance is needed.

Thanks,
- Jeff
--
Jeff Bohmer
VisionLink, Inc.
_
303.402.0170
www.visionlink.org
_
People. Tools. Change. Community.
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [PERFORM] Hardware suggestions for Linux/PGSQL server

2003-12-13 Thread Jeff Bohmer

Shridhar Daithankar wrote:
FWIW, there are only two pieces of software that need 64bit aware 
for a typical server job. Kernel and glibc. Rest of the apps can do 
fine as 32 bits unless you are oracle and insist on outsmarting OS.

In fact running 32 bit apps on 64 bit OS has plenty of advantages 
like effectively using the cache. Unless you need 64bit, going for 
64bit software is not advised.
This is a good point. While doing research on this matter a few 
months back, I saw comments by people testing 64-bit MySQL that some 
operations would run faster and some slower due to the use of 64-bit 
datatypes versus 32-bit. The best solution in the end is probably to 
run 32-bit Postgres under a 64-bit kernel -- unless your DB tends to 
have a lot of 64-bit datatypes.


Thanks Shridhar and William,

This advice has been very helpful.  I would imagine a lot of folks 
are, or will soon be looking at 32- vs. 64-bit just for memory 
reasons and not 64-bit apps.

- Jeff
--
Jeff Bohmer
VisionLink, Inc.
_
303.402.0170
www.visionlink.org
_
People. Tools. Change. Community.
---(end of broadcast)---
TIP 6: Have you searched our list archives?
  http://archives.postgresql.org


Re: [PERFORM] Hardware suggestions for Linux/PGSQL server

2003-12-12 Thread William Yu
Shridhar Daithankar wrote:
FWIW, there are only two pieces of software that need 64bit aware for a 
typical server job. Kernel and glibc. Rest of the apps can do fine as 32 
bits unless you are oracle and insist on outsmarting OS.

In fact running 32 bit apps on 64 bit OS has plenty of advantages like 
effectively using the cache. Unless you need 64bit, going for 64bit 
software is not advised.
This is a good point. While doing research on this matter a few months 
back, I saw comments by people testing 64-bit MySQL that some operations 
would run faster and some slower due to the use of 64-bit datatypes 
versus 32-bit. The best solution in the end is probably to run 32-bit 
Postgres under a 64-bit kernel -- unless your DB tends to have a lot of 
64-bit datatypes.

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
 subscribe-nomail command to [EMAIL PROTECTED] so that your
 message can get through to the mailing list cleanly


Re: [PERFORM] Hardware suggestions for Linux/PGSQL server

2003-12-11 Thread William Yu
Jeff Bohmer wrote:
We're willing to shell out extra bucks to get something that will 
undoubtedly handle the projected peak load in 12 months with excellent 
performance.  But we're not familiar with PG's performance on Linux and 
don't like to waste money.
Properly tuned, PG on Linux runs really nice. A few people have 
mentioned the VM swapping algorithm on Linux is semi-dumb. I get around 
that problem by having a ton of memory and almost no swap.

I've been thinking of this (overkill? not enough?):
2 Intel 32-bit CPUs
Lowest clock speed chip for the fastest available memory bus
4 GB RAM (maybe we only need 3 GB to start with?)
SCSI RAID 1 for OS
For PostgreSQL data and logs ...
15k rpm SCSI disks
RAID 5, 7 disks, 256MB battery-backed write cache
(Should we save $ and get a 4-disk RAID 10 array?)
I wonder about the 32bit+bigmem vs. 64bit question.  At what database 
size will we need more than 4GB RAM?
With 4GB of RAM, you're already running into bigmem. By default, Linux 
gives 2GB of address space to programs and 2GB to kernel. I usually see 
people quote 5%-15% penalty in general for using PAE versus a flat 
address space. I've seen simple MySQL benchmarks where 64-bit versions 
run 35%+ faster versus 32-bit+PAE but how that translates to PG, I dunno 
yet.

We'd like to always have enough RAM to cache the entire database. While 
64bit is in our long-term future, we're willing to stick with 32bit 
Linux until 64bit Linux on Itanium/Opteron and 64bit PostgreSQL settle 
in to proven production-quality.
Well if this is the case, you probably should get an Opteron server 
*now* and just run 32-bit Linux on it until you're sure about the 
software. No point in buying a Xeon and then throwing the machine away 
in a year when you decide you need 64-bit for more speed.

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [PERFORM] Hardware suggestions for Linux/PGSQL server

2003-12-11 Thread William Yu
Jeff Bohmer wrote:
It seems I don't fully understand the bigmem situation.  I've searched 
the archives, googled, checked RedHat's docs, etc.  But I'm getting 
conflicting, incomplete and/or out of date information.  Does anyone 
have pointers to bigmem info or configuration for the 2.4 kernel?
Bigmem is the name for Linux's PAE support.

If Linux is setup with 2GB for kernel and 2GB for user, would that be OK 
with a DB size of 2-2.5 GB?  I'm figuring the kernel will cache most/all 
of the DB in it's 2GB and there's 2GB left for PG processes. Where does 
PG's SHM buffers live, kernel or user?  (I don't plan on going crazy 
with buffers, but will guess we'd need about 128MB, 256MB at most.)
PG's SHM buffers live in user. Whether Linux's OS caches lives in user 
or kernel, I think it's in kernel and I remember reading a max of ~950KB 
w/o bigmem which means your 3.5GB of available OS memory will definitely 
have to be swapped in and out of kernel space using PAE.

Well if this is the case, you probably should get an Opteron server 
*now* and just run 32-bit Linux on it until you're sure about the 
software. No point in buying a Xeon and then throwing the machine away 
in a year when you decide you need 64-bit for more speed.
That's a good point.  I had forgotten about the option to run 32bit on 
an Operton.  If we had 3GB or 4GB initially on an Opteron, we'd need 
bigmem for 32bit Linux, right?

This might work nicely since we'd factor in the penalty from PAE for now 
and have the performance boost from moving to 64bit available on 
demand.  Not having to build another DB server in a year would also be 
nice.

FYI, we need stability first and performance second.
We ordered a 2x Opteron server the moment the CPU was released and it's 
been perfect -- except for one incident where the PCI riser card had 
drifted out of the PCI slot due to the heavy SCSI cables connected to 
the card.

I think most of the Opteron server MBs are pretty solid but you want 
extra peace-of-mind, you could get a server from Newisys as they pack in 
a cartload of extra monitoring features.

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings