Re: [PERFORM] Hardware/OS recommendations for large databases (5TB)

2005-11-17 Thread Vivek Khera
On Nov 16, 2005, at 4:50 PM, Claus Guttesen wrote: I'm (also) FreeBSD-biased but I'm not shure whether the 5 TB fs will work so well if tools like fsck are needed. Gvinum could be one option but I don't have any experience in that area. Then look into an external filer and mount via NFS.

Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)

2005-11-16 Thread William Yu
James Mello wrote: Unless there was a way to guarantee consistency, it would be hard at best to make this work. Convergence on large data sets across boxes is non-trivial, and diffing databases is difficult at best. Unless there was some form of automated way to ensure consistency, going 8 ways

Re: [PERFORM] Hardware/OS recommendations for large databases (5TB)

2005-11-16 Thread Vivek Khera
On Nov 15, 2005, at 3:28 AM, Claus Guttesen wrote: Hardware-wise I'd say dual core opterons. One dual-core-opteron performs better than two single-core at the same speed. Tyan makes at 5TB data, i'd vote that the application is disk I/O bound, and the difference in CPU speed at the level

Re: [PERFORM] Hardware/OS recommendations for large databases (5TB)

2005-11-16 Thread Claus Guttesen
at 5TB data, i'd vote that the application is disk I/O bound, and the difference in CPU speed at the level of dual opteron vs. dual-core opteron is not gonna be noticed. to maximize disk, try getting a dedicated high-end disk system like nstor or netapp file servers hooked up to fiber

Re: [PERFORM] Hardware/OS recommendations for large databases (5TB)

2005-11-15 Thread Claus Guttesen
Does anyone have recommendations for hardware and/or OS to work with around 5TB datasets? Hardware-wise I'd say dual core opterons. One dual-core-opteron performs better than two single-core at the same speed. Tyan makes some boards that have four sockets, thereby giving you 8 cpu's (if you

Re: [PERFORM] Hardware/OS recommendations for large databases (5TB)

2005-11-15 Thread Merlin Moncure
Hardware-wise I'd say dual core opterons. One dual-core-opteron performs better than two single-core at the same speed. Tyan makes some boards that have four sockets, thereby giving you 8 cpu's (if you need that many). Sun and HP also makes nice hardware although the Tyan board is more

Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)

2005-11-15 Thread Merlin Moncure
Merlin, just FYI: tyan makes a 8 socket motherboard (up to 16 cores!): http://www.swt.com/vx50.html It can be loaded with up to 128 gb memory if all the sockets are filled :). Another thought - I priced out a maxed out machine with 16 cores and 128GB of RAM and 1.5TB of

Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)

2005-11-15 Thread Adam Weisberg
Luke, -Original Message- From: Luke Lonergan [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 15, 2005 7:10 AM To: Adam Weisberg Cc: pgsql-performance@postgresql.org Subject: RE: [PERFORM] Hardware/OS recommendations for large databases ( 5TB) Adam, -Original Message- From

Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)

2005-11-15 Thread William Yu
Merlin Moncure wrote: You could instead buy 8 machines that total 16 cores, 128GB RAM and It's hard to say what would be better. My gut says the 5u box would be a lot better at handling high cpu/high concurrency problems...like your typical business erp backend. This is pure speculation of

Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)

2005-11-15 Thread James Mello
15, 2005 10:57 AM To: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB) Merlin Moncure wrote: You could instead buy 8 machines that total 16 cores, 128GB RAM and It's hard to say what would be better. My gut says the 5u box would

[PERFORM] Hardware/OS recommendations for large databases (5TB)

2005-11-14 Thread Adam Weisberg
Does anyone have recommendations for hardware and/or OS to work with around 5TB datasets? The data is for analysis, so there is virtually no inserting besides a big bulk load. Analysis involves full-database aggregations - mostlybasic arithmetic and grouping. In addition, much