Does anyone have recommendations for hardware and/or OS to work with around
5TB datasets?
Hardware-wise I'd say dual core opterons. One dual-core-opteron
performs better than two single-core at the same speed. Tyan makes
some boards that have four sockets, thereby giving you 8 cpu's (if you
Because I think we need to. The above would only delete rows
that have name = 'obsid' and value = 'oid080505'. We need to
delete all rows that have the same ids as those rows.
However, from what you note, I bet we could do:
DELETE FROM tmp_table2 WHERE id IN
(SELECT id FROM
Adam,
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
Claus Guttesen
Sent: Tuesday, November 15, 2005 12:29 AM
To: Adam Weisberg
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Hardware/OS recommendations for large
databases ( 5TB)
Hardware-wise I'd say dual core opterons. One dual-core-opteron
performs better than two single-core at the same speed. Tyan makes
some boards that have four sockets, thereby giving you 8 cpu's (if you
need that many). Sun and HP also makes nice hardware although the Tyan
board is more
Luke,Have you tried the areca cards, they are slightly faster yet.DaveOn 15-Nov-05, at 7:09 AM, Luke Lonergan wrote: I agree - you can get a very good one from www.acmemicro.com or www.rackable.com with 8x 400GB SATA disks and the new 3Ware 9550SX SATA RAID controller for about $6K with two
Magnus Hagander wrote:
Because I think we need to. The above would only delete rows
that have name = 'obsid' and value = 'oid080505'. We need to
delete all rows that have the same ids as those rows.
However, from what you note, I bet we could do:
DELETE FROM tmp_table2 WHERE id IN
Dave,
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Dave
CramerSent: Tuesday, November 15, 2005 6:15 AMTo: Luke
LonerganCc: Adam Weisberg;
pgsql-performance@postgresql.orgSubject: Re: [PERFORM] Hardware/OS
recommendations for large databases (
Luke,
Merlin,
just FYI: tyan makes a 8 socket motherboard (up to 16 cores!):
http://www.swt.com/vx50.html
It can be loaded with up to 128 gb memory if all the sockets
are filled :).
Cool!
Just remember that you can't get more than 1 CPU working on a query at a
time without a parallel
Merlin,
just FYI: tyan makes a 8 socket motherboard (up to 16 cores!):
http://www.swt.com/vx50.html
It can be loaded with up to 128 gb memory if all the sockets are
filled :).
Another thought - I priced out a maxed out machine with 16 cores and
128GB of RAM and 1.5TB of usable disk -
On Tue, Nov 15, 2005 at 09:33:25AM -0500, Luke Lonergan wrote:
write performance is now up to par with the best cards I believe. We
find that you still need to set Linux readahead to at least 8MB
(blockdev --setra) to get maximum read performance on them, is that your
What on earth does that
Merlin,
just FYI: tyan makes a 8 socket motherboard (up to 16 cores!):
http://www.swt.com/vx50.html
It can be loaded with up to 128 gb memory if all the sockets are
filled :).
Another thought - I priced out a maxed out machine with 16 cores and
128GB of RAM and 1.5TB of
Luke,
-Original Message-
From: Luke Lonergan [mailto:[EMAIL PROTECTED]
Sent: Tuesday, November 15, 2005 7:10 AM
To: Adam Weisberg
Cc: pgsql-performance@postgresql.org
Subject: RE: [PERFORM] Hardware/OS recommendations for large databases (
5TB)
Adam,
-Original Message-
From:
Because only 1 cpu is used on each query.
- Luke
--
Sent from my BlackBerry Wireless Device
-Original Message-
From: Adam Weisberg [EMAIL PROTECTED]
To: Luke Lonergan [EMAIL PROTECTED]
CC: pgsql-performance@postgresql.org pgsql-performance@postgresql.org
Sent: Tue
Hi Dave,
On Mon, 2005-11-14 at 18:51 -0500, Dave Cramer wrote:
Joost,
I've got experience with these controllers and which version do you
have. I'd expect to see higher than 50MB/s although I've never tried
RAID 5
I routinely see closer to 100MB/s with RAID 1+0 on their 9000 series
Title: Re: [PERFORM] Performance PG 8.0 on dual opteron / 4GB / 3ware
Joost,
On 11/15/05 8:35 AM, Joost Kraaijeveld [EMAIL PROTECTED] wrote:
thousand go relatively fast, after that PostgreSQL crawls to a halt
(other benchmarks like bonnie++ or just dd'ing a big file don't have
this behavior).
Title: Re: [PERFORM] Hardware/OS recommendations for large databases (
Mike,
On 11/15/05 6:55 AM, Michael Stone [EMAIL PROTECTED] wrote:
On Tue, Nov 15, 2005 at 09:33:25AM -0500, Luke Lonergan wrote:
write performance is now up to par with the best cards I believe. We
find that you still
Title: Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)
Merlin,
On 11/15/05 7:20 AM, Merlin Moncure [EMAIL PROTECTED] wrote:
It's hard to say what would be better. My gut says the 5u box would be
a lot better at handling high cpu/high concurrency problems...like your
Merlin Moncure wrote:
You could instead buy 8 machines that total 16 cores, 128GB RAM and
It's hard to say what would be better. My gut says the 5u box would be
a lot better at handling high cpu/high concurrency problems...like your
typical business erp backend. This is pure speculation of
Hi Luke,
On Tue, 2005-11-15 at 10:42 -0800, Luke Lonergan wrote:
With RAID5, it could matter a lot what block size you run your “dd
bigfile” test with. You should run “dd if=/dev/zero of=bigfile bs=8k
count=50” for a 2GB main memory machine, multiply the count by
(your mem/2GB).
If I
I have a query that's making the planner do the wrong thing (for my
definition of wrong) and I'm looking for advice on what to tune to make
it do what I want.
The query consists or SELECT'ing a few fields from a table for a large
number of rows. The table has about seventy thousand rows and
Joost Kraaijeveld wrote:
If I understand correctly (I have 4GB ram):
[EMAIL PROTECTED]:~/tmp$ dd if=/dev/zero of=bigfile bs=8k count=100
100+0 records in
100+0 records out
819200 bytes transferred in 304.085269 seconds (26939812 bytes/sec)
Which looks suspicious: 26308
On Tue, 2005-11-15 at 12:41 -0700, Steve Wampler wrote:
Joost Kraaijeveld wrote:
If I understand correctly (I have 4GB ram):
[EMAIL PROTECTED]:~/tmp$ dd if=/dev/zero of=bigfile bs=8k count=100
100+0 records in
100+0 records out
819200 bytes transferred in 304.085269
On Tue, 2005-11-15 at 13:12, Bill McGonigle wrote:
I have a query that's making the planner do the wrong thing (for my
definition of wrong) and I'm looking for advice on what to tune to make
it do what I want.
The query consists or SELECT'ing a few fields from a table for a large
number
Unless there was a way to guarantee consistency, it would be hard at
best to make this work. Convergence on large data sets across boxes is
non-trivial, and diffing databases is difficult at best. Unless there
was some form of automated way to ensure consistency, going 8 ways into
separate boxes
On 11/15/05, Luke Lonergan [EMAIL PROTECTED] wrote:
Adam,
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
Claus Guttesen
Sent: Tuesday, November 15, 2005 12:29 AM
To: Adam Weisberg
Cc: pgsql-performance@postgresql.org
Subject: Re:
Not at random access in RAID 10 they aren't, and anyone with their
head screwed on right is using RAID 10. The 9500S will still beat the
Areca cards at RAID 10 database access patern.
Alex.
On 11/15/05, Dave Cramer [EMAIL PROTECTED] wrote:
Luke,
Have you tried the areca cards, they are
Title: Re: [PERFORM] Performance PG 8.0 on dual opteron / 4GB / 3ware
Joost,
On 11/15/05 11:51 AM, Joost Kraaijeveld [EMAIL PROTECTED] wrote:
On Tue, 2005-11-15 at 12:41 -0700, Steve Wampler wrote:
Joost Kraaijeveld wrote:
If I understand correctly (I have 4GB ram):
[EMAIL
Title: Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)
James,
On 11/15/05 11:07 AM, James Mello [EMAIL PROTECTED] wrote:
Unless there was a way to guarantee consistency, it would be hard at
best to make this work. Convergence on large data sets across boxes is
Hi Luke,
On Tue, 2005-11-15 at 22:07 -0800, Luke Lonergan wrote:
You might update your driver,
I will do that (but I admit that I am not looking forward to it. When I
was young and did not make money with my computer, I liked challenges
like compiling kernels and not being able to boot the
29 matches
Mail list logo
