> Does anyone have recommendations for hardware and/or OS to work with around
> 5TB datasets?
Hardware-wise I'd say dual core opterons. One dual-core-opteron
performs better than two single-core at the same speed. Tyan makes
some boards that have four sockets, thereby giving you 8 cpu's (if you
ne
> Because I think we need to. The above would only delete rows
> that have name = 'obsid' and value = 'oid080505'. We need to
> delete all rows that have the same ids as those rows.
> However, from what you note, I bet we could do:
>
>DELETE FROM "tmp_table2" WHERE id IN
> (SELECT
Adam,
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of
> Claus Guttesen
> Sent: Tuesday, November 15, 2005 12:29 AM
> To: Adam Weisberg
> Cc: pgsql-performance@postgresql.org
> Subject: Re: [PERFORM] Hardware/OS recommendations for large
> database
> Hardware-wise I'd say dual core opterons. One dual-core-opteron
> performs better than two single-core at the same speed. Tyan makes
> some boards that have four sockets, thereby giving you 8 cpu's (if you
> need that many). Sun and HP also makes nice hardware although the Tyan
> board is more co
Luke,Have you tried the areca cards, they are slightly faster yet.DaveOn 15-Nov-05, at 7:09 AM, Luke Lonergan wrote: I agree - you can get a very good one from www.acmemicro.com or www.rackable.com with 8x 400GB SATA disks and the new 3Ware 9550SX SATA RAID controller for about $6K with two Opteron
Magnus Hagander wrote:
>>Because I think we need to. The above would only delete rows
>>that have name = 'obsid' and value = 'oid080505'. We need to
>>delete all rows that have the same ids as those rows.
>>However, from what you note, I bet we could do:
>>
>> DELETE FROM "tmp_table2" WHERE
Dave,
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Dave
CramerSent: Tuesday, November 15, 2005 6:15 AMTo: Luke
LonerganCc: Adam Weisberg;
pgsql-performance@postgresql.orgSubject: Re: [PERFORM] Hardware/OS
recommendations for large databases (
Luke,
Merlin,
> just FYI: tyan makes a 8 socket motherboard (up to 16 cores!):
> http://www.swt.com/vx50.html
>
> It can be loaded with up to 128 gb memory if all the sockets
> are filled :).
Cool!
Just remember that you can't get more than 1 CPU working on a query at a
time without a parallel data
Merlin,
> > just FYI: tyan makes a 8 socket motherboard (up to 16 cores!):
> > http://www.swt.com/vx50.html
> >
> > It can be loaded with up to 128 gb memory if all the sockets are
> > filled :).
Another thought - I priced out a maxed out machine with 16 cores and
128GB of RAM and 1.5TB of usab
On Tue, Nov 15, 2005 at 09:33:25AM -0500, Luke Lonergan wrote:
write performance is now up to par with the best cards I believe. We
find that you still need to set Linux readahead to at least 8MB
(blockdev --setra) to get maximum read performance on them, is that your
What on earth does that d
> Merlin,
>
> > > just FYI: tyan makes a 8 socket motherboard (up to 16 cores!):
> > > http://www.swt.com/vx50.html
> > >
> > > It can be loaded with up to 128 gb memory if all the sockets are
> > > filled :).
>
> Another thought - I priced out a maxed out machine with 16 cores and
> 128GB of RAM
Luke,
-Original Message-
From: Luke Lonergan [mailto:[EMAIL PROTECTED]
Sent: Tuesday, November 15, 2005 7:10 AM
To: Adam Weisberg
Cc: pgsql-performance@postgresql.org
Subject: RE: [PERFORM] Hardware/OS recommendations for large databases (
5TB)
Adam,
> -Original Message-
> From:
Because only 1 cpu is used on each query.
- Luke
--
Sent from my BlackBerry Wireless Device
-Original Message-
From: Adam Weisberg <[EMAIL PROTECTED]>
To: Luke Lonergan <[EMAIL PROTECTED]>
CC: pgsql-performance@postgresql.org
Sent: Tue Nov 15 10:40:53 2005
Subject
Hi Dave,
On Mon, 2005-11-14 at 18:51 -0500, Dave Cramer wrote:
> Joost,
>
> I've got experience with these controllers and which version do you
> have. I'd expect to see higher than 50MB/s although I've never tried
> RAID 5
>
> I routinely see closer to 100MB/s with RAID 1+0 on their 9000 se
Title: Re: [PERFORM] Performance PG 8.0 on dual opteron / 4GB / 3ware
Joost,
On 11/15/05 8:35 AM, "Joost Kraaijeveld" <[EMAIL PROTECTED]> wrote:
thousand go relatively fast, after that PostgreSQL crawls to a halt
(other "benchmarks" like bonnie++ or just dd'ing a big file don't have
this behav
Title: Re: [PERFORM] Hardware/OS recommendations for large databases (
Mike,
On 11/15/05 6:55 AM, "Michael Stone" <[EMAIL PROTECTED]> wrote:
On Tue, Nov 15, 2005 at 09:33:25AM -0500, Luke Lonergan wrote:
>write performance is now up to par with the best cards I believe. We
>find that you sti
Title: Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)
Merlin,
On 11/15/05 7:20 AM, "Merlin Moncure" <[EMAIL PROTECTED]> wrote:
It's hard to say what would be better. My gut says the 5u box would be
a lot better at handling high cpu/high concurrency problems...like your
t
Merlin Moncure wrote:
You could instead buy 8 machines that total 16 cores, 128GB RAM and
It's hard to say what would be better. My gut says the 5u box would be
a lot better at handling high cpu/high concurrency problems...like your
typical business erp backend. This is pure speculation of c
Hi Luke,
On Tue, 2005-11-15 at 10:42 -0800, Luke Lonergan wrote:
> With RAID5, it could matter a lot what block size you run your “dd
> bigfile” test with. You should run “dd if=/dev/zero of=bigfile bs=8k
> count=50” for a 2GB main memory machine, multiply the count by
> (/2GB).
If I understa
I have a query that's making the planner do the wrong thing (for my
definition of wrong) and I'm looking for advice on what to tune to make
it do what I want.
The query consists or SELECT'ing a few fields from a table for a large
number of rows. The table has about seventy thousand rows and t
Joost Kraaijeveld wrote:
> If I understand correctly (I have 4GB ram):
>
> [EMAIL PROTECTED]:~/tmp$ dd if=/dev/zero of=bigfile bs=8k count=100
> 100+0 records in
> 100+0 records out
> 819200 bytes transferred in 304.085269 seconds (26939812 bytes/sec)
>
> Which looks suspicious: 2
On Tue, 2005-11-15 at 12:41 -0700, Steve Wampler wrote:
> Joost Kraaijeveld wrote:
> > If I understand correctly (I have 4GB ram):
> >
> > [EMAIL PROTECTED]:~/tmp$ dd if=/dev/zero of=bigfile bs=8k count=100
> > 100+0 records in
> > 100+0 records out
> > 819200 bytes transferred in
On Tue, 2005-11-15 at 13:12, Bill McGonigle wrote:
> I have a query that's making the planner do the wrong thing (for my
> definition of wrong) and I'm looking for advice on what to tune to make
> it do what I want.
>
> The query consists or SELECT'ing a few fields from a table for a large
> nu
Unless there was a way to guarantee consistency, it would be hard at
best to make this work. Convergence on large data sets across boxes is
non-trivial, and diffing databases is difficult at best. Unless there
was some form of automated way to ensure consistency, going 8 ways into
separate boxes is
On 11/15/05, Luke Lonergan <[EMAIL PROTECTED]> wrote:
> Adam,
>
> > -Original Message-
> > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED] On Behalf Of
> > Claus Guttesen
> > Sent: Tuesday, November 15, 2005 12:29 AM
> > To: Adam Weisberg
> > Cc: pgsql-performance@postgresql.org
> > S
Not at random access in RAID 10 they aren't, and anyone with their
head screwed on right is using RAID 10. The 9500S will still beat the
Areca cards at RAID 10 database access patern.
Alex.
On 11/15/05, Dave Cramer <[EMAIL PROTECTED]> wrote:
> Luke,
>
> Have you tried the areca cards, they are s
Title: Re: [PERFORM] Performance PG 8.0 on dual opteron / 4GB / 3ware
Joost,
On 11/15/05 11:51 AM, "Joost Kraaijeveld" <[EMAIL PROTECTED]> wrote:
On Tue, 2005-11-15 at 12:41 -0700, Steve Wampler wrote:
> Joost Kraaijeveld wrote:
> > If I understand correctly (I have 4GB ram):
> >
> > [EMAIL PR
Title: Re: [PERFORM] Hardware/OS recommendations for large databases ( 5TB)
James,
On 11/15/05 11:07 AM, "James Mello" <[EMAIL PROTECTED]> wrote:
Unless there was a way to guarantee consistency, it would be hard at
best to make this work. Convergence on large data sets across boxes is
non-tri
Hi Luke,
On Tue, 2005-11-15 at 22:07 -0800, Luke Lonergan wrote:
> You might update your driver,
I will do that (but I admit that I am not looking forward to it. When I
was young and did not make money with my computer, I liked challenges
like compiling kernels and not being able to boot the com
29 matches
Mail list logo
