-Original Message-
From: Greg Smith [mailto:g...@2ndquadrant.com]
Sent: Saturday, December 11, 2010 2:18 AM
To: Benjamin Krajmalnik
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Hardware recommendations
What sort of total read/write rates are you seeing when
recommendations
-Original Message-
From: Greg Smith [mailto:g...@2ndquadrant.com]
Sent: Saturday, December 11, 2010 2:18 AM
To: Benjamin Krajmalnik
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Hardware recommendations
Have you increased checkpoint parameters like
John W Strange wrote:
http://www.fusionio.com/products/iodrive/ - BEST in slot currently IMHO.
http://www.intel.com/design/flash/nand/extreme/index.htm?wapkw=(X25-E) - not a
bad alternative.
The FusionIO drives are OK, so long as you don't mind the possibility
that your system will be
Benjamin Krajmalnik wrote:
I am already having serious I/O bottlenecks with iostat -x showing extended
periods where the disk subsystem on the data partition (the one with all the
random i/o) at over 85% busy. The system is running FreeBSD 7.2 amd64 and
PostgreSQL 8.4.4 on
We use ZFS and use SSDs for both the log device and
L2ARC. All disks
and SSDs are behind a 3ware with BBU in single disk
mode.
Out of curiosity why do you put your log on SSD? Log is all sequential IOs, an
area in which SSD is not any faster than HDD. So I'd think putting log on SSD
On 10-12-2010 14:58 Andy wrote:
We use ZFS and use SSDs for both the log device and L2ARC. All
disks and SSDs are behind a 3ware with BBU in single disk mode.
Out of curiosity why do you put your log on SSD? Log is all
sequential IOs, an area in which SSD is not any faster than HDD. So
I'd
On 10-12-2010 18:57 Arjen van der Meijden wrote:
Have a look here: http://www.anandtech.com/show/2829/21
The sequential writes-graphs consistently put several SSD's at twice the
performance of the VelociRaptor 300GB 10k rpm disk and that's a test
from over a year old, current SSD's have
On Fri, Dec 10, 2010 at 11:05 AM, Arjen van der Meijden
acmmail...@tweakers.net wrote:
On 10-12-2010 18:57 Arjen van der Meijden wrote:
Have a look here: http://www.anandtech.com/show/2829/21
The sequential writes-graphs consistently put several SSD's at twice the
performance of the
The common knowledge you based that comment on, may
actually not be very up-to-date anymore. Current
consumer-grade SSD's can achieve up to 200MB/sec when
writing sequentially and they can probably do that a lot
more consistent than a hard disk.
Have a look here:
On Thu, Dec 9, 2010 at 04:28, Scott Marlowe scott.marl...@gmail.com wrote:
On Wed, Dec 8, 2010 at 5:03 PM, Benjamin Krajmalnik k...@servoyant.com
wrote:
My biggest concern with SSD drives is their life expectancy,
Generally that's not a big issue, especially as the SSDs get larger.
Being
To: Scott Marlowe
Cc: Benjamin Krajmalnik; John W Strange; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Hardware recommendations
On Thu, Dec 9, 2010 at 04:28, Scott Marlowe scott.marl...@gmail.com wrote:
On Wed, Dec 8, 2010 at 5:03 PM, Benjamin Krajmalnik k...@servoyant.com
wrote:
My
On Wed, Dec 8, 2010 at 3:03 PM, Benjamin Krajmalnik k...@servoyant.com wrote:
I need to build a new high performance server to replace our current
production database server.
We run FreeBSD 8.1 with PG 8.4 (soon to upgrade to PG 9). Hardware is:
Supermicro 2u 6026T-NTR+
2x Intel Xeon E5520
I need to build a new high performance server to replace our current production
database server.
The current server is a SuperMicro 1U with 2 RAID-1 containers (one for data,
one for log, SAS - data is 600GB, Logs 144GB), 16GB of RAM, running 2 quad core
processors (E5405 @ 2GHz), Adaptec 5405
If you are IO-bound, you might want to consider using SSD.
A single SSD could easily give you more IOPS than 16 15k SAS in RAID 10.
--- On Wed, 12/8/10, Benjamin Krajmalnik k...@servoyant.com wrote:
From: Benjamin Krajmalnik k...@servoyant.com
Subject: [PERFORM] Hardware recommendations
Sent from my android device.
-Original Message-
From: Benjamin Krajmalnik k...@servoyant.com
To: pgsql-performance@postgresql.org
Sent: Wed, 08 Dec 2010 17:14
Subject: [PERFORM] Hardware recommendations
Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP
(EHLO
-performance-ow...@postgresql.org [mailto:pgsql-performance-
ow...@postgresql.org] On Behalf Of Benjamin Krajmalnik
Sent: Wednesday, December 08, 2010 5:04 PM
To: pgsql-performance@postgresql.org
Subject: [PERFORM] Hardware recommendations
I need to build a new high performance server
-Original Message-
From: pgsql-performance-ow...@postgresql.org
[mailto:pgsql-performance-ow...@postgresql.org] On Behalf Of Benjamin Krajmalnik
Sent: Wednesday, December 08, 2010 5:04 PM
To: pgsql-performance@postgresql.org
Subject: [PERFORM] Hardware recommendations
I need to build a new
On Thu, Dec 9, 2010 at 01:26, Andy angelf...@yahoo.com wrote:
If you are IO-bound, you might want to consider using SSD.
A single SSD could easily give you more IOPS than 16 15k SAS in RAID 10.
Are there any that don't risk your data on power loss, AND are cheaper
than SAS RAID 10?
Regards,
Sent from my android device.
-Original Message-
From: Benjamin Krajmalnik k...@servoyant.com
To: pgsql-performance@postgresql.org
Sent: Wed, 08 Dec 2010 17:14
Subject: [PERFORM] Hardware recommendations
Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP
(EHLO
If you are IO-bound, you might want to consider using
SSD.
A single SSD could easily give you more IOPS than 16
15k SAS in RAID 10.
Are there any that don't risk your data on power loss, AND
are cheaper
than SAS RAID 10?
Vertex 2 Pro has a built-in supercapacitor to save data on
Sent from my android device.
-Original Message-
From: Benjamin Krajmalnik k...@servoyant.com
To: pgsql-performance@postgresql.org
Sent: Wed, 08 Dec 2010 17:14
Subject: [PERFORM] Hardware recommendations
Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP
(EHLO
Sent from my android device.
-Original Message-
From: Benjamin Krajmalnik k...@servoyant.com
To: pgsql-performance@postgresql.org
Sent: Wed, 08 Dec 2010 17:14
Subject: [PERFORM] Hardware recommendations
Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP
(EHLO
Sent from my android device.
-Original Message-
From: Benjamin Krajmalnik k...@servoyant.com
To: pgsql-performance@postgresql.org
Sent: Wed, 08 Dec 2010 17:14
Subject: [PERFORM] Hardware recommendations
Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP
(EHLO
Sent from my android device.
-Original Message-
From: Benjamin Krajmalnik k...@servoyant.com
To: pgsql-performance@postgresql.org
Sent: Wed, 08 Dec 2010 17:14
Subject: [PERFORM] Hardware recommendations
Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP
(EHLO
On Wed, Dec 8, 2010 at 5:03 PM, Benjamin Krajmalnik k...@servoyant.com wrote:
John,
The platform is a network monitoring system, so we have quite a lot of
inserts/updates (every data point has at least one record insert as well as
at least 3 record updates). The management GUI has a lot of
-Original Message-
From: pgsql-performance-ow...@postgresql.org
[mailto:pgsql-performance-ow...@postgresql.org] On Behalf Of Andy
Sent: Wednesday, December 08, 2010 5:24 PM
To: Marti Raudsepp
Cc: pgsql-performance@postgresql.org; Benjamin Krajmalnik
Subject: Re: [PERFORM] Hardware
To: Matt Clark
Cc: Ron Johnson; PgSQL Performance ML
Subject:Re: [PERFORM] Hardware recommendations to scale to silly load
Matt Clark wrote:
Just a data point, but on my Dual Xeon 2.4Gig machine with a 10k SCSI
drive I can do 4k inserts/second if I turn fsync off. If you have
AS == Andrew Sullivan [EMAIL PROTECTED] writes:
AS On Fri, Aug 29, 2003 at 12:05:03AM -0700, William Yu wrote:
We should see a boost when we move to 64-bit Linux and hopefully another
one when NUMA for Linux is production-stable.
AS According to the people who've worked with SGIs, NUMA
MC == Matt Clark [EMAIL PROTECTED] writes:
MC And concurrency is very high, because it's a web app, and each
MC httpd has one connection to PG, and there can be hundreds of
MC active httpd processes. Some kind of connection pooling scheme
MC might be in order when there are that many active
Vivek Khera [EMAIL PROTECTED] writes:
The front-end small processes get to deal with your dialup customers
trickling down the data since it buffers your backend for you.
Huh. Well, I used to think this. But I think I was wrong. I used to have
apache proxy servers running in front of the
matt wrote:
Are you *sure* about that 3K updates/inserts per second xlates
to 10,800,000 per hour. That, my friend, is a WHOLE HECK OF A LOT!
Yup, I know!
Just a data point, but on my Dual Xeon 2.4Gig machine with a 10k SCSI
drive I can do 4k inserts/second if I turn fsync off. If
My question is: is that current peak rate (300 inserts/updates
*or* 2500 selects) based upon 1 connection, or many connections?
With 4 CPUs, and a 4 disk RAID10, I wouldn't be surprised if 4 con-
current connections gives the optimum speed.
Optimum number of active workers is probably
Shridhar Daithankar wrote:
Be careful here, we've seen that with the P4 Xeon's that are
hyper-threaded and a system that has very high disk I/O causes the
system to be sluggish and slow. But after disabling the hyper-threading
itself, our system flew..
Anybody has opteron working? Hows' the
We should see a boost when we move to 64-bit Linux and hopefully another
one when NUMA for Linux is production-stable.
Assuming SCO doesn't make them remove it :P
Chris
---(end of broadcast)---
TIP 8: explain analyze is your friend
On 29 Aug 2003 at 0:05, William Yu wrote:
Shridhar Daithankar wrote:
Be careful here, we've seen that with the P4 Xeon's that are
hyper-threaded and a system that has very high disk I/O causes the
system to be sluggish and slow. But after disabling the hyper-threading
itself, our system
On Fri, 2003-08-29 at 03:18, Shridhar Daithankar wrote:
On 29 Aug 2003 at 0:05, William Yu wrote:
Shridhar Daithankar wrote:
[snip]
As for performance, the scaling is magnificient -- even when just using
PAE instead of 64-bit addressing. At low transaction counts, it's only
~75%
On Fri, Aug 29, 2003 at 12:05:03AM -0700, William Yu wrote:
We should see a boost when we move to 64-bit Linux and hopefully another
one when NUMA for Linux is production-stable.
According to the people who've worked with SGIs, NUMA actually seems
to make things worse. It has something to do
Shridhar Daithankar wrote:
Just a guess here but does a precompiled postgresql for x86 and a x86-64
optimized one makes difference?
Opteron is one place on earth you can watch difference between 32/64
bit on same machine. Can be handy at times..
I don't know yet. I tried building a 64-bit
On Fri, 2003-08-29 at 11:33, William Yu wrote:
Shridhar Daithankar wrote:
[snip]
I am sure. But is 64 bit environment, Xeon is not the compitition. It's PA-RSC-
8700, ultraSparcs, Power series and if possible itanium.
Well, just because the Opteron is 64-bit doesn't mean it's direct
You probably, more than anything, should look at some kind of
superfast, external storage array
Yeah, I think that's going to be a given. Low end EMC FibreChannel
boxes can do around 20,000 IOs/sec, which is probably close to good
enough.
You mentioned using multiple RAID controllers as a
On 27 Aug 2003, matt wrote:
I'm wondering if the good people out there could perhaps give me some
pointers on suitable hardware to solve an upcoming performance issue.
I've never really dealt with these kinds of loads before, so any
experience you guys have would be invaluable. Apologies in
Don't know how cheap they are.
I have an app that does large batch updates. I found that if I dropped
the indexes, did the updates and recreated the indexes, it was faster
than doing the updates while the indexes were intact.
Yeah, unfortunately it's not batch work, but real time financial
Martha Stewart called it a Good Thing [EMAIL PROTECTED] (matt)wrote:
I'm also looking at renting equipment, or even trying out IBM/HP's
'on-demand' offerings.
You're assuming that this is likely to lead to REAL savings, and that
seems unlikely.
During the recent power outage in the NorthEast,
Are you sure? Have you tested the overall application to see if possibly
you gain more on insert performance than you lose on select performanc?
Unfortunately dropping any of the indexes results in much worse select
performance that is not remotely clawed back by the improvement in
insert
On Wed, Aug 27, 2003 at 02:35:13AM +0100, matt wrote:
I need to increase the overall performance by a factor of 10, while at
the same time the DB size increases by a factor of 50. e.g. 3000
inserts/updates or 25,000 selects per second, over a 25GB database with
most used tables of 5,000,000
After takin a swig o' Arrakan spice grog, [EMAIL PROTECTED]
(scott.marlowe) belched out... :-):
whether you like it or not, you're gonna need heavy iron if you need
to do this all in one hour once a week.
The other thing worth considering is trying to see if there is a way
of partitioning the
Christopher Browne wrote:
Martha Stewart called it a Good Thing [EMAIL PROTECTED] (matt)wrote:
I'm also looking at renting equipment, or even trying out IBM/HP's
'on-demand' offerings.
You're assuming that this is likely to lead to REAL savings, and that
seems unlikely.
During the recent power
On Tue, 2003-08-26 at 20:35, matt wrote:
I'm wondering if the good people out there could perhaps give me some
pointers on suitable hardware to solve an upcoming performance issue.
I've never really dealt with these kinds of loads before, so any
experience you guys have would be invaluable.
matt wrote:
I'm wondering if the good people out there could perhaps give me some
pointers on suitable hardware to solve an upcoming performance issue.
I've never really dealt with these kinds of loads before, so any
experience you guys have would be invaluable. Apologies in advance for
the
On Wed, 2003-08-27 at 21:26, Bill Moran wrote:
Christopher Browne wrote:
Martha Stewart called it a Good Thing [EMAIL PROTECTED] (matt)wrote:
[snip]
With FreeBSD, you have jails, which allow multiple users to share
hardware without having to worry about user A looking at user B's
stuff.
2003-08-27 ragyog napjn matt ezt zente:
Yeah, I can imagine getting 5% extra from a slim kernel and
super-optimised PG.
Hm, about 20%, but only for the correctness - 20% not help you also :(
The FS is ext3, metadata journaling (the default), mounted noatime.
Worst fs under linux :) Try xfs.
Are you *sure* about that 3K updates/inserts per second xlates
to 10,800,000 per hour. That, my friend, is a WHOLE HECK OF A LOT!
Yup, I know!
During the 1 hour surge, will SELECTs at 10 minutes after the
hour depend on INSERTs at 5 minutes after the hour?
Yes, they do. It's a
On Tue, 2003-08-26 at 23:59, Ron Johnson wrote:
What a fun exercises. Ok, lets see:
Postgres 7.3.4
RH AS 2.1
12GB RAM
motherboard with 64 bit 66MHz PCI slots
4 - Xenon 3.0GHz (1MB cache) CPUs
8 - 36GB 15K RPM as RAID10 on a 64 bit 66MHz U320 controller
having 512MB cache (for
On 28 Aug 2003 at 11:05, Chris Bowlby wrote:
On Tue, 2003-08-26 at 23:59, Ron Johnson wrote:
What a fun exercises. Ok, lets see:
Postgres 7.3.4
RH AS 2.1
12GB RAM
motherboard with 64 bit 66MHz PCI slots
4 - Xenon 3.0GHz (1MB cache) CPUs
8 - 36GB 15K RPM as RAID10 on a 64 bit
sm On 27 Aug 2003, matt wrote:
My app is likely to come under some serious load in the next 6 months,
but the increase will be broadly predictable, so there is time to throw
hardware at the problem.
Currently I have a ~1GB DB, with the largest (and most commonly accessed
and updated) two
I need to increase the overall performance by a factor of 10, while at
the same time the DB size increases by a factor of 50. e.g. 3000
inserts/updates or 25,000 selects per second, over a 25GB database with
most used tables of 5,000,000 and 1,000,000 rows.
Ok.. I would be surprised if you
On 27 Aug 2003, matt wrote:
You probably, more than anything, should look at some kind of
superfast, external storage array
Yeah, I think that's going to be a given. Low end EMC FibreChannel
boxes can do around 20,000 IOs/sec, which is probably close to good
enough.
You mentioned
Just how big do you expect your DB to grow? For a 1GB disk-space
database, I'd probably just splurge for an SSD hooked up either via
SCSI or FibreChannel. Heck, up to about 5Gb or so it is not that
expensive (about $25k) and adding another 5Gb should set you back
probably another $20k. I
Ok.. I would be surprised if you needed much more actual CPU power. I
suspect they're mostly idle waiting on data -- especially with a Quad
Xeon (shared memory bus is it not?).
In reality the CPUs get pegged: about 65% PG and 35% system. But I agree that memory
throughput and latency is an
On Thu, 2003-08-28 at 12:37, Matt Clark wrote:
Ok.. I would be surprised if you needed much more actual CPU power. I
suspect they're mostly idle waiting on data -- especially with a Quad
Xeon (shared memory bus is it not?).
In reality the CPUs get pegged: about 65% PG and 35% system. But
I'm wondering if the good people out there could perhaps give me some
pointers on suitable hardware to solve an upcoming performance issue.
I've never really dealt with these kinds of loads before, so any
experience you guys have would be invaluable. Apologies in advance for
the amount of info
matt wrote:
Are you sure? Have you tested the overall application to see if possibly
you gain more on insert performance than you lose on select performanc?
Unfortunately dropping any of the indexes results in much worse select
performance that is not remotely clawed back by the improvement in
62 matches
Mail list logo
