Re: [PERFORM] Hardware recommendations

2010-12-13 Thread mark
e: [PERFORM] Hardware recommendations > > > > > -Original Message- > > From: Greg Smith [mailto:g...@2ndquadrant.com] > > Sent: Saturday, December 11, 2010 2:18 AM > > To: Benjamin Krajmalnik > > Cc: pgsql-performance@postgresql.org > > Subject: R

Re: [PERFORM] Hardware recommendations

2010-12-13 Thread Benjamin Krajmalnik
> -Original Message- > From: Greg Smith [mailto:g...@2ndquadrant.com] > Sent: Saturday, December 11, 2010 2:18 AM > To: Benjamin Krajmalnik > Cc: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] Hardware recommendations > > > What sort of total read

Re: [PERFORM] Hardware recommendations

2010-12-11 Thread Greg Smith
Benjamin Krajmalnik wrote: I am already having serious I/O bottlenecks with iostat -x showing extended periods where the disk subsystem on the data partition (the one with all the random i/o) at over 85% busy. The system is running FreeBSD 7.2 amd64 and PostgreSQL 8.4.4 on amd64-portbld-freeb

Re: [PERFORM] Hardware recommendations

2010-12-11 Thread Greg Smith
John W Strange wrote: http://www.fusionio.com/products/iodrive/ - BEST in slot currently IMHO. http://www.intel.com/design/flash/nand/extreme/index.htm?wapkw=(X25-E) - not a bad alternative. The FusionIO drives are OK, so long as you don't mind the possibility that your system will be down

Re: [PERFORM] Hardware recommendations

2010-12-10 Thread Andy
> The "common knowledge" you based that comment on, may > actually not be very up-to-date anymore. Current > consumer-grade SSD's can achieve up to 200MB/sec when > writing sequentially and they can probably do that a lot > more consistent than a hard disk. > > Have a look here: http://www.anandt

Re: [PERFORM] Hardware recommendations

2010-12-10 Thread Scott Marlowe
On Fri, Dec 10, 2010 at 11:05 AM, Arjen van der Meijden wrote: > > On 10-12-2010 18:57 Arjen van der Meijden wrote: >> >> Have a look here: http://www.anandtech.com/show/2829/21 >> The sequential writes-graphs consistently put several SSD's at twice the >> performance of the VelociRaptor 300GB 10k

Re: [PERFORM] Hardware recommendations

2010-12-10 Thread Arjen van der Meijden
On 10-12-2010 18:57 Arjen van der Meijden wrote: Have a look here: http://www.anandtech.com/show/2829/21 The sequential writes-graphs consistently put several SSD's at twice the performance of the VelociRaptor 300GB 10k rpm disk and that's a test from over a year old, current SSD's have increase

Re: [PERFORM] Hardware recommendations

2010-12-10 Thread Arjen van der Meijden
On 10-12-2010 14:58 Andy wrote: We use ZFS and use SSDs for both the log device and L2ARC. All disks and SSDs are behind a 3ware with BBU in single disk mode. Out of curiosity why do you put your log on SSD? Log is all sequential IOs, an area in which SSD is not any faster than HDD. So I'd thi

Re: [PERFORM] Hardware recommendations

2010-12-10 Thread Andy
> We use ZFS and use SSDs for both the log device and > L2ARC.  All disks > and SSDs are behind a 3ware with BBU in single disk > mode.  Out of curiosity why do you put your log on SSD? Log is all sequential IOs, an area in which SSD is not any faster than HDD. So I'd think putting log on SSD

Re: [PERFORM] Hardware recommendations

2010-12-09 Thread alan bryan
On Wed, Dec 8, 2010 at 3:03 PM, Benjamin Krajmalnik wrote: > I need to build a new high performance server to replace our current > production database server. We run FreeBSD 8.1 with PG 8.4 (soon to upgrade to PG 9). Hardware is: Supermicro 2u 6026T-NTR+ 2x Intel Xeon E5520 Nehalem 2.26GHz

Re: [PERFORM] Hardware recommendations

2010-12-09 Thread John W Strange
6:09 AM To: Scott Marlowe Cc: Benjamin Krajmalnik; John W Strange; pgsql-performance@postgresql.org Subject: Re: [PERFORM] Hardware recommendations On Thu, Dec 9, 2010 at 04:28, Scott Marlowe wrote: > On Wed, Dec 8, 2010 at 5:03 PM, Benjamin Krajmalnik > wrote: >> My biggest concern wi

Re: [PERFORM] Hardware recommendations

2010-12-09 Thread Marti Raudsepp
On Thu, Dec 9, 2010 at 04:28, Scott Marlowe wrote: > On Wed, Dec 8, 2010 at 5:03 PM, Benjamin Krajmalnik > wrote: >> My biggest concern with SSD drives is their life expectancy, > > Generally that's not a big issue, especially as the SSDs get larger. > Being able to survive a power loss without

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread mark
-Original Message- From: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance-ow...@postgresql.org] On Behalf Of Andy Sent: Wednesday, December 08, 2010 5:24 PM To: Marti Raudsepp Cc: pgsql-performance@postgresql.org; Benjamin Krajmalnik Subject: Re: [PERFORM] Hardware

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread Scott Marlowe
On Wed, Dec 8, 2010 at 5:03 PM, Benjamin Krajmalnik wrote: > John, > > The platform is a network monitoring system, so we have quite a lot of > inserts/updates (every data point has at least one record insert as well as > at least 3 record updates).  The management GUI has a lot of selects.  We

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread alaricd
Sent from my android device. -Original Message- From: Benjamin Krajmalnik To: pgsql-performance@postgresql.org Sent: Wed, 08 Dec 2010 17:14 Subject: [PERFORM] Hardware recommendations Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP (EHLO mx2.hub.org) (ArGo

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread alaricd
Sent from my android device. -Original Message- From: Benjamin Krajmalnik To: pgsql-performance@postgresql.org Sent: Wed, 08 Dec 2010 17:14 Subject: [PERFORM] Hardware recommendations Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP (EHLO mx2.hub.org) (ArGo

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread alaricd
Sent from my android device. -Original Message- From: Benjamin Krajmalnik To: pgsql-performance@postgresql.org Sent: Wed, 08 Dec 2010 17:14 Subject: [PERFORM] Hardware recommendations Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP (EHLO mx2.hub.org) (ArGo

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread alaricd
Sent from my android device. -Original Message- From: Benjamin Krajmalnik To: pgsql-performance@postgresql.org Sent: Wed, 08 Dec 2010 17:14 Subject: [PERFORM] Hardware recommendations Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP (EHLO mx2.hub.org) (ArGo

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread Andy
> > If you are IO-bound, you might want to consider using > SSD. > > > > A single SSD could easily give you more IOPS than 16 > 15k SAS in RAID 10. > > Are there any that don't risk your data on power loss, AND > are cheaper > than SAS RAID 10? > Vertex 2 Pro has a built-in supercapacitor to s

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread alaricd
Sent from my android device. -Original Message- From: Benjamin Krajmalnik To: pgsql-performance@postgresql.org Sent: Wed, 08 Dec 2010 17:14 Subject: [PERFORM] Hardware recommendations Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP (EHLO mx2.hub.org) (ArGo

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread Marti Raudsepp
On Thu, Dec 9, 2010 at 01:26, Andy wrote: > If you are IO-bound, you might want to consider using SSD. > > A single SSD could easily give you more IOPS than 16 15k SAS in RAID 10. Are there any that don't risk your data on power loss, AND are cheaper than SAS RAID 10? Regards, Marti -- Sent vi

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread John W Strange
Ben, It would help if you could tell us a bit more about the read/write mix and transaction requirements. *IF* you are heavy writes I would suggest moving off the RAID1 configuration to a RAID10 setup. I would highly suggest looking at SLC based solid state drives or if your budget has legs, l

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread Benjamin Krajmalnik
John, The platform is a network monitoring system, so we have quite a lot of inserts/updates (every data point has at least one record insert as well as at least 3 record updates). The management GUI has a lot of selects. We are refactoring the database to some degree to aid in the performanc

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread alaricd
Sent from my android device. -Original Message- From: Benjamin Krajmalnik To: pgsql-performance@postgresql.org Sent: Wed, 08 Dec 2010 17:14 Subject: [PERFORM] Hardware recommendations Received: from mx2.hub.org [200.46.204.254] by mail.pengdows.com with SMTP (EHLO mx2.hub.org) (ArGo

Re: [PERFORM] Hardware recommendations

2010-12-08 Thread Andy
If you are IO-bound, you might want to consider using SSD. A single SSD could easily give you more IOPS than 16 15k SAS in RAID 10. --- On Wed, 12/8/10, Benjamin Krajmalnik wrote: > From: Benjamin Krajmalnik > Subject: [PERFORM] Hardware recommendations > To: pgsql-performance@postgresql.org

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-09-10 Thread Bruce Momjian
Gregory S. Williamson wrote: > Nitpicking -- > > Perhaps the 4th data line is meant to be: > Inserts in separate transactions 2500 inserts/second >^^^ Oh, yes, sorry. It is: > Sorry to be replying late. Here is what I found. > > fsync o

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-09-10 Thread Gregory S. Williamson
: Matt Clark Cc: Ron Johnson; PgSQL Performance ML Subject:Re: [PERFORM] Hardware recommendations to scale to silly load Matt Clark wrote: > > Just a data point, but on my Dual Xeon 2.4Gig machine with a 10k SCSI > > drive I can do 4k inserts/second if I turn fsync off.

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-09-09 Thread Bruce Momjian
Matt Clark wrote: > > Just a data point, but on my Dual Xeon 2.4Gig machine with a 10k SCSI > > drive I can do 4k inserts/second if I turn fsync off. If you have a > > battery-backed controller, you should be able to do the same. (You will > > not need to turn fsync off --- fsync will just be fas

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-09-02 Thread Ron Johnson
On Tue, 2003-09-02 at 11:01, Vivek Khera wrote: > > "AS" == Andrew Sullivan <[EMAIL PROTECTED]> writes: > > AS> On Fri, Aug 29, 2003 at 12:05:03AM -0700, William Yu wrote: > >> We should see a boost when we move to 64-bit Linux and hopefully another > >> one when NUMA for Linux is production-

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-09-02 Thread Greg Stark
Vivek Khera <[EMAIL PROTECTED]> writes: > The front-end small processes get to deal with your dialup customers > trickling down the data since it buffers your backend for you. Huh. Well, I used to think this. But I think I was wrong. I used to have apache proxy servers running in front of the mo

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-09-02 Thread Vivek Khera
> "MC" == Matt Clark <[EMAIL PROTECTED]> writes: MC> And concurrency is very high, because it's a web app, and each MC> httpd has one connection to PG, and there can be hundreds of MC> active httpd processes. Some kind of connection pooling scheme MC> might be in order when there are that man

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-09-02 Thread Vivek Khera
> "AS" == Andrew Sullivan <[EMAIL PROTECTED]> writes: AS> On Fri, Aug 29, 2003 at 12:05:03AM -0700, William Yu wrote: >> We should see a boost when we move to 64-bit Linux and hopefully another >> one when NUMA for Linux is production-stable. AS> According to the people who've worked with SG

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-30 Thread Matt Clark
> Just a data point, but on my Dual Xeon 2.4Gig machine with a 10k SCSI > drive I can do 4k inserts/second if I turn fsync off. If you have a > battery-backed controller, you should be able to do the same. (You will > not need to turn fsync off --- fsync will just be fast because of the > disk dr

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-30 Thread Matt Clark
> SELECT > IF > BEGIN > INSERT >or > UPDATE > COMMIT; > > He says his current h/w peaks at 1/10th that rate. > > My question is: is that current peak rate ("300 inserts/updates > *or* 2500 selects") based upon 1 connection, or many c

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-30 Thread Rod Taylor
> My question is: is that current peak rate ("300 inserts/updates > *or* 2500 selects") based upon 1 connection, or many connections? > With 4 CPUs, and a 4 disk RAID10, I wouldn't be surprised if 4 con- > current connections gives the optimum speed. Optimum number of active workers is probably

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-30 Thread Ron Johnson
On Fri, 2003-08-29 at 21:44, Bruce Momjian wrote: > matt wrote: > > > Are you *sure* about that 3K updates/inserts per second xlates > > > to 10,800,000 per hour. That, my friend, is a WHOLE HECK OF A LOT! > > > > Yup, I know! > > Just a data point, but on my Dual Xeon 2.4Gig machine with

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-30 Thread Bruce Momjian
matt wrote: > > Are you *sure* about that 3K updates/inserts per second xlates > > to 10,800,000 per hour. That, my friend, is a WHOLE HECK OF A LOT! > > Yup, I know! Just a data point, but on my Dual Xeon 2.4Gig machine with a 10k SCSI drive I can do 4k inserts/second if I turn fsync off

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-29 Thread Ron Johnson
On Fri, 2003-08-29 at 11:33, William Yu wrote: > Shridhar Daithankar wrote: [snip] > > I am sure. But is 64 bit environment, Xeon is not the compitition. It's PA-RSC- > > 8700, ultraSparcs, Power series and if possible itanium. > > Well, just because the Opteron is 64-bit doesn't mean it's direct

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-29 Thread William Yu
Shridhar Daithankar wrote: Just a guess here but does a precompiled postgresql for x86 and a x86-64 optimized one makes difference? > > Opteron is one place on earth you can watch difference between 32/64 > bit on same machine. Can be handy at times.. I don't know yet. I tried building a 64-bit ke

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-29 Thread Andrew Sullivan
On Fri, Aug 29, 2003 at 12:05:03AM -0700, William Yu wrote: > We should see a boost when we move to 64-bit Linux and hopefully another > one when NUMA for Linux is production-stable. According to the people who've worked with SGIs, NUMA actually seems to make things worse. It has something to do

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-29 Thread Ron Johnson
On Fri, 2003-08-29 at 03:18, Shridhar Daithankar wrote: > On 29 Aug 2003 at 0:05, William Yu wrote: > > > Shridhar Daithankar wrote: [snip] > > As for performance, the scaling is magnificient -- even when just using > > PAE instead of 64-bit addressing. At low transaction counts, it's only > > ~

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-29 Thread Shridhar Daithankar
On 29 Aug 2003 at 0:05, William Yu wrote: > Shridhar Daithankar wrote: > >> Be careful here, we've seen that with the P4 Xeon's that are > >>hyper-threaded and a system that has very high disk I/O causes the > >>system to be sluggish and slow. But after disabling the hyper-threading > >>itself, ou

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-29 Thread Christopher Kings-Lynne
> We should see a boost when we move to 64-bit Linux and hopefully another > one when NUMA for Linux is production-stable. Assuming SCO doesn't make them remove it :P Chris ---(end of broadcast)--- TIP 8: explain analyze is your friend

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-29 Thread William Yu
Shridhar Daithankar wrote: Be careful here, we've seen that with the P4 Xeon's that are hyper-threaded and a system that has very high disk I/O causes the system to be sluggish and slow. But after disabling the hyper-threading itself, our system flew.. Anybody has opteron working? Hows' the perform

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Rod Taylor
On Thu, 2003-08-28 at 12:37, Matt Clark wrote: > > Ok.. I would be surprised if you needed much more actual CPU power. I > > suspect they're mostly idle waiting on data -- especially with a Quad > > Xeon (shared memory bus is it not?). > > In reality the CPUs get pegged: about 65% PG and 35% syste

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Matt Clark
> Ok.. I would be surprised if you needed much more actual CPU power. I > suspect they're mostly idle waiting on data -- especially with a Quad > Xeon (shared memory bus is it not?). In reality the CPUs get pegged: about 65% PG and 35% system. But I agree that memory throughput and latency is an

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Matt Clark
> Just how big do you expect your DB to grow? For a 1GB disk-space > database, I'd probably just splurge for an SSD hooked up either via > SCSI or FibreChannel. Heck, up to about 5Gb or so it is not that > expensive (about $25k) and adding another 5Gb should set you back > probably another $20k.

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Andrew Sullivan
On Wed, Aug 27, 2003 at 05:49:25PM +0100, matt wrote: > > I'm also looking at renting equipment, or even trying out IBM/HP's > 'on-demand' offerings. To handle that kind of load, you're not going to be able to do it with cheap hardware. Renting may be your answer. a -- Andrew Sullivan

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread scott.marlowe
On 27 Aug 2003, matt wrote: > > You probably, more than anything, should look at some kind of > > superfast, external storage array > > Yeah, I think that's going to be a given. Low end EMC FibreChannel > boxes can do around 20,000 IOs/sec, which is probably close to good > enough. > > You men

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Rod Taylor
> I need to increase the overall performance by a factor of 10, while at > the same time the DB size increases by a factor of 50. e.g. 3000 > inserts/updates or 25,000 selects per second, over a 25GB database with > most used tables of 5,000,000 and 1,000,000 rows. Ok.. I would be surprised if yo

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Vivek Khera
sm> On 27 Aug 2003, matt wrote: >> My app is likely to come under some serious load in the next 6 months, >> but the increase will be broadly predictable, so there is time to throw >> hardware at the problem. >> >> Currently I have a ~1GB DB, with the largest (and most commonly accessed >> and u

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Shridhar Daithankar
On 28 Aug 2003 at 11:05, Chris Bowlby wrote: > On Tue, 2003-08-26 at 23:59, Ron Johnson wrote: > > > What a fun exercises. Ok, lets see: > > Postgres 7.3.4 > > RH AS 2.1 > > 12GB RAM > > motherboard with 64 bit 66MHz PCI slots > > 4 - Xenon 3.0GHz (1MB cache) CPUs > > 8 - 36GB 15K RPM as RAID10

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Chris Bowlby
On Tue, 2003-08-26 at 23:59, Ron Johnson wrote: > What a fun exercises. Ok, lets see: > Postgres 7.3.4 > RH AS 2.1 > 12GB RAM > motherboard with 64 bit 66MHz PCI slots > 4 - Xenon 3.0GHz (1MB cache) CPUs > 8 - 36GB 15K RPM as RAID10 on a 64 bit 66MHz U320 controller > having 512MB cache (for

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Ron Johnson
On Thu, 2003-08-28 at 03:17, matt wrote: > > Are you *sure* about that 3K updates/inserts per second xlates > > to 10,800,000 per hour. That, my friend, is a WHOLE HECK OF A LOT! > > Yup, I know! > > > During the 1 hour surge, will SELECTs at 10 minutes after the > > hour depend on INSER

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
> Are you *sure* about that 3K updates/inserts per second xlates > to 10,800,000 per hour. That, my friend, is a WHOLE HECK OF A LOT! Yup, I know! > During the 1 hour surge, will SELECTs at 10 minutes after the > hour depend on INSERTs at 5 minutes after the hour? Yes, they do. It's a

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Tomka Gergely
2003-08-27 ragyogó napján Bill Moran ezt üzente: > With FreeBSD, you have jails, which allow multiple users to share > hardware without having to worry about user A looking at user B's > stuff. Does such a paradigm exist on any heavy iron? I have no Of course. All IBM hw can do this, because on

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Tomka Gergely
2003-08-27 ragyogó napján matt ezt üzente: > Yeah, I can imagine getting 5% extra from a slim kernel and > super-optimised PG. Hm, about 20%, but only for the correctness - 20% not help you also :( > The FS is ext3, metadata journaling (the default), mounted noatime. Worst fs under linux :) Try

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Ron Johnson
On Wed, 2003-08-27 at 21:26, Bill Moran wrote: > Christopher Browne wrote: > > Martha Stewart called it a Good Thing [EMAIL PROTECTED] (matt)wrote: [snip] > With FreeBSD, you have jails, which allow multiple users to share > hardware without having to worry about user A looking at user B's > stuff.

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Bill Moran
matt wrote: I'm wondering if the good people out there could perhaps give me some pointers on suitable hardware to solve an upcoming performance issue. I've never really dealt with these kinds of loads before, so any experience you guys have would be invaluable. Apologies in advance for the amoun

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Ron Johnson
On Tue, 2003-08-26 at 20:35, matt wrote: > I'm wondering if the good people out there could perhaps give me some > pointers on suitable hardware to solve an upcoming performance issue. > I've never really dealt with these kinds of loads before, so any > experience you guys have would be invaluable

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Bill Moran
Christopher Browne wrote: Martha Stewart called it a Good Thing [EMAIL PROTECTED] (matt)wrote: I'm also looking at renting equipment, or even trying out IBM/HP's 'on-demand' offerings. You're assuming that this is likely to lead to REAL savings, and that seems unlikely. During the recent power out

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Christopher Browne
After takin a swig o' Arrakan spice grog, [EMAIL PROTECTED] ("scott.marlowe") belched out... :-): > whether you like it or not, you're gonna need heavy iron if you need > to do this all in one hour once a week. The other thing worth considering is trying to see if there is a way of partitioning th

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Andrew Sullivan
On Wed, Aug 27, 2003 at 02:35:13AM +0100, matt wrote: > I need to increase the overall performance by a factor of 10, while at > the same time the DB size increases by a factor of 50. e.g. 3000 > inserts/updates or 25,000 selects per second, over a 25GB database with > most used tables of 5,000,0

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
> Are you sure? Have you tested the overall application to see if possibly > you gain more on insert performance than you lose on select performanc? Unfortunately dropping any of the indexes results in much worse select performance that is not remotely clawed back by the improvement in insert per

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread Christopher Browne
Martha Stewart called it a Good Thing [EMAIL PROTECTED] (matt)wrote: > I'm also looking at renting equipment, or even trying out IBM/HP's > 'on-demand' offerings. You're assuming that this is likely to lead to REAL savings, and that seems unlikely. During the recent power outage in the NorthEast,

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
> Don't know how "cheap" they are. > > I have an app that does large batch updates. I found that if I dropped > the indexes, did the updates and recreated the indexes, it was faster > than doing the updates while the indexes were intact. Yeah, unfortunately it's not batch work, but real time fina

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread scott.marlowe
On 27 Aug 2003, matt wrote: > I'm wondering if the good people out there could perhaps give me some > pointers on suitable hardware to solve an upcoming performance issue. > I've never really dealt with these kinds of loads before, so any > experience you guys have would be invaluable. Apologies

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
> You probably, more than anything, should look at some kind of > superfast, external storage array Yeah, I think that's going to be a given. Low end EMC FibreChannel boxes can do around 20,000 IOs/sec, which is probably close to good enough. You mentioned using multiple RAID controllers as a b

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-27 Thread Bill Moran
matt wrote: Are you sure? Have you tested the overall application to see if possibly you gain more on insert performance than you lose on select performanc? Unfortunately dropping any of the indexes results in much worse select performance that is not remotely clawed back by the improvement in ins