Re: [PERFORM] PostgreSQL data on a NAS device ?
I have never worked with a XEON CPU before. Does anyone know how it performs running PostgreSQL 7.3.4 / 7.4 on RedHat 9 ? Is it faster than a Pentium 4? I believe the main difference is cache memory, right? Aside from cache mem, it's basically a Pentium 4, or am I wrong? Well, see the problem is of course, there's so many flavors of P4s and Xeons that it's hard to tell which is faster unless you specify the exact model. And even then, it would depend on the workload. Would a Xeon/3GHz/2MB L3/400FSB be faster than a P4C/3GHz/800FSB? No idea as no one has complete number breakdowns on these comparisons. Oh yeah, you could get a big round number that says on SPEC or something one CPU is faster than the other but whether that's faster for Postgres and your PG app is a totally different story. That in mind, I wouldn't worry about it. The CPU is probably plenty fast for what you need to do. I'd look into two things in the server: memory and CPU expandability. I know you already plan on 4GB but you may need even more in the future. Few things can dramatically improve performance more than moving disk access to disk cache. And if there's a 2nd socket where you can pop another CPU in, that would leave you extra room if your server becomes CPU limited. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] PostgreSQL data on a NAS device ?
Just an interesting comparison: I don't have the specifics, but a Dell 2 x 2.4GHZ/512KB L3 / 2GB RAM machine timed a query much faster than an older Sun E4000 with 6 x ~300MHZ CPUs / 2GB RAM. One on RH(8 or 9, don't remember) and one on Solaris 9. -anjan -Original Message- From: William Yu [mailto:[EMAIL PROTECTED] Sent: Tue 10/21/2003 12:12 PM To: [EMAIL PROTECTED] Cc: Subject: Re: [PERFORM] PostgreSQL data on a NAS device ? I have never worked with a XEON CPU before. Does anyone know how it performs running PostgreSQL 7.3.4 / 7.4 on RedHat 9 ? Is it faster than a Pentium 4? I believe the main difference is cache memory, right? Aside from cache mem, it's basically a Pentium 4, or am I wrong? Well, see the problem is of course, there's so many flavors of P4s and Xeons that it's hard to tell which is faster unless you specify the exact model. And even then, it would depend on the workload. Would a Xeon/3GHz/2MB L3/400FSB be faster than a P4C/3GHz/800FSB? No idea as no one has complete number breakdowns on these comparisons. Oh yeah, you could get a big round number that says on SPEC or something one CPU is faster than the other but whether that's faster for Postgres and your PG app is a totally different story. That in mind, I wouldn't worry about it. The CPU is probably plenty fast for what you need to do. I'd look into two things in the server: memory and CPU expandability. I know you already plan on 4GB but you may need even more in the future. Few things can dramatically improve performance more than moving disk access to disk cache. And if there's a 2nd socket where you can pop another CPU in, that would leave you extra room if your server becomes CPU limited. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] PostgreSQL data on a NAS device ?
The machine is going to be used for a pretty large database (well over 100 tables with some of them containing over a million records from the start, number of tables and records will grow (much?) larger in the future). This database is going to be used by a pretty large number of employees. The number of concurrent users will vary between 1 - 100 or so, depending on the time of day etc. This will be a database containing client and supplier data as well as product descriptions and prices/ingredients/labels/brands etc. Database use will include lots of SELECTS but also lots of INSERTS/UPDATES, i.e. the database will be pretty active during bussiness hours... I think you (Scott and Will) are right when you say that NAS devices are not ideal for this kind of thing. I have been thinking about the hardware configuration for this machine for some time now (and had a lot of hints through this list already) and decided to go for a SCSI RAID config after all. The extra costs will be worth it :) The machine I have in mind now is like this : Dell PowerEdge 1750 machine with Intel Xeon CPU at 3 GHz and 4 GB of RAM. This machine will contain a PERC4/Di RAID controller with 128MB of battery backed cache memory. The O/S and logfiles will be placed on a RAID-1 setup of two 36Gb SCSI-U320 drives (15.000rpm). Database data will be placed on a Dell PowerVault 220S rack-module containing six 36Gb SCSI-U320 drives (15.000rpm) in a RAID-10 setup. This PowerVault will be connected to the DB server via a SCSI cable... This machine will be a bit more expensive than I thought at first (it's going to be about EUR 14.000, but that's including 3 years of on-site support from Dell (24x7, 4-hour response) and peripherals like UPS etc... Do you think this machine wil be OK for this task? Thanks for your help so far :) Kind regards, Alexander Priem. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] PostgreSQL data on a NAS device ?
Alexander Priem wrote: Dell PowerEdge 1750 machine with Intel Xeon CPU at 3 GHz and 4 GB of RAM. This machine will contain a PERC4/Di RAID controller with 128MB of battery backed cache memory. The O/S and logfiles will be placed on a RAID-1 setup of two 36Gb SCSI-U320 drives (15.000rpm). Database data will be placed on a Dell PowerVault 220S rack-module containing six 36Gb SCSI-U320 drives (15.000rpm) in a RAID-10 setup. This PowerVault will be connected to the DB server via a SCSI cable... This machine will be a bit more expensive than I thought at first (it's going to be about EUR 14.000, but that's including 3 years of on-site support from Dell (24x7, 4-hour response) and peripherals like UPS etc... Check opteron as well.. I don't know much about european resellers. IBM sells eserver 325 which has opterons. Apparently they scale much better at higher load. Of course pricing,availability and support are most important. http://theregister.co.uk/content/61/33378.html http://www.pc.ibm.com/us/eserver/opteron/325/ Any concrete benchmarks for postgresql w.r.t xeons and opterons? A collection would be nice to have..:-) Shridhar ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] PostgreSQL data on a NAS device ?
I have considered Opteron, yes. But I think there are too many uncertainties, like : * It's a new CPU that has not proven itself yet. * I don't think I can buy directly from IBM (according to their site), so how about support (24x7) ? This will be very important to our client. * I need to install and configure a 64bit Linux flavour which I don't know (yet) Any suggestions about the usability of the system I described before? Here is the description again: Dell PowerEdge 1750 machine with Intel Xeon CPU at 3 GHz and 4 GB of RAM. This machine will contain a PERC4/Di RAID controller with 128MB of battery backed cache memory. The O/S and logfiles will be placed on a RAID-1 setup of two 36Gb SCSI-U320 drives (15.000rpm). Database data will be placed on a Dell PowerVault 220S rack-module containing six 36Gb SCSI-U320 drives (15.000rpm) in a RAID-10 setup. This PowerVault will be connected to the DB server via a SCSI cable... I have never worked with a XEON CPU before. Does anyone know how it performs running PostgreSQL 7.3.4 / 7.4 on RedHat 9 ? Is it faster than a Pentium 4? I believe the main difference is cache memory, right? Aside from cache mem, it's basically a Pentium 4, or am I wrong? Kind regards, Alexander. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] PostgreSQL data on a NAS device ?
Alexander Priem wrote: I have considered Opteron, yes. But I think there are too many uncertainties, like : * It's a new CPU that has not proven itself yet. * I don't think I can buy directly from IBM (according to their site), so how about support (24x7) ? This will be very important to our client. * I need to install and configure a 64bit Linux flavour which I don't know (yet) See http://www.monarchcomputer.com/ they custom build operton systems and preload them with Linux. You don't pay the Microsoft tax. -- Until later, Geoffrey [EMAIL PROTECTED] ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] PostgreSQL data on a NAS device ?
On Tue, 2003-10-21 at 08:40, scott.marlowe wrote: SNIP So that brings up my question, which is better, the Perc4 or Perc3 controllers, and what's the difference between them? I find Dell's tendency to hide other people's hardware behind their own model numbers mildly bothersome, as it makes it hard to comparison shop. Perc4 has n LSI 1030 chip http://docs.us.dell.com/docs/storage/perc4di/en/ug/features.htm Perc3 depending on the model can be a couple of things but I think they are all U160 controllers and not U320 SNIP Will signature.asc Description: This is a digitally signed message part
[PERFORM] PostgreSQL data on a NAS device ?
Hi all, Does anyone have any experience with putting PostgreSQL data on a NAS device? I am asking this because a NAS device is much cheaper to set up than a couple of SCSI disks. I would like to use a relatively cheap NAS device which uses four IDE drives (7.200 rpm), like the Dell PowerVault 725N. The disks themselves would be much slower than SCSI disks, I know, but a NAS device can be equipped with 3 Gb of memory, so this would make a very large disk cache, right? If this NAS would be dedicated only to PostgreSQL, would this be slower/faster than a SCSI RAID-10 setup of 6 disks? It would be much cheaper... Any advice on this would be appreciated :) Kind regards, Alexander Priem. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PERFORM] PostgreSQL data on a NAS device ?
On Mon, 20 Oct 2003 09:12:35 +0200 Alexander Priem [EMAIL PROTECTED] wrote: I am asking this because a NAS device is much cheaper to set up than a couple of SCSI disks. I would like to use a relatively cheap NAS device which uses four IDE drives (7.200 rpm), like the Dell PowerVault 725N. The disks themselves would be much slower than SCSI disks, I know, but a NAS device can be equipped with 3 Gb of memory, so this would make a very large disk cache, right? If this NAS would be dedicated only to PostgreSQL, would this be slower/faster than a SCSI RAID-10 setup of 6 disks? It would be much cheaper... The big concern would be the network connection, unless you are going fiber. You need to use _AT LEAST_ gigabit. _at least_.If you do go that route it'd be interesting to see bonnie results. And the other thing - remember that just because you are running NAS doesn't mean you can attach another machine running postgres and have a cluster. (See archives for more info about this). I suppose it all boils down to your budget (I usually get to work with a budget of $0). And I mentioned this in another post- If you don't mind refurb disks(or slightly used) check out ebay - you can get scsi disks by the truckload for cheap. -- Jeff Trout [EMAIL PROTECTED] http://www.jefftrout.com/ http://www.stuarthamm.net/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] PostgreSQL data on a NAS device ?
Thanks for your reply, Jeff. If we are going to use a NAS device for storage, then it will be attached through a gigabit ethernet connection. Fiber will not be an option, since that would negate the savings we can make by using an IDE NAS device instead of SCSI-RAID, fiber's pretty expensive, right? Using a NAS device (that is used only by PostgreSQL, so it's dedicated) with 3Gb of RAM and four 7200 rpm IDE harddisks, connected using a gigabit ethernet connection to the PostgreSQL server, do you think it will be a match for a SCSI-RAID config using 4 or 6 15000rpm disks (RAID-10) through a SCSI-RAID controller having 128mb of writeback cache (battery-backed)? The SCSI-RAID config would be a lot more expensive. I can't purchase both configs and test which one wil be faster, but if the NAS solution would be (almost) as fast as the SCSI-RAID solution, it would be cheaper and easier to maintain... About clustering: I know this can't be done by hooking multiple postmasters to one and the same NAS. This would result in data corruption, i've read... Kind regards, Alexander. - Original Message - From: Jeff [EMAIL PROTECTED] To: Alexander Priem [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Monday, October 20, 2003 2:20 PM Subject: Re: [PERFORM] PostgreSQL data on a NAS device ? On Mon, 20 Oct 2003 09:12:35 +0200 Alexander Priem [EMAIL PROTECTED] wrote: I am asking this because a NAS device is much cheaper to set up than a couple of SCSI disks. I would like to use a relatively cheap NAS device which uses four IDE drives (7.200 rpm), like the Dell PowerVault 725N. The disks themselves would be much slower than SCSI disks, I know, but a NAS device can be equipped with 3 Gb of memory, so this would make a very large disk cache, right? If this NAS would be dedicated only to PostgreSQL, would this be slower/faster than a SCSI RAID-10 setup of 6 disks? It would be much cheaper... The big concern would be the network connection, unless you are going fiber. You need to use _AT LEAST_ gigabit. _at least_.If you do go that route it'd be interesting to see bonnie results. And the other thing - remember that just because you are running NAS doesn't mean you can attach another machine running postgres and have a cluster. (See archives for more info about this). I suppose it all boils down to your budget (I usually get to work with a budget of $0). And I mentioned this in another post- If you don't mind refurb disks(or slightly used) check out ebay - you can get scsi disks by the truckload for cheap. -- Jeff Trout [EMAIL PROTECTED] http://www.jefftrout.com/ http://www.stuarthamm.net/ ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] PostgreSQL data on a NAS device ?
Even better than the four-disk NAS I mentioned earlier is the following: Promise UltraTrak RM8000. This is a so-called SCSI-to-IDE RAID system. Basically it's a RAID setup of eight IDE disks, using a hardware RAID engine, that's connected to (in this case) the PostgreSQL server via a SCSI Ultra160 interface (!). So the server won't know any better than that there's a SCSI disk attached, but in reality it's a IDE RAID setup. It supports RAID levels 0, 1, 0+1, 5, 50 and JBOD and supports hot-swapping. Such a NAS config would cost around EUR 3700 (ex. VAT), using 8x40 Gb IDE disks (7200rpm). A SCSI RAID-10 setup using 6x18Gb (15000rpm) disks would cost around EUR 6000 (ex. VAT) so it's a big difference... Does anyone have experience with this NAS device or other SCSI-to-IDE RAID systems? Are they OK in terms of performance and reliability? Kind regards, Alexander. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] PostgreSQL data on a NAS device ?
Alexander Priem kirjutas E, 20.10.2003 kell 16:04: Even better than the four-disk NAS I mentioned earlier is the following: Promise UltraTrak RM8000. This is a so-called SCSI-to-IDE RAID system. While you are at it, you could also check out http://www.3ware.com/ I guess one of these with 1 rpm 36GB SATA drivest would be pretty fast and possibly cheaper than SCSI raid. -- Hannu ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] PostgreSQL data on a NAS device ?
Hello Alexander, On Mon, 2003-10-20 at 06:04, Alexander Priem wrote: Even better than the four-disk NAS I mentioned earlier is the following: Promise UltraTrak RM8000. This is a so-called SCSI-to-IDE RAID system. Basically it's a RAID setup of eight IDE disks, using a hardware RAID engine, that's connected to (in this case) the PostgreSQL server via a SCSI Ultra160 interface (!). So the server won't know any better than that there's a SCSI disk attached, but in reality it's a IDE RAID setup. It supports RAID levels 0, 1, 0+1, 5, 50 and JBOD and supports hot-swapping. We have a Promise FasTrak 4000 in our development server connected to 120 Gig western digital 8mb cache drives. Basically the fastest drives we could get for an ide configuration. This system works well, however there are a few things you need to consider. The biggest is that you have very limited control over your devices with the Promise controllers. The bios of the raid controller doesn't have many options on it. You basically plug everything together, and just hope it works. It usually does, but there have been times in the past that really gave us a scare. And we had a situation that in a hard poweroff ( UPS died ) we suffered complete corruptions of 2 of our 4 drives. Performance wise it is =okay= but definitely not on par with either our Megaraid elite 1650 controller or a solution I'm going to suggest to you later in this mail. Your biggest hit is going to be multiple simultaneous accesses. The controller and drives just can't keep up to it. Realistically with my experiences I cannot recommend this solution for a production machine, even with the budget constraints you have put forth. Such a NAS config would cost around EUR 3700 (ex. VAT), using 8x40 Gb IDE disks (7200rpm). A SCSI RAID-10 setup using 6x18Gb (15000rpm) disks would cost around EUR 6000 (ex. VAT) so it's a big difference... I'm not sure where you have your figures, but I would like to propose the following solution for you. for your boot device use either a single ide drive and keep an exact duplicate of the drive in the event of a drive failure, or use 2 drives and use software raid to mirror the two. In this manner you can spend approx $100 USD for each drive and no additional cost for your controller as you will use the motherboards IDE controller. For your postgresql partition or even /var use software raid on an adaptec 29320-R SCSI controller. ( http://www.adaptec.com/worldwide/product/proddetail.html?sess=nolanguage=English+USprodkey=ASC-39320-Rcat=%2fTechnology%2fSCSI%2fUltra320+SCSI ) cost: $399 USD IF you bought it from adaptec Match this with 6 Seagate 10k 36G Cheetah U320 scsi drives: ( http://www.c-source.com/csource/newsite/ttechnote.asp?part_no=207024 ) for a cost of $189 USD per drive. If you have 6 of them it brings the total price for your drives to $1134 USD. Total cost for this would be approx $1633 before shipping costs. We use this configuration in our two file servers and have nothing but positive results. If you are totally unable to use software raid you could still buy 6 of those drives, and spend approx $900 USD on an LSI Megaraid 1650 controller. I really believe you'll find either of those options to be superior in terms of price for you. Sincerely, Will LaShell Does anyone have experience with this NAS device or other SCSI-to-IDE RAID systems? Are they OK in terms of performance and reliability? Kind regards, Alexander. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings signature.asc Description: This is a digitally signed message part
Re: [PERFORM] PostgreSQL data on a NAS device ?
On Mon, 20 Oct 2003, Alexander Priem wrote: Hi all, Does anyone have any experience with putting PostgreSQL data on a NAS device? I am asking this because a NAS device is much cheaper to set up than a couple of SCSI disks. I would like to use a relatively cheap NAS device which uses four IDE drives (7.200 rpm), like the Dell PowerVault 725N. The disks themselves would be much slower than SCSI disks, I know, but a NAS device can be equipped with 3 Gb of memory, so this would make a very large disk cache, right? If this NAS would be dedicated only to PostgreSQL, would this be slower/faster than a SCSI RAID-10 setup of 6 disks? It would be much cheaper... Any advice on this would be appreciated :) How important is this data? With a local SCSI RAID controller and SCSI drives, you can pull the power cord out the back of the machine during 1000 transactions, and your database will come back up in a coherent state. If you need that kind of reliability, then you'll likely want to use local SCSI drives. Note that you should test your setup to be sure, i.e. pull the network cord and see how the machine recovers (if the machine recovers). Running storage on a NAS is a bit of a tightrope act with your data, as is using IDE drives with write cache enabled. But depending on your application, using NAS may be a good solution. So, what's this database gonna be used for? ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
