Re: [PERFORM] hardware raid suggestions
> We're looking into getting an Adaptec 2200S or the Megaraid 320 2x > which have better processors, and hopefully better performance. We > feel that the use of the AIC7930 as the CPU on the ZCR just doesn't > cut it and a faster raid controller would work better. Does anyone out > there have any experience with these cards with postgresql and linux? > If so, would you be willing to share your experiences and possibly give > a recommendation? I have worked with at least four major name brands of scsi and ide raid controllers and so far the one I have found to be generally the most featured and fastest is the ICP Vortex controllers (http://www.icp-vortex.com/). It is also more expensive than the others but has been worth the cost IMHO. It has a command line utility to measure disk performance and I believe the source code for it is available. I have measured over 200 MB/s reads off these controllers on 3u disk array units. I'm sure I could have gotten more with additional tuning. Fred ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[PERFORM] Scaling with lazy index updates
Pg Performers, This might be a out of the ordinary question, or perhaps I have been out of the loop for a while but does PostgreSQL (or any other database) have support for lazy index updates. What I mean by lazy index updates is index updating which occur at a regular interval rather than per transaction. I have found that inserts and updates tend to slow down when the database gets really big. I think it is likely an effect of updating indexes when the insert or update occurs. Looking forward to feedback and possibly direction on my lazy index update question. TIA, Fred ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] Value of Quad vs. Dual Processor machine
One thing I learned after spending about a week comparing the Athlon (2 ghz, 333 mhz frontside bus) and Xeon (2.4 ghz, 266 mhz frontside bus) platforms was that on average the select queries I was benchmarking ran 30% faster on the Athlon (this was with data cached in memory so may not apply to the larger data sets where I/O is the limiting factor.) I benchmarked against the Opteron 244 when it came out and it came in about the same as the Athlon (makes sense since both were 333 mhz memory). The results within +/- 5-10% that of the Athlon. From testing against a couple of other machines I noticed that the memory bus speeds were almost directly proportional to the query times under these conditions. Not sure how these compare against the quad sun but the AMD chips returned the select queries faster than the Xeons from the informal investigations I did. Definitely try it before you buy it if possible. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Field Sent: Tuesday, November 11, 2003 6:13 PM To: Ron Johnson; PgSQL Performance ML Subject: Re: [PERFORM] Value of Quad vs. Dual Processor machine we are looking at Xeon, We are currently running it on a quad sun v880 compiled to be 64bit and have been getting dreadful performance. I don't think we really have much to gain from going 64bit. - Original Message - From: "Ron Johnson" <[EMAIL PROTECTED]> To: "PgSQL Performance ML" <[EMAIL PROTECTED]> Sent: Tuesday, November 11, 2003 8:24 PM Subject: Re: [PERFORM] Value of Quad vs. Dual Processor machine > On Tue, 2003-11-11 at 17:32, Chris Field wrote: > > We are getting ready to spec out a new machine and are wondering > > about the wisdom of buying a quad versus a dual processor machine. > > Seing as how postgres in not a threaded application, and this server > > will only be used for log/transaction analysis (it will only ever > > have a few large queries running). Is there any performance to be > > gained, and if so is it worth the large cost? Any > > thoughts/experience are much appreciated... > > Xeon or Opteron? The faster Opterons *really* blaze, especially in > 64-bit mode. As others have said, though, RAM and I/O are most > important. > > -- > - > Ron Johnson, Jr. [EMAIL PROTECTED] > Jefferson, LA USA > > "As I like to joke, I may have invented it, but Microsoft made it > popular" David Bradley, regarding Ctrl-Alt-Del > > > ---(end of > broadcast)--- > TIP 5: Have you checked our extensive FAQ? > >http://www.postgresql.org/docs/faqs/FAQ.html > > ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] Has anyone run on the new G5 yet
> We are running into issues with IO saturation obviously. Since this > thing is only going to get bigger we are looking for some advice on > how to accommodate DB's of this size. > Second and more radical, has anyone run > postgreSQL on the new Apple G5 with an XRaid system? This seems like > a great value combination. Fast CPU, wide bus, Fibre Channel IO, > 2.5TB all for ~17k. If you are going for I/O performance you are best off with one of the Xserve competitors listed at http://www.apple.com/xserve/raid/. The Xserve is based on IDE drives which have a lower seek time (say 8.9 ms) compared to scsi (3.6 ms for seagate cheetah). For small random read/write operations (like databases) this will give you a noticable improvement in performance over ide drives. Also make sure to get as many drives as possible, more spindles equals better I/O performance. > I keep see references to terabyte postgreSQL installations, I was > wondering if anyone on this list is in charge of one of those and can > offer some advice on how to position ourselves hardware wise. I've gone to about half terabyte size and all I can say is you should plan for at least one quarter to one half a rack of drivespace (assuming 14 drives per 4u that's 42 to 84 drives). Do yourself a favor and get more rather than less, you will really appreciate it. I averaged about 2 mb/s average per drive via the raid controller stats on 14 drive array during I/O bound seek and update operations in 2 raid 10 arrays (half xlogs and half data). That comes out to around 2 hours for a terabyte with 70 drives assuming a constant scaling. You may be able to get more or less depending on your setup and query workload. > Thanks. > > --sean > > > ---(end of > broadcast)--- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to [EMAIL PROTECTED] so that your > message can get through to the mailing list cleanly > ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PERFORM] Scaling further up
On Tue, 2004-03-02 at 17:42, William Yu wrote: > Anjan Dave wrote: > > We have a Quad-Intel XEON 2.0GHz (1MB cache), 12GB memory, running RH9, > > PG 7.4.0. There's an internal U320, 10K RPM RAID-10 setup on 4 drives. > > > > We are expecting a pretty high load, a few thousands of 'concurrent' > > users executing either select, insert, update, statments. > > The quick and dirty method would be to upgrade to the recently announced > 3GHz Xeon MPs with 4MB of L3. My semi-educated guess is that you'd get > another +60% there due to the huge L3 hiding the Xeon's shared bus penalty. If you are going to have thousands of 'concurrent' users you should seriously consider the 2.6 kernel if you are running Linux or as an alternative going with FreeBSD. You will need to load test your system and become an expert on tuning Postgres to get the absolute maximum performance from each and every query you have. And you will need lots of hard drives. By lots I mean dozen(s) in a raid 10 array with a good controller. Thousands of concurrent users means hundreds or thousands of transactions per second. I've personally seen it scale that far but in my opinion you will need a lot more hard drives and ram than cpu. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
