Re: [PERFORM] How much ram is too much
Hi Andrew On 11-Jun-07, at 11:34 AM, Andrew Sullivan wrote: On Mon, Jun 11, 2007 at 11:09:42AM -0400, Dave Cramer wrote: and set them to anything remotely close to 128GB. Well, we'd give 25% of it to postgres, and the rest to the OS. Are you quite sure that PostgreSQL's management of the buffers is efficient with such a large one? No, I'm not sure of this. In the past, that wasn't the case for relatively small buffers; with the replacement of single-pass LRU, that has certainly changed, but I'd be surprised if anyone tested a buffer as large as 32G. So does anyone have experience above 32G ? Dave A -- Andrew Sullivan | [EMAIL PROTECTED] The whole tendency of modern prose is away from concreteness. --George Orwell ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] How much ram is too much
Actually this one is an opteron, so it looks like it's all good. Dave On 8-Jun-07, at 3:41 PM, Guy Rouillier wrote: Dave Cramer wrote: It's an IBM x3850 using linux redhat 4.0 I had to look that up, web site says it is a 4-processor, dual-core (so 8 cores) Intel Xeon system. It also says Up to 64GB DDR II ECC memory, so are you sure you can even get 128 GB RAM? If you could, I'd expect diminishing returns from the Xeon northbridge memory access. If you are willing to spend that kind of money on memory, you'd be better off with Opteron or Sparc. -- Guy Rouillier ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] How much ram is too much
On Jun 8, 2007, at 11:31 AM, Dave Cramer wrote: Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? Providing to what? PostgreSQL? The OS? My bet is that you'll run into issues with how shared_buffers are managed if you actually try and set them to anything remotely close to 128GB. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] How much ram is too much
On 10-Jun-07, at 11:11 PM, Jim Nasby wrote: On Jun 8, 2007, at 11:31 AM, Dave Cramer wrote: Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? Providing to what? PostgreSQL? The OS? My bet is that you'll run into issues with how shared_buffers are managed if you actually try and set them to anything remotely close to 128GB. Well, we'd give 25% of it to postgres, and the rest to the OS. What is it specifically you are referring to ? Dave -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PERFORM] How much ram is too much
On Mon, Jun 11, 2007 at 11:09:42AM -0400, Dave Cramer wrote: and set them to anything remotely close to 128GB. Well, we'd give 25% of it to postgres, and the rest to the OS. Are you quite sure that PostgreSQL's management of the buffers is efficient with such a large one? In the past, that wasn't the case for relatively small buffers; with the replacement of single-pass LRU, that has certainly changed, but I'd be surprised if anyone tested a buffer as large as 32G. A -- Andrew Sullivan | [EMAIL PROTECTED] The whole tendency of modern prose is away from concreteness. --George Orwell ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [PERFORM] How much ram is too much
Hi Andrew On 11-Jun-07, at 11:34 AM, Andrew Sullivan wrote: On Mon, Jun 11, 2007 at 11:09:42AM -0400, Dave Cramer wrote: and set them to anything remotely close to 128GB. Well, we'd give 25% of it to postgres, and the rest to the OS. Are you quite sure that PostgreSQL's management of the buffers is efficient with such a large one? No, I'm not sure of this. In the past, that wasn't the case for relatively small buffers; with the replacement of single-pass LRU, that has certainly changed, but I'd be surprised if anyone tested a buffer as large as 32G. So does anyone have experience above 32G ? Dave A -- Andrew Sullivan | [EMAIL PROTECTED] The whole tendency of modern prose is away from concreteness. --George Orwell ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[PERFORM] How much ram is too much
Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? Dave ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] How much ram is too much
What is your expected data size and usage pattern? What are the other components in the system? On Fri, 8 Jun 2007, Dave Cramer wrote: Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? Dave ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] How much ram is too much
Dave Cramer wrote: Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? What CPU and OS are you considering? -- Guy Rouillier ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [PERFORM] How much ram is too much
It's an IBM x3850 using linux redhat 4.0 On 8-Jun-07, at 12:46 PM, Guy Rouillier wrote: Dave Cramer wrote: Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? What CPU and OS are you considering? -- Guy Rouillier ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PERFORM] How much ram is too much
On Fri, 8 Jun 2007, Dave Cramer wrote: Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? the only way 128G of ram would be too much is if your total database size (including indexes) is smaller then this. now it may not gain you as much of an advantage going from 64G to 128G as it does going from 32G to 64G, but that depends on many variables as others have been asking. David Lang ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[OT] Re: [PERFORM] How much ram is too much
Dave Cramer írta: It's an IBM x3850 using linux redhat 4.0 Isn't that a bit old? I have a RedHat 4.2 somewhere that was bundled with Applixware 3. :-) -- -- Zoltán Böszörményi Cybertec Geschwinde Schönig GmbH http://www.postgresql.at/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] How much ram is too much
[EMAIL PROTECTED] wrote: On Fri, 8 Jun 2007, Dave Cramer wrote: Is it possible that providing 128G of ram is too much ? Will other systems in the server bottleneck ? the only way 128G of ram would be too much is if your total database size (including indexes) is smaller then this. now it may not gain you as much of an advantage going from 64G to 128G as it does going from 32G to 64G, but that depends on many variables as others have been asking. I don't know about the IBM but I know some of the HPs require slower ram to actually get to 128G. Joshua D. Drake David Lang ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [OT] Re: [PERFORM] How much ram is too much
Zoltan Boszormenyi wrote: Dave Cramer írta: It's an IBM x3850 using linux redhat 4.0 Isn't that a bit old? I have a RedHat 4.2 somewhere that was bundled with Applixware 3. :-) He means redhat ES/AS 4 I assume. J -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [OT] Re: [PERFORM] How much ram is too much
On 8-Jun-07, at 2:10 PM, Joshua D. Drake wrote: Zoltan Boszormenyi wrote: Dave Cramer írta: It's an IBM x3850 using linux redhat 4.0 Isn't that a bit old? I have a RedHat 4.2 somewhere that was bundled with Applixware 3. :-) He means redhat ES/AS 4 I assume. Yes AS4 J -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/ donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [OT] Re: [PERFORM] How much ram is too much
Joshua D. Drake írta: Zoltan Boszormenyi wrote: Dave Cramer írta: It's an IBM x3850 using linux redhat 4.0 Isn't that a bit old? I have a RedHat 4.2 somewhere that was bundled with Applixware 3. :-) He means redhat ES/AS 4 I assume. J I guessed that, hence the smiley. But it's very unfortunate that version numbers are reused - it can cause confusion. There was a RH 4.0 already a long ago, when the commercial and the community version were the same. I think Microsoft will avoid reusing its versions when year 2095 comes. :-) -- -- Zoltán Böszörményi Cybertec Geschwinde Schönig GmbH http://www.postgresql.at/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [OT] Re: [PERFORM] How much ram is too much
On Fri, Jun 08, 2007 at 08:54:39PM +0200, Zoltan Boszormenyi wrote: Joshua D. Drake írta: Zoltan Boszormenyi wrote: Dave Cramer írta: It's an IBM x3850 using linux redhat 4.0 Isn't that a bit old? I have a RedHat 4.2 somewhere that was bundled with Applixware 3. :-) He means redhat ES/AS 4 I assume. I guessed that, hence the smiley. But it's very unfortunate that version numbers are reused - it can cause confusion. There was a RH 4.0 already a long ago, when the commercial and the community version were the same. I think Microsoft will avoid reusing its versions when year 2095 comes. :-) He should have written RHEL 4.0. RH 4.0 is long enough ago, though, that I think few would assume it meant the much older release. You'll find a similar thing with products like CuteFTP 7.0 or CuteFTP Pro 3.0. Cheers, mark -- [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] __ . . _ ._ . . .__. . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/|_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [OT] Re: [PERFORM] How much ram is too much
[EMAIL PROTECTED] írta: On Fri, Jun 08, 2007 at 08:54:39PM +0200, Zoltan Boszormenyi wrote: Joshua D. Drake írta: Zoltan Boszormenyi wrote: Dave Cramer írta: It's an IBM x3850 using linux redhat 4.0 Isn't that a bit old? I have a RedHat 4.2 somewhere that was bundled with Applixware 3. :-) He means redhat ES/AS 4 I assume. I guessed that, hence the smiley. But it's very unfortunate that version numbers are reused - it can cause confusion. There was a RH 4.0 already a long ago, when the commercial and the community version were the same. I think Microsoft will avoid reusing its versions when year 2095 comes. :-) He should have written RHEL 4.0. RH 4.0 is long enough ago, though, that I think few would assume it meant the much older release. Yes. But up until RHEL 8.0/9.0 ( or plain 9 without decimals ;-) ) I can make cheap jokes telling that I can give you a free upgrade. :-) You'll find a similar thing with products like CuteFTP 7.0 or CuteFTP Pro 3.0. I am sure there are others, too. But enough of this OT, I am really interested in the main thread's topic. Best regards, -- -- Zoltán Böszörményi Cybertec Geschwinde Schönig GmbH http://www.postgresql.at/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] How much ram is too much
Dave Cramer wrote: It's an IBM x3850 using linux redhat 4.0 I had to look that up, web site says it is a 4-processor, dual-core (so 8 cores) Intel Xeon system. It also says Up to 64GB DDR II ECC memory, so are you sure you can even get 128 GB RAM? If you could, I'd expect diminishing returns from the Xeon northbridge memory access. If you are willing to spend that kind of money on memory, you'd be better off with Opteron or Sparc. -- Guy Rouillier ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [OT] Re: [PERFORM] How much ram is too much
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Zoltan Boszormenyi wrote: Joshua D. Drake írta: Zoltan Boszormenyi wrote: Dave Cramer írta: It's an IBM x3850 using linux redhat 4.0 Isn't that a bit old? I have a RedHat 4.2 somewhere that was bundled with Applixware 3. :-) He means redhat ES/AS 4 I assume. J I guessed that, hence the smiley. But it's very unfortunate that version numbers are reused - it can cause confusion. There was a RH 4.0 already a long ago, when the commercial and the community version were the same. I think Microsoft will avoid reusing its versions when year 2095 comes. :-) Well, RedHat Linux, and RedHat Linux Enterprise Server/Advanced Servers are clearly different products :-P And yes, I even owned Applix :) Andreas -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGac2FHJdudm4KnO0RAkpcAJwI+RTIJgAc5Db1bnsu7tRNiU9vzACeIGvl LP0CSxc5dML0BMerI+u1xYc= =qiye -END PGP SIGNATURE- ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
