Re: [HACKERS] Options for growth
On Wed, 12 Feb 2003, GB Clark wrote: > On Thu, 23 Jan 2003 11:19:36 -0700 (MST) > "scott.marlowe" <[EMAIL PROTECTED]> wrote: > > > On 23 Jan 2003, Hannu Krosing wrote: > > > > > Curt Sampson kirjutas N, 23.01.2003 kell 17:42: > > > > If the OS can handle the scheduling (which, last I checked, Linux couldn't, > > > > > > When did you do your checking ? > > > (just curious, not to start a flame war ;) > > > > > > > at least not without patches), eight or sixteen > > > > CPUs will be fine. > > > > Yeah, take a look here: > > > > http://www.sgi.com/servers/altix/ > > > > 64 CPUs seems scalable enough for me. :-) When can we expect BSD to run > > on this system and use all 64 CPUs efficiently? > > > > I think FreeBSD 5.[1|2] will be able to. That was the entire reason for SMPng and > KSE. There is not too much of the kernel left untouched from the 4.0 split. > > As far as NetBSD or OpenBSD goes, I would not expect it too soon... I just downloaded 5.0 last week and I've a pretty little dual PPro sitting here that needs to be ridden hard. It has lots of spare drives and Linux is already on one, so this will be a nice box for playing with different distros and what not. Now I just need an altix... Even a little one would do. Now how do I convince the powers that be where I work that we have a need for an 8 to 64 way SMP monster box? ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Options for growth
On Thu, 23 Jan 2003 11:19:36 -0700 (MST) "scott.marlowe" <[EMAIL PROTECTED]> wrote: > On 23 Jan 2003, Hannu Krosing wrote: > > > Curt Sampson kirjutas N, 23.01.2003 kell 17:42: > > > If the OS can handle the scheduling (which, last I checked, Linux couldn't, > > > > When did you do your checking ? > > (just curious, not to start a flame war ;) > > > > > at least not without patches), eight or sixteen > > > CPUs will be fine. > > Yeah, take a look here: > > http://www.sgi.com/servers/altix/ > > 64 CPUs seems scalable enough for me. :-) When can we expect BSD to run > on this system and use all 64 CPUs efficiently? > I think FreeBSD 5.[1|2] will be able to. That was the entire reason for SMPng and KSE. There is not too much of the kernel left untouched from the 4.0 split. As far as NetBSD or OpenBSD goes, I would not expect it too soon... GB -- GB Clark II | Roaming FreeBSD Admin [EMAIL PROTECTED] | General Geek CTHULU for President - Why choose the lesser of two evils? ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Options for growth
On Thu, Jan 16, 2003 at 12:23:52PM -0500, Neil Conway wrote: > > The estimates I've heard from a couple parties are that PostgreSQL tends > to scale well up to 4 CPUs. I've been meaning to take a look at > improving that, but I haven't had a chance yet... I can definitely tell you that Postgres scales _fine_ beyond 4 processors. Indeed, we have found under some loads that 4 processors is not enough; but when we put it into an 8- or more-way box, it is much faster. That's on Solaris, though, which is generally very good at handling greater-than-4 CPUs. That's why Solaris is a good platform for us, even though its fork() times rot. > think the cost of subsidizing some of that development would be a > fraction of the license fees you'll end up paying Oracle over the > years... And it's worth pointing out what those ORAC licenses really cost: it might be as little as the savings of a single year. By the way ORAC may not be _quite_ as bulletproof as it seems. It shares file areas, and there are rumours of locking troubles that people trip over. Nothing they'll share with you, of course: the license forbids as much. But if you ask someone over the top of a glass, he or she might tell you about it. A -- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada <[EMAIL PROTECTED]> M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Options for growth
On 23 Jan 2003, Hannu Krosing wrote: > Curt Sampson kirjutas N, 23.01.2003 kell 17:42: > > If the OS can handle the scheduling (which, last I checked, Linux couldn't, > > When did you do your checking ? > (just curious, not to start a flame war ;) > > > at least not without patches), eight or sixteen > > CPUs will be fine. Yeah, take a look here: http://www.sgi.com/servers/altix/ 64 CPUs seems scalable enough for me. :-) When can we expect BSD to run on this system and use all 64 CPUs efficiently? ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Options for growth
On Fri, 23 Jan 2003, Hannu Krosing wrote: > Curt Sampson kirjutas N, 23.01.2003 kell 17:42: > > If the OS can handle the scheduling (which, last I checked, Linux couldn't, > > When did you do your checking ? > (just curious, not to start a flame war ;) This was perhaps a year or so ago. IBM had some patches to fix a lot of the scheduler problems. I wouldn't be surprised if things are in a much better state now. Anyway, there are lots of work-arounds. Find the appropriate patches if the kernel still doesn't have them, run Solaris, whatever cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Options for growth
Curt Sampson kirjutas N, 23.01.2003 kell 17:42: > If the OS can handle the scheduling (which, last I checked, Linux couldn't, When did you do your checking ? (just curious, not to start a flame war ;) > at least not without patches), eight or sixteen > CPUs will be fine. > > cjs -- Hannu Krosing <[EMAIL PROTECTED]> ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Options for growth
On Thu, 16 Jan 2003, D'Arcy J.M. Cain wrote: > Due to the fact that we are growing out of our current system > (PostgreSQL on PCs) we are looking for ways to expand and one of the > suggestions has been to toss PostgreSQL in favour of Oracle with > Remote Access Cluster (RAC) software. The theory is that you can just > plug machines into the cluster if the database appears to be straining > and they automagically take over some of the load. > ... > My idea is to create a new middleware layer that allows me to split > things up based on various criteria without changing my application. It's a basic principle of clustering that doing it in an application- aware way will always be more efficient than trying to hide it from the application. If you've not read it already, I strongly suggest reading _In Search of Clusters_ by Gregory F. Pfister. > And finally, if you had your dream machine to run on, what would it > be? We are also looking outside of PC hardware but we are concerned > about not having access to that nice, cheap, generic hardware for when > we need to grow again or for redundant backup. If you can manage to stick with PC hardware, you are going to save a *lot* of money. If you're considering buying a reasonably well loaded Sun E6000 or similar, it's well worth spending twenty or thirty thousand dollars on a big PC system and spending some time to see if that will do the trick before you shell out a couple hundred thousand for the Sun. As for how well postgres uses multiple CPUs: so long as you've got lots of connections with the load distributed among them, it's dependent on the OS, postgres. If the OS can handle the scheduling (which, last I checked, Linux couldn't, at least not without patches), eight or sixteen CPUs will be fine. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Options for growth
On Wed, 22 Jan 2003, Sean Chittenden wrote: > > > By the way, I too wonder which supported OS platform would support > > > over 4GB of memory on a PC.. > > > > Linux? I don't think there's any problem handling more than 4G > > memory in the system. On 32bit architectures, there's of course the > > 3G (I think) per process limit, but as postgres uses multiprocess > > and not multithreading, this issue doesn't hit so soon. Of course, > > if the per process memory is the problem, you'd have to go to 64bit. > > Heh, don't kid yourself. x86 can only handle 4GB of memory > addressing. The hack that Linux uses is to swap out 2GB sections of > RAM to a 4GB+ memory range, then copy the memory range it needs down > into usable memory space. Can we say large page tables? :) > > You need an actual 64bit CPU to access more than 4GB of RAM without > paying for it through the nose. -sc No, you do not. If you need to access more than two to three GB (depending on the OS) of RAM on a 32-bit machine *within a single process* (as mentioned above), you have a problem. But this problem does not necessarially involve copying; you could use, e.g., mmap to remap chunks of your address space. If you have multiple processes, and your OS is sensibly written, no memory copying is necessary on the process side. All you do is change the page tables, and the appropriate physical memory, no matter where in the physical address space it resides, will be mapped into the 32-bit virtual memory address space. That's not to say that there might not be other issues with I/O on, say, 32-bit PCI buses. IIRC, typically PCI bus controllers use physical, not virtual addresses on the bus for DMA, so you're going to have to use bounce buffers if you wish a 32-bit PCI card to do I/O outside the bottom 4 GB of memory. But on the other hand, if you're spending the money on a motherboard that can take more than 4 GB of RAM, you're almost certainly getting a few 64-bit PCI slots, and probably you'd also be spending the money to buy 64-bit PCI disk controllers. This is not to say you shouldn't go for a 64-bit system, especially given that the AMD ones are probably going to get awfully cheap fairly soon. But postgres itself is today not equipment to take any more advantage of one than it is of a 32-bit system with a greater than 32-bit physical address space. (And there's been doubt about whether the techniques that would take advantage of this would provide all that much of a performance improvement, anyway. Still, it seems to me that it would be pretty cool, when you're doing I/O on a table, just to say, with one system call, "mmap this entire file containing the table into my address space," and not have to worry about running out of address space when you do this on multiple large tables. (And yes, I know this would actually be, "map this 1 GB chunk of this large table" in the current postgres implemenation.) cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Options for growth
> > That would depend on the OS. Not many 'pc-based unix' support over > > 4 GB of memory, some don't even go that far. > > > By the way, I too wonder which supported OS platform would support > > over 4GB of memory on a PC.. > > Linux? I don't think there's any problem handling more than 4G > memory in the system. On 32bit architectures, there's of course the > 3G (I think) per process limit, but as postgres uses multiprocess > and not multithreading, this issue doesn't hit so soon. Of course, > if the per process memory is the problem, you'd have to go to 64bit. Heh, don't kid yourself. x86 can only handle 4GB of memory addressing. The hack that Linux uses is to swap out 2GB sections of RAM to a 4GB+ memory range, then copy the memory range it needs down into usable memory space. Can we say large page tables? :) You need an actual 64bit CPU to access more than 4GB of RAM without paying for it through the nose. -sc -- Sean Chittenden msg27765/pgp0.pgp Description: PGP signature
Re: [HACKERS] Options for growth
[no cc:s please] On Mon, 2003-01-20 at 10:31, Daniel Kalchev wrote: > >>>"D'Arcy J.M. Cain" said: > > On Thursday 16 January 2003 11:59, Adrian 'Dagurashibanipal' von Bidder wrot > e: > > > On Thu, 2003-01-16 at 17:42, D'Arcy J.M. Cain wrote: > > > > We are also looking at hardware solutions, multi-CPU PCs with tons (24GB > ) > > > > of memory. I know that memory will improve access if it prevents > > > > swapping but how well does PostgreSQL utilize multiple CPUs? > > > > > > At most one CPU is used for any single postgres backend (that means for > > > any single database connection). So, if your load problem is single > > > queries being too slow, thee's nothing you can do with adding more CPUs. > > > If your problem is many connections maxing out the db, PostgreSQL can > > > take full advantage of multiple CPUs. > > > > I most definitely have multiple queries running at once. My main issue is > > whether PostgreSQL scales up properly or does it get bogged down with too > > many locked queries. > > That would depend on the OS. Not many 'pc-based unix' support over 4 GB of > memory, some don't even go that far. > By the way, I too wonder which supported OS platform would support over 4GB of > memory on a PC.. Linux? I don't think there's any problem handling more than 4G memory in the system. On 32bit architectures, there's of course the 3G (I think) per process limit, but as postgres uses multiprocess and not multithreading, this issue doesn't hit so soon. Of course, if the per process memory is the problem, you'd have to go to 64bit. cheers -- vbi -- featured link: http://fortytwo.ch/gpg/intro signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Options for growth
>>>"D'Arcy J.M. Cain" said: > On Thursday 16 January 2003 11:59, Adrian 'Dagurashibanipal' von Bidder wrot e: > > On Thu, 2003-01-16 at 17:42, D'Arcy J.M. Cain wrote: > > > We are also looking at hardware solutions, multi-CPU PCs with tons (24GB ) > > > of memory. I know that memory will improve access if it prevents > > > swapping but how well does PostgreSQL utilize multiple CPUs? > > > > At most one CPU is used for any single postgres backend (that means for > > any single database connection). So, if your load problem is single > > queries being too slow, thee's nothing you can do with adding more CPUs. > > If your problem is many connections maxing out the db, PostgreSQL can > > take full advantage of multiple CPUs. > > I most definitely have multiple queries running at once. My main issue is > whether PostgreSQL scales up properly or does it get bogged down with too > many locked queries. That would depend on the OS. Not many 'pc-based unix' support over 4 GB of memory, some don't even go that far. If memory is an issue, have you considered going to 64bit CPU? Memory is indeed an issue for a complex database setup, especially if you want to give the backends enough shared and sort memory. As already said, PostgreSQL will utilize multiple CPUs - as effectively as your OS can do this of course. PostgreSQL is not an OS by itself and does not really control these resources. I have also found it very helpful to split database from application servers (wish I do it as often as I recommend it :) - thus you can optimize the part that needs most resources.. In many cases the requirements are quite different. With todays gigabit LANs, bandwidth between machines shouldn't be an issue. By the way, I too wonder which supported OS platform would support over 4GB of memory on a PC.. Daniel ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Options for growth
On Thursday 16 January 2003 11:59, Adrian 'Dagurashibanipal' von Bidder wrote: > On Thu, 2003-01-16 at 17:42, D'Arcy J.M. Cain wrote: > > We are also looking at hardware solutions, multi-CPU PCs with tons (24GB) > > of memory. I know that memory will improve access if it prevents > > swapping but how well does PostgreSQL utilize multiple CPUs? > > At most one CPU is used for any single postgres backend (that means for > any single database connection). So, if your load problem is single > queries being too slow, thee's nothing you can do with adding more CPUs. > If your problem is many connections maxing out the db, PostgreSQL can > take full advantage of multiple CPUs. I most definitely have multiple queries running at once. My main issue is whether PostgreSQL scales up properly or does it get bogged down with too many locked queries. > Of course, most db apps still are not cpu bound, so you'd have to do > some careful benchmarking first or you'll be spending too much money. Natch. -- D'Arcy J.M. Cain| Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Options for growth
On Thursday 16 January 2003 20:54, Christopher Kings-Lynne wrote: > > toss PostgreSQL in favour of Oracle with Remote Access Cluster (RAC) > > software. > > You mean Real Application Clusters? Oops, yes. -- D'Arcy J.M. Cain| Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Options for growth
On Thursday 16 January 2003 12:23, Neil Conway wrote: > On Thu, 2003-01-16 at 11:42, D'Arcy J.M. Cain wrote: > > Is [Oracle RAC] really as simple as it sounds or would we just be > > giving up the other two for a new set of problems. > > That's a question you should be asking to an authority on Oracle RAC > (which pgsql-hackers is not). True but I already have their perspective. Now I am looking for reasons to stay with PostgreSQL. > > My idea is to create a new middleware layer that allows me to split > > things up based on various criteria without changing my application. > > Personally, I would not be very eager to use home-brew replication for a > heavy-load, production-critical application (which is what your app > sounds like). But YMMV... Not replication per se although I suppose that could be built in. What I am talking about is an application that knows our business logic and brokers requests for data from the database(s) in an OO way. The idea is to split data up in the middleware whenever necessary. > > We are also looking at hardware solutions, multi-CPU PCs with tons (24GB) > > of memory. I know that memory will improve access if it prevents > > swapping but how well does PostgreSQL utilize multiple CPUs? > > The estimates I've heard from a couple parties are that PostgreSQL tends > to scale well up to 4 CPUs. I've been meaning to take a look at > improving that, but I haven't had a chance yet... Cool. I am looking at Tyan boards that have 4 CPUs and 24GB memory. > Another option is to put some money toward the current development > effort to get truly scalable replication for PostgreSQL. In the end, I'd > think the cost of subsidizing some of that development would be a > fraction of the license fees you'll end up paying Oracle over the > years... This is definitely an option. I can probably put both people and money into such an effort. I would be happy to hear from people who can work on that. -- D'Arcy J.M. Cain| Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Options for growth
> Due to the fact that we are growing out of our current system > (PostgreSQL on > PCs) we are looking for ways to expand and one of the suggestions > has been to > toss PostgreSQL in favour of Oracle with Remote Access Cluster (RAC) > software. You mean Real Application Clusters? Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Options for growth
On Thu, 2003-01-16 at 11:42, D'Arcy J.M. Cain wrote: > Is [Oracle RAC] really as simple as it sounds or would we just be > giving up the other two for a new set of problems. That's a question you should be asking to an authority on Oracle RAC (which pgsql-hackers is not). > My idea is to create a new middleware layer that allows me to split things up > based on various criteria without changing my application. Personally, I would not be very eager to use home-brew replication for a heavy-load, production-critical application (which is what your app sounds like). But YMMV... > We are also looking at hardware solutions, multi-CPU PCs with tons (24GB) of > memory. I know that memory will improve access if it prevents swapping but > how well does PostgreSQL utilize multiple CPUs? The estimates I've heard from a couple parties are that PostgreSQL tends to scale well up to 4 CPUs. I've been meaning to take a look at improving that, but I haven't had a chance yet... Another option is to put some money toward the current development effort to get truly scalable replication for PostgreSQL. In the end, I'd think the cost of subsidizing some of that development would be a fraction of the license fees you'll end up paying Oracle over the years... Cheers, Neil -- Neil Conway <[EMAIL PROTECTED]> || PGP Key ID: DB3C29FC ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Options for growth
On Thu, 2003-01-16 at 17:42, D'Arcy J.M. Cain wrote: > We are also looking at hardware solutions, multi-CPU PCs with tons (24GB) of > memory. I know that memory will improve access if it prevents swapping but > how well does PostgreSQL utilize multiple CPUs? At most one CPU is used for any single postgres backend (that means for any single database connection). So, if your load problem is single queries being too slow, thee's nothing you can do with adding more CPUs. If your problem is many connections maxing out the db, PostgreSQL can take full advantage of multiple CPUs. Of course, most db apps still are not cpu bound, so you'd have to do some careful benchmarking first or you'll be spending too much money. cheers -- vbi -- get my gpg key here: http://fortytwo.ch/gpg/92082481 signature.asc Description: This is a digitally signed message part
[HACKERS] Options for growth
Due to the fact that we are growing out of our current system (PostgreSQL on PCs) we are looking for ways to expand and one of the suggestions has been to toss PostgreSQL in favour of Oracle with Remote Access Cluster (RAC) software. The theory is that you can just plug machines into the cluster if the database appears to be straining and they automagically take over some of the load. Not knowing much about it I can only argue about price and source code availability. The first has some value but the second is harder to argue without knowing about RAC. Is it really as simple as it sounds or would we just be giving up the other two for a new set of problems. My idea is to create a new middleware layer that allows me to split things up based on various criteria without changing my application. RAC sounds like it does that at the database/SQL level. Does it? We are also looking at hardware solutions, multi-CPU PCs with tons (24GB) of memory. I know that memory will improve access if it prevents swapping but how well does PostgreSQL utilize multiple CPUs? And finally, if you had your dream machine to run on, what would it be? We are also looking outside of PC hardware but we are concerned about not having access to that nice, cheap, generic hardware for when we need to grow again or for redundant backup. Thanks for any tips and suggestions. -- D'Arcy J.M. Cain| Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
