Re: [gentoo-user] 4/8 CPU Gentoo server
gentuxx wrote: The main purpose of this box is going to be log crunching and archival. We have logs that range from tens of MBs to a GB a piece (uncompressed). The scripts running on it will be transferring (over the network), decompressing, grepping, normalizing, recompressing, and inserting into a local database from dozens of sources simultaneously. In a prior situation, I had a Sun e4500 with 8 UltraSPARC IIIs, 12GB of RAM, and about a TB of disk attached (SCSI and FC). There were times it wasn't enough. I'm hoping to at least match that functionality (preferably better it). Single boxes don't scale. :-) That's probably simplifying too much, but there is some truth there as well. I'd seriously think about setting up a preprocessing farm of 1U boxes that do most of the crunching and then doing inserts into a smaller db box. I've even worked on systems where logs go directly into a db, then are pulled and processed by another farm, then inserted into more of a data warehouse system for reporting later. You'll have to decide if the application can be broken up into separate processing units or whatever. If you do go that route it's usually an easy sell to management. We have one $15k box, but it's too small. We could buy one $30k box to do everything and completely replace the $15k box that we haven't fully depreciated yet or I can buy three $4k boxes to sit in front of our existing server which will share the load. And next year when we start to slow down again instead of buying an even bigger $60k box we just buy three more $4k boxes. Can I have some programmer time to make some architecture changes so I can save you around $65k over the next two years? The more comparisons and reviews I read are leaning me in that direction. However, it doesn't look like HP offers a 4-way Opteron box. I'll have to ask the vendor. from hp.com the DL585's appear to be configurable for 4-way once you get into their config tool. kashani -- gentoo-user@gentoo.org mailing list
[gentoo-user] 4/8 CPU Gentoo server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I say 4/8 because we're looking at Xeons that'll, at a minimum, have HT but could possibly be dual-core. I run gentoo on a P4 w/ HT and it runs great! But I have no idea how it will scale to this many processors. I've done some preliminary googling, but haven't come up with muchprobably using the wrong search terms. Base anticipated specs below: 4x 3.66Ghz Intel Xeons 8GB RAM 0.9 TB Disk (4x 300GB Ultra360 drives in RAID5) I would appreciate any insights, comments, advice, etc. Thanks. - -- gentux echo hfouvyAdpy/ofu | perl -pe 's/(.)/chr(ord($1)-1)/ge' gentux's gpg fingerprint == 34CE 2E97 40C7 EF6E EC40 9795 2D81 924A 6996 0993 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFD6oQhLYGSSmmWCZMRAgDrAKDkzh1wl7vSR2sr9WuG7L5wzc4EugCglqhb l3gtw27Z1tUvcAew0+wZS2I= =Yioc -END PGP SIGNATURE- -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] 4/8 CPU Gentoo server
On Wed, 2006-02-08 at 15:52 -0800, gentuxx wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I say 4/8 because we're looking at Xeons that'll, at a minimum, have HT but could possibly be dual-core. If you have the choice, go for dual core! HT is nice to look at (I have it on my laptop) but it's nowhere near as fast a dual core, because you still only have one cpu. Although you pay for dual core of course... I run gentoo on a P4 w/ HT and it runs great! But I have no idea how it will scale to this many processors. AFAIK, linux scales very well to many processors. You have some different kernel options to configure, some playing with MAKEOPTS=-j9 (sounds fun :), etc. I've done some preliminary googling, but haven't come up with muchprobably using the wrong search terms. you're probably more interested in linux performance on multiple cpus, rather than gentoo performace, as its the kernel that will run particular processes on particular cpus. Base anticipated specs below: 4x 3.66Ghz Intel Xeons 8GB RAM 0.9 TB Disk (4x 300GB Ultra360 drives in RAID5) make a second one for me :) I would appreciate any insights, comments, advice, etc. I think your biggest problem will be buying hardware pieces that are compatible with each other (unless its already built) and making sure linux has modules for the particular hardware you want. HTH, -- Iain Buchanan iain at netspace dot net dot au Manoj I *like* the chicken -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] 4/8 CPU Gentoo server
Hash: SHA1 Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I say 4/8 because we're looking at Xeons that'll, at a minimum, have HT but could possibly be dual-core. I run gentoo on a P4 w/ HT and it runs great! But I have no idea how it will scale to this many processors. I've done some preliminary googling, but haven't come up with muchprobably using the wrong search terms. Base anticipated specs below: 4x 3.66Ghz Intel Xeons 8GB RAM 0.9 TB Disk (4x 300GB Ultra360 drives in RAID5) I would appreciate any insights, comments, advice, etc. Thanks. Hi, I'm not in here much since I am away from home. I have a old Compaq Proliant 6000 server that has quad, allbeit slow, 200MHz CPUs. It works fine and it only needed me to change the kernel to a SMP kernel and set the max CPU to the max it will hold, 8 for you and 4 for me. It takes care of the rest pretty well. I'm not on this list to much so if you need to ask me something, please reply directly or CC me and put rdalek instead of dalek, just add a r to the front of the email addy. Please don't post the whole thing though. I check it more right now because I'm at my ladies house and I am doing this through the website with no thread options like in Mozilla. Sort of ticks me off. I plan to bring my rigs up here soon. She has never heard of Linux or Moilla. O_O Let me know if I can help. Keep in mind I don't have a Linux rig here though. Dale :D :D -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] 4/8 CPU Gentoo server
On Wed, 08 Feb 2006 15:52:01 -0800 gentuxx [EMAIL PROTECTED] wrote: | Just wondering if anyone here has any experience with gentoo on a 4/8 | CPU server. I say 4/8 because we're looking at Xeons that'll, at a | minimum, have HT but could possibly be dual-core. I run gentoo on a | P4 w/ HT and it runs great! But I have no idea how it will scale to | this many processors. I've done some preliminary googling, but | haven't come up with muchprobably using the wrong search terms. We use a 16 way for MIPS builds. For packages that parallelise it's brilliant. For small packages or those oh so icky packages that force -j1, things aren't so shiny... Still, there aren't any particular problems running Gentoo on SMP systems. -- Ciaran McCreesh : Gentoo Developer (King of all Londinium) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-user] 4/8 CPU Gentoo server
On Wed, 08 Feb 2006 15:52:01 -0800 gentuxx [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I've been running it for years on a 4P PIII Xeon and my take is I won't run more than 2 Intel processors on their Front Side Bus - The bus saturates very quickly. The cpus stall out waiting for memory. If you are going to run more than 2 cpus, go for an Opteron solution. It scales much better and the dual-core limits the FSB to 2 cpus per connect. Also, if you look around at places like - 2cpu.com, you'll see at 4 cpus, the Opteron's massive memory bandwidth leaves the Xeon way behind on most benchmarks. Finally, outside of very, very specific tasks, Intel's HT actually slows down performance. Again check the web sites for specific benchmarks. If your application doesn't fall into the use area where HT actually helps, it's best to turn it off. As to Linux scaling, I've run Linux, not Gentoo, on a few different 8P, 16P, 20P, 32P, and 64P systems, and on one 512P system. The 512P was kind of fun - kicking off and stopping 512 setiathome instances, all at the same time. Bob - -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] 4/8 CPU Gentoo server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bob Sanders wrote: On Wed, 08 Feb 2006 15:52:01 -0800 gentuxx [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I've been running it for years on a 4P PIII Xeon and my take is I won't run more than 2 Intel processors on their Front Side Bus - The bus saturates very quickly. The cpus stall out waiting for memory. I was more curious how the virtual CPUs would affect things. And if there was a limit on the number of processes supported by gentoo. I knew that the linux kernel (generically) supports well over 8 CPUs, but I just was trying to nail down any distro-specific issues. If you are going to run more than 2 cpus, go for an Opteron solution. It scales much better and the dual-core limits the FSB to 2 cpus per connect. Also, if you look around at places like - 2cpu.com, you'll see at 4 cpus, the Opteron's massive memory bandwidth leaves the Xeon way behind on most benchmarks. Finally, outside of very, very specific tasks, Intel's HT actually slows down performance. Again check the web sites for specific benchmarks. If your application doesn't fall into the use area where HT actually helps, it's best to turn it off. I'm getting this box from a vendor, and I thought they had limited options. We've always gone Intel before, and I'm relatively new to this organization. I'm going to poke around for some benchmarks. I would be interested in seeing them, both from the perspective of my home P4/HT box, as well as this one. As to Linux scaling, I've run Linux, not Gentoo, on a few different 8P, 16P, 20P, 32P, and 64P systems, and on one 512P system. The 512P was kind of fun - kicking off and stopping 512 setiathome instances, all at the same time. Bob - - -- gentux echo hfouvyAdpy/ofu | perl -pe 's/(.)/chr(ord($1)-1)/ge' gentux's gpg fingerprint == 34CE 2E97 40C7 EF6E EC40 9795 2D81 924A 6996 0993 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFD6qiNLYGSSmmWCZMRAj+9AJ4wUtyNQ2j5M7VyTfm60hG+g2FQdgCdGgKw B8H/GyixTImDLAqT89irnD8= =hRdS -END PGP SIGNATURE- -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] 4/8 CPU Gentoo server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Iain Buchanan wrote: On Wed, 2006-02-08 at 15:52 -0800, gentuxx wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I say 4/8 because we're looking at Xeons that'll, at a minimum, have HT but could possibly be dual-core. If you have the choice, go for dual core! HT is nice to look at (I have it on my laptop) but it's nowhere near as fast a dual core, because you still only have one cpu. Although you pay for dual core of course... I do have a choice! ;-) Based on Bob Sanders' comments, I'm going to do a little more research of Opterons, Xeons, and 4+ way computing. I run gentoo on a P4 w/ HT and it runs great! But I have no idea how it will scale to this many processors. AFAIK, linux scales very well to many processors. You have some different kernel options to configure, some playing with MAKEOPTS=-j9 (sounds fun :), etc. I've done some preliminary googling, but haven't come up with muchprobably using the wrong search terms. you're probably more interested in linux performance on multiple cpus, rather than gentoo performace, as its the kernel that will run particular processes on particular cpus. Base anticipated specs below: 4x 3.66Ghz Intel Xeons 8GB RAM 0.9 TB Disk (4x 300GB Ultra360 drives in RAID5) make a second one for me :) I would appreciate any insights, comments, advice, etc. I think your biggest problem will be buying hardware pieces that are compatible with each other (unless its already built) and making sure linux has modules for the particular hardware you want. Well, I'm getting this through a vendor. So I know the parts will be compatible with each other. The interesting question is will there be drivers/modules. I have built gentoo on these systems before without too much trouble, but not one spec'd out this way. The chassis is an HP DL580. HTH, - -- gentux echo hfouvyAdpy/ofu | perl -pe 's/(.)/chr(ord($1)-1)/ge' gentux's gpg fingerprint == 34CE 2E97 40C7 EF6E EC40 9795 2D81 924A 6996 0993 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFD6qmyLYGSSmmWCZMRAr/eAJ41nZfswC3ojPNPqFyvHyg23f+MmgCgo3IM E3LOLdbiITQaSoZ6/3aNkj4= =I+Ir -END PGP SIGNATURE- -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] 4/8 CPU Gentoo server
gentuxx wrote: Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I say 4/8 because we're looking at Xeons that'll, at a minimum, have HT but could possibly be dual-core. I run gentoo on a P4 w/ HT and it runs great! But I have no idea how it will scale to this many processors. I've done some preliminary googling, but haven't come up with muchprobably using the wrong search terms. I've been running a Dell 6650, 4 x 1.9 GHZ Xeons for about a year under Gentoo. Linux sees it as 8 processors with the HT stuff turned on. I sort of inherited that machine and we've never come close to pushing it, but it's been great for Mysql which is highly threaded. My only advice is that quad physical CPU boxes and up are much more expensive than dual proc boxes though that seems to be changing. Make sure you really need that sort of concentrated CPU power rather than three or four smaller boxes. Also remember that most of your dual core CPUs can have significantly less cache than single core CPUs. The Intels top out at 8MB on single and 2MB on dual core from a quick look around. On the application side you're want something highly threaded or with a large number of processes. No point in having eight procs when six are likely to be sitting around doing nothing. I'll also second the AMD recommendation. A number of LAMP people have mentioned that they're getting much better performace out of their 64bit AMD's than the equivalent Intels. Specifically the Cnet/Gamestop guys have been retiring three dual Xeon DL380s for each dual dual core DL385 they install. kashani -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] 4/8 CPU Gentoo server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 kashani wrote: gentuxx wrote: Hi all, Just wondering if anyone here has any experience with gentoo on a 4/8 CPU server. I say 4/8 because we're looking at Xeons that'll, at a minimum, have HT but could possibly be dual-core. I run gentoo on a P4 w/ HT and it runs great! But I have no idea how it will scale to this many processors. I've done some preliminary googling, but haven't come up with muchprobably using the wrong search terms. I've been running a Dell 6650, 4 x 1.9 GHZ Xeons for about a year under Gentoo. Linux sees it as 8 processors with the HT stuff turned on. I sort of inherited that machine and we've never come close to pushing it, but it's been great for Mysql which is highly threaded. My only advice is that quad physical CPU boxes and up are much more expensive than dual proc boxes though that seems to be changing. Make sure you really need that sort of concentrated CPU power rather than three or four smaller boxes. Also remember that most of your dual core CPUs can have significantly less cache than single core CPUs. The Intels top out at 8MB on single and 2MB on dual core from a quick look around. On the application side you're want something highly threaded or with a large number of processes. No point in having eight procs when six are likely to be sitting around doing nothing. The main purpose of this box is going to be log crunching and archival. We have logs that range from tens of MBs to a GB a piece (uncompressed). The scripts running on it will be transferring (over the network), decompressing, grepping, normalizing, recompressing, and inserting into a local database from dozens of sources simultaneously. In a prior situation, I had a Sun e4500 with 8 UltraSPARC IIIs, 12GB of RAM, and about a TB of disk attached (SCSI and FC). There were times it wasn't enough. I'm hoping to at least match that functionality (preferably better it). I'll also second the AMD recommendation. A number of LAMP people have mentioned that they're getting much better performace out of their 64bit AMD's than the equivalent Intels. Specifically the Cnet/Gamestop guys have been retiring three dual Xeon DL380s for each dual dual core DL385 they install. The more comparisons and reviews I read are leaning me in that direction. However, it doesn't look like HP offers a 4-way Opteron box. I'll have to ask the vendor. kashani - -- gentux echo hfouvyAdpy/ofu | perl -pe 's/(.)/chr(ord($1)-1)/ge' gentux's gpg fingerprint == 34CE 2E97 40C7 EF6E EC40 9795 2D81 924A 6996 0993 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFD6sDYLYGSSmmWCZMRAgh/AKCHWWtoEspJrNSCaaWpI/4lMLrQyQCZAQGa 5g3NIwWoSunXSdgcB1jc2f8= =PuJT -END PGP SIGNATURE- -- gentoo-user@gentoo.org mailing list
