Re: [CentOS] managing a rack full of centos servers
I'm managing two data centers and some instances on rackspace cloud servers. Currently running Cobbler+Puppet+Mcollective. So far it's been great for a team of one, myself. At the moment I'm looking into either using Aeolus or Openstack to bridge the gap of my data centers and the public cloud still keeping Puppet+Mcollective in the mix and seeing if Cobbler is still needed. Anyone out there tried both Aeolus *and* Openstack yet? I'm looking to supplement my research on these two private/public cloud tools. =) On Thu, Jul 21, 2011 at 3:19 PM, Marian Marinov m...@yuhu.biz wrote: On Thursday 21 July 2011 18:36:17 Devin Reade wrote: --On Wednesday, July 20, 2011 11:02:42 PM -0700 RC cool...@gmail.com wrote: On Wed, 20 Jul 2011 10:07:06 -0600 Devin Reade g...@gno.org wrote: It should be considered as complementing the automated config management tools like cfengine et al, not as a replacement for them (they're doing different jobs). That's not entirely fair. A little shell scripting and pdsh and pdcp can certainly do everything cfengine/puppet can do I wasn't referring to pdsh/pdcp; I was referring to pconsole. The reason I said complementing is that sometimes it is good to have stuff under a configuration management system like cfengine/puppet, but sometimes you need to run ad-hoc commands, in an identical fashion, on lots of similar machines, which pconsole is good at (subject to the caveats I previously mentioned). I made no comments on pdsh/pdcp at all, and make no claims on where it fits in the spectrum. Devin You can actually achieve the same functionality of pdsh/pdcp and pconsole with a quite simple bash script :) http://multy-command.sourceforge.net/ I think it is a matter of what the admin will prefer to do. When you have a lot of identical machines, sometimes it is better to have cfengine/puppet, but sometimes it just an overkill to use them if you are the only one administrating those machines. cfengine and puppet have a very good place on machines that are administered by a team of people. But solutions like pdsh/pconsole and multy-command, in my opinion are more suitable when there are only one or two guys administering those machines. Marian ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- james h nguyen | lead systems architect | www.callfire.com | 1.949.625.4263 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Sep 5, 2011 2:47 AM, James Nguyen ja...@callfire.com wrote: I'm managing two data centers and some instances on rackspace cloud servers. Currently running Cobbler+Puppet+Mcollective. So far it's been great for a team of one, myself. At the moment I'm looking into either using Aeolus or Openstack to bridge the gap of my data centers and the public cloud still keeping Puppet+Mcollective in the mix and seeing if Cobbler is still needed. Anyone out there tried both Aeolus *and* Openstack yet? I'm looking to supplement my research on these two private/public cloud tools. =) On Thu, Jul 21, 2011 at 3:19 PM, Marian Marinov m...@yuhu.biz wrote: On Thursday 21 July 2011 18:36:17 Devin Reade wrote: --On Wednesday, July 20, 2011 11:02:42 PM -0700 RC cool...@gmail.com wrote: On Wed, 20 Jul 2011 10:07:06 -0600 Devin Reade g...@gno.org wrote: It should be considered as complementing the automated config management tools like cfengine et al, not as a replacement for them (they're doing different jobs). That's not entirely fair. A little shell scripting and pdsh and pdcp can certainly do everything cfengine/puppet can do I wasn't referring to pdsh/pdcp; I was referring to pconsole. The reason I said complementing is that sometimes it is good to have stuff under a configuration management system like cfengine/puppet, but sometimes you need to run ad-hoc commands, in an identical fashion, on lots of similar machines, which pconsole is good at (subject to the caveats I previously mentioned). I made no comments on pdsh/pdcp at all, and make no claims on where it fits in the spectrum. Devin You can actually achieve the same functionality of pdsh/pdcp and pconsole with a quite simple bash script :) http://multy-command.sourceforge.net/ I think it is a matter of what the admin will prefer to do. When you have a lot of identical machines, sometimes it is better to have cfengine/puppet, but sometimes it just an overkill to use them if you are the only one administrating those machines. cfengine and puppet have a very good place on machines that are administered by a team of people. But solutions like pdsh/pconsole and multy-command, in my opinion are more suitable when there are only one or two guys administering those machines. Marian ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- james h nguyen | lead systems architect | www.callfire.com | 1.949.625.4263 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos +1 for Puppet. I manage only around 20 servers all running a mix of CentOS 5.6 and CentOS 6 very well with Puppet. The initial configuration and understanding for it is daunting but WELL worth it in the end. Also for system provisioning ( kickstart and pxe) look at Foreman, which uses Puppet after initial installation. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Wed, 20 Jul 2011 10:07:06 -0600 Devin Reade g...@gno.org wrote: You get one master xterm, a bunch of slave xterms, and you can either type in the master to affect all nodes or selectively type in the slaves. Yes, but I don't want a bunch of XTerms. I can slide my phone open, ssh in and manage my cluster using pdsh. And I've written plenty of serious scripts using pdsh/pdcp, which obviously wouldn't work with XTerms popping up. It should be considered as complementing the automated config management tools like cfengine et al, not as a replacement for them (they're doing different jobs). That's not entirely fair. A little shell scripting and pdsh and pdcp can certainly do everything cfengine/puppet can do, and obviously more that they can't. Some of it may be a bit more clumsy this way, though it has other advantages like being atomic so to speak, and not lumbering around, slowly putting things in-sync. I don't want to sound like a zealot by any means. It's got plenty of marks against it. But it most definitely works, in some very demanding circumstances, and it still hasn't become a problem, even as we keep asking it to do ever-more. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Wed, 20 Jul 2011, m.r...@5-cent.us wrote: Two and a quarter years ago, I got stuck with Spacewalk where I had a short-term contract, and it was a horror. (Note that while I was working on it, it went from 0.4 to 0.5) As Iain said, it requires Oracle, and I found I had to add an addition setting to Oracle - the free version has a max of 1G of memory, and I had to max it out (I think I set shared memory in Oracle's control panel to 994M), just to get it to work. It's also complex to configure and use, so if you're not looking at dozens or hundreds of machines, I wouldn't use it. It /did/ require Oracle, but it doesn't any more. To be honest, getting it working with Oracle was a piece of cake, and the 1G limit wasn't really an issue as the active database isn't that big. The 4Gbyte limit on database size on disk with the XE edition /was/ an issue once you were managing a large numer of machines. The defaults that it ship with work fine, other than a couple of values that can improve performance that are documented on the spacewalk web site. I've used spacewalk since 0.1, and it's really not that bad at all. There's bits of it I think could be better, but it's not the 'horror' you seem to think it is. jh ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Wed, 20 Jul 2011, Patrick Lists wrote: On 07/20/2011 06:11 PM, Iain Morris wrote: Spacewalk is great, but be prepared for some significant configuration time and energy. Also, it requires Oracle (postgres is in progress last I checked). From what I read the PostgreSQL support is functional for regular usage and has been improving significantly the last few releases. Worth a try if you don't want to fund Larry's next superyacht. This is true, certainly 1.4 is almost completely working on PostgreSQL. Monitoring is the only thing listed not to work with PostgreSQL on 1.5 (which isn't quite out yet), but that's not the end of the world. jh ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On 7/19/11 6:43 PM, John R Pierce wrote: its looking like I might need to setup a deployment of a dozen or 2 basically identical machines, all running pretty much the same sorts of stuff. I have zero experience with the sorts of management tools folks use to automate this type of configuration, both initial setup, and ongoing management (system updates, user application updates, configuration changes, etc). anyone care to suggest any such tools, maybe some real-world pros and cons? of course, being centos, I prefer FOSS tools. for various reasons, this environment likely will NOT be virtualized (although I may emulate a test setup with vmware). If the server hardware is really identical including disks of the same sizes, the fastest way to roll them out is probably clonezilla, which has the big advantage of being mostly agnostic toward the target OS. You can try it out with the bootable iso, using sshfs or nfs to connect to network storage for the image. If you like it, set up the drbl server to pxe boot into it. It is rpm packaged, but I ended up using ubuntu on the server because it used to only use the server's kernel on the clients and I needed something newer than Centos provided for windows machines. Now I believe you can configure it to pxe boot the image from a current livecd even if you run it on a different server. Besides the ability to clone windows and other OS's, it also is good for snapshot backups of systems (if you can take them down for the copy) and it duplicates everything, including any local installs and config edits. -- Les Mikesell lesmikes...@gmail.cm ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
--On Wednesday, July 20, 2011 11:02:42 PM -0700 RC cool...@gmail.com wrote: On Wed, 20 Jul 2011 10:07:06 -0600 Devin Reade g...@gno.org wrote: It should be considered as complementing the automated config management tools like cfengine et al, not as a replacement for them (they're doing different jobs). That's not entirely fair. A little shell scripting and pdsh and pdcp can certainly do everything cfengine/puppet can do I wasn't referring to pdsh/pdcp; I was referring to pconsole. The reason I said complementing is that sometimes it is good to have stuff under a configuration management system like cfengine/puppet, but sometimes you need to run ad-hoc commands, in an identical fashion, on lots of similar machines, which pconsole is good at (subject to the caveats I previously mentioned). I made no comments on pdsh/pdcp at all, and make no claims on where it fits in the spectrum. Devin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Thursday 21 July 2011 18:36:17 Devin Reade wrote: --On Wednesday, July 20, 2011 11:02:42 PM -0700 RC cool...@gmail.com wrote: On Wed, 20 Jul 2011 10:07:06 -0600 Devin Reade g...@gno.org wrote: It should be considered as complementing the automated config management tools like cfengine et al, not as a replacement for them (they're doing different jobs). That's not entirely fair. A little shell scripting and pdsh and pdcp can certainly do everything cfengine/puppet can do I wasn't referring to pdsh/pdcp; I was referring to pconsole. The reason I said complementing is that sometimes it is good to have stuff under a configuration management system like cfengine/puppet, but sometimes you need to run ad-hoc commands, in an identical fashion, on lots of similar machines, which pconsole is good at (subject to the caveats I previously mentioned). I made no comments on pdsh/pdcp at all, and make no claims on where it fits in the spectrum. Devin You can actually achieve the same functionality of pdsh/pdcp and pconsole with a quite simple bash script :) http://multy-command.sourceforge.net/ I think it is a matter of what the admin will prefer to do. When you have a lot of identical machines, sometimes it is better to have cfengine/puppet, but sometimes it just an overkill to use them if you are the only one administrating those machines. cfengine and puppet have a very good place on machines that are administered by a team of people. But solutions like pdsh/pconsole and multy-command, in my opinion are more suitable when there are only one or two guys administering those machines. Marian signature.asc Description: This is a digitally signed message part. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Tue, Jul 19, 2011 at 6:00 PM, Fajar Priyanto fajar...@arinet.org wrote: On Wed, Jul 20, 2011 at 8:25 AM, Patrick Lists centos-l...@puzzled.xs4all.nl wrote: On 07/20/2011 02:03 AM, Fajar Priyanto wrote: Redhat satellite can handle it. Too bad I don't know if there is foss alternative for it. There is http://spacewalk.redhat.com/ They have it? Awesome! Thanks for the info!! Spacewalk is great, but be prepared for some significant configuration time and energy. Also, it requires Oracle (postgres is in progress last I checked). The free version of Oracle has a single processor limitation. I'd say about 20 systems is the threshold for when the up-front config time starts paying off. -- -- - Iain Morris iain.t.mor...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
--On Tuesday, July 19, 2011 08:45:54 PM -0700 cool...@gmail.com wrote: On Jul 19, 2011 6:48 PM, Jay Leafey jay.lea...@mindless.com wrote: I usually use SSH keys in conjunction with ClusterSSH In the same vein, I instead recomend pdsh. Another variant that has been around a long time is pconsole. http://freshmeat.net/projects/pconsole This is a tool for executing the same command on many similar machines at once, and doesn't require anything to be on the target machines other than ssh. You get one master xterm, a bunch of slave xterms, and you can either type in the master to affect all nodes or selectively type in the slaves. It should be considered as complementing the automated config management tools like cfengine et al, not as a replacement for them (they're doing different jobs). pconsole is more intended for concurrent ad-hoc changes. The only thing to keep in mind with pconsole is screen real-estate. You can have your slave xterms small (like 40x4), but if you have more nodes than you can get slave xterms on your screen at one time, it's less effective. Devin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
Iain Morris wrote: On Tue, Jul 19, 2011 at 6:00 PM, Fajar Priyanto fajar...@arinet.org wrote: On Wed, Jul 20, 2011 at 8:25 AM, Patrick Lists centos-l...@puzzled.xs4all.nl wrote: On 07/20/2011 02:03 AM, Fajar Priyanto wrote: Redhat satellite can handle it. Too bad I don't know if there is foss alternative for it. There is http://spacewalk.redhat.com/ They have it? Awesome! Thanks for the info!! Spacewalk is great, but be prepared for some significant configuration time and energy. Also, it requires Oracle (postgres is in progress last I checked). The free version of Oracle has a single processor limitation. snip Two and a quarter years ago, I got stuck with Spacewalk where I had a short-term contract, and it was a horror. (Note that while I was working on it, it went from 0.4 to 0.5) As Iain said, it requires Oracle, and I found I had to add an addition setting to Oracle - the free version has a max of 1G of memory, and I had to max it out (I think I set shared memory in Oracle's control panel to 994M), just to get it to work. It's also complex to configure and use, so if you're not looking at dozens or hundreds of machines, I wouldn't use it. mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
2011/7/20 m.r...@5-cent.us: Iain Morris wrote: On Tue, Jul 19, 2011 at 6:00 PM, Fajar Priyanto fajar...@arinet.org wrote: On Wed, Jul 20, 2011 at 8:25 AM, Patrick Lists centos-l...@puzzled.xs4all.nl wrote: On 07/20/2011 02:03 AM, Fajar Priyanto wrote: Redhat satellite can handle it. Too bad I don't know if there is foss alternative for it. There is http://spacewalk.redhat.com/ They have it? Awesome! Thanks for the info!! Spacewalk is great, but be prepared for some significant configuration time and energy. Also, it requires Oracle (postgres is in progress last I checked). The free version of Oracle has a single processor limitation. snip Two and a quarter years ago, I got stuck with Spacewalk where I had a short-term contract, and it was a horror. (Note that while I was working on it, it went from 0.4 to 0.5) As Iain said, it requires Oracle, and I found I had to add an addition setting to Oracle - the free version has a max of 1G of memory, and I had to max it out (I think I set shared memory in Oracle's control panel to 994M), just to get it to work. It's also complex to configure and use, so if you're not looking at dozens or hundreds of machines, I wouldn't use it. Oracle named user license is very cheap .. -- Eero ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On 07/20/2011 06:11 PM, Iain Morris wrote: Spacewalk is great, but be prepared for some significant configuration time and energy. Also, it requires Oracle (postgres is in progress last I checked). From what I read the PostgreSQL support is functional for regular usage and has been improving significantly the last few releases. Worth a try if you don't want to fund Larry's next superyacht. Regards, Patrick ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On 07/20/11 10:12 AM, Eero Volotinen wrote: Oracle named user license is very cheap .. but if you read the license, every single node that generates data is considered a 'user', even if it goes through a webservice or other form of 'data concentrator' and doesn't directly connect to SQL. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On 7/19/2011 7:43 PM, John R Pierce wrote: to date, I've done all my administration on a manual 1 at a time basis, as each system has been pretty much unique. its looking like I might need to setup a deployment of a dozen or 2 basically identical machines, all running pretty much the same sorts of stuff. I have zero experience with the sorts of management tools folks use to automate this type of configuration, both initial setup, and ongoing management (system updates, user application updates, configuration changes, etc). anyone care to suggest any such tools, maybe some real-world pros and cons? of course, being centos, I prefer FOSS tools. for various reasons, this environment likely will NOT be virtualized (although I may emulate a test setup with vmware). webmin is a good free option depending on your admin needs. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Wed, Jul 20, 2011 at 7:43 AM, John R Pierce pie...@hogranch.com wrote: to date, I've done all my administration on a manual 1 at a time basis, as each system has been pretty much unique. its looking like I might need to setup a deployment of a dozen or 2 basically identical machines, all running pretty much the same sorts of stuff. I have zero experience with the sorts of management tools folks use to automate this type of configuration, both initial setup, and ongoing management (system updates, user application updates, configuration changes, etc). Redhat satellite can handle it. Too bad I don't know if there is foss alternative for it. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Tue, Jul 19, 2011 at 8:43 PM, John R Pierce pie...@hogranch.com wrote: to date, I've done all my administration on a manual 1 at a time basis, as each system has been pretty much unique. its looking like I might need to setup a deployment of a dozen or 2 basically identical machines, all running pretty much the same sorts of stuff. I have zero experience with the sorts of management tools folks use to automate this type of configuration, both initial setup, and ongoing management (system updates, user application updates, configuration changes, etc). anyone care to suggest any such tools, maybe some real-world pros and cons? of course, being centos, I prefer FOSS tools. for various reasons, this environment likely will NOT be virtualized (although I may emulate a test setup with vmware). You might want to look at automation tools like Puppet, Chef or Cfengine (in no particular order). -- Giovanni Tirloni ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On 07/20/2011 02:03 AM, Fajar Priyanto wrote: Redhat satellite can handle it. Too bad I don't know if there is foss alternative for it. There is http://spacewalk.redhat.com/ Or check out: http://pulpproject.org/ https://fedorahosted.org/candlepin/ http://theforeman.org/ (or look at https://fedorahosted.org/cobbler/ ) The above managed from: http://www.katello.org/ And then there's also for the Cloud: https://www.aeolusproject.org/ And off course the workhorse: http://www.puppetlabs.com/ Regards, Patrick ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
Spacewalk is the free alternative to Satellite, and is pretty dang awesome if you ask me. On Tue, Jul 19, 2011 at 8:25 PM, Patrick Lists centos-l...@puzzled.xs4all.nl wrote: On 07/20/2011 02:03 AM, Fajar Priyanto wrote: Redhat satellite can handle it. Too bad I don't know if there is foss alternative for it. There is http://spacewalk.redhat.com/ Or check out: http://pulpproject.org/ https://fedorahosted.org/candlepin/ http://theforeman.org/ (or look at https://fedorahosted.org/cobbler/ ) The above managed from: http://www.katello.org/ And then there's also for the Cloud: https://www.aeolusproject.org/ And off course the workhorse: http://www.puppetlabs.com/ Regards, Patrick ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On 20/07/11 10:30 AM, David Lemcoe wrote: Spacewalk is the free alternative to Satellite, and is pretty dang awesome if you ask me. +1 for spacewalk. I use it in combination with kickstarts (have not fiddled with the cobbler/PXE provisioning interface yet) to rollout identical deployments for HPC grids. Then manage patches and updates via spacewalk. Cheers -pete signature.asc Description: OpenPGP digital signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Wed, Jul 20, 2011 at 8:25 AM, Patrick Lists centos-l...@puzzled.xs4all.nl wrote: On 07/20/2011 02:03 AM, Fajar Priyanto wrote: Redhat satellite can handle it. Too bad I don't know if there is foss alternative for it. There is http://spacewalk.redhat.com/ They have it? Awesome! Thanks for the info!! ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On 7/19/11 6:43 PM, John R Pierce wrote: to date, I've done all my administration on a manual 1 at a time basis, as each system has been pretty much unique. its looking like I might need to setup a deployment of a dozen or 2 basically identical machines, all running pretty much the same sorts of stuff. I have zero experience with the sorts of management tools folks use to automate this type of configuration, both initial setup, and ongoing management (system updates, user application updates, configuration changes, etc). anyone care to suggest any such tools, maybe some real-world pros and cons? of course, being centos, I prefer FOSS tools. for various reasons, this environment likely will NOT be virtualized (although I may emulate a test setup with vmware). It doesn't take that much time to manage a server. For a dozen or two you probably can't save enough time to be worth setting up anything more than ssh keys on one that you use for management and a couple of scripts that loop over them to do things like 'ssh $host yum -y update that you might do frequently. For more ad-hoc things you can just open a bunch of terminal windows ssh'd to each and paste in the commands. For the install you can copy the kickstart file that the first install creates to a web server and use it to duplicate the setup on the others. You might want something like backuppc to keep a history of recent copies of at least /etc and anywhere else you have modified files. If you do any complicated programming or scripting, you'll probably want subversion or some other version control system to manage the revisions. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
I usually use SSH keys in conjunction with ClusterSSH (http://clusterssh.sourceforge.net), I have been using the 3.27 version with good results. It makes managing batches of servers a bit easier, allowing the execution of the same command across multiple systems at the same time. -- Jay Leafey - jay.lea...@mindless.com Memphis, TN smime.p7s Description: S/MIME Cryptographic Signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] managing a rack full of centos servers
On Jul 19, 2011 6:48 PM, Jay Leafey jay.lea...@mindless.com wrote: I usually use SSH keys in conjunction with ClusterSSH ( http://clusterssh.sourceforge.net), I have been using the 3.27 version with good results. It makes managing batches of servers a bit easier, allowing the execution of the same command across multiple systems at the same time. In the same vein, I instead recomend pdsh. It has a few quirks which increases the learning curve, but it works very, very well, assuming that's the level of control you want/need. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos