Re: [gentoo-user] Ansible, puppet and chef
On 2014-09-16 22:43, Alan McKinnon wrote: Anyone here used ansible and at least one of puppet/chef? What are your thoughts? I've made several attempts over the years to get puppet going but never really got it off the ground. Chef I stay away from (likely due to the first demo of it I saw and how badly that went) Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. Plus puppet's language and configs get large and hard to keep track of - lots and lots of directory trees with many things mentioning other things. (Nagios has the same problem if you start keeping host, services, groups and commands in many different files) I've stumbled upon ansible, it seems much better than puppet for smallish sites with good odds I might even keep the whole thing in my head at any one time :-) Anyone care to share experiences? We use ansible. I like it because you don't need any agents to install, just the ssh keys and python, which is mandatory on gentoo anyway. We use a minimalistic script that bootstraps machines (xen-domU) and then everything else is configured via ansible. Since version 1.6 there is the portage module to install software and you can do pretty stuff with replace/lineinfile/template/copy modules. The roles are a good way of keeping your systems equal. We have a common role for all gentoo machines, then roles specific for dom0 and domU machines and then the actual roles of a project (project-app for application server of a project). You can even more abstract it to have a common application server or a common database, but since you can include other playbooks, we don't use it that way (also to not get lost in too many levels of abstractions). For upgrades you either write precise playbooks (for example, before you used a specific testing package and now you want a newer testing one) where you delete the previous package.accept_keywords line and insert the new one. Or by having a small number of servers it's often faster by clusterssh.
Re: [gentoo-user] Ansible, puppet and chef
On 17/09/2014 03:30, Alec Ten Harmsel wrote: We use bcfg2, and all I can say is to stay away. XML abuse runs rampant in bcfg2. From what I've heard from other professional sysadmins, Puppet is the favorite, but that's mostly conjecture. XML. Ugh. OSSEC works like that too. The software itself works well but the config is painful. Alec On 09/16/2014 04:43 PM, Alan McKinnon wrote: Anyone here used ansible and at least one of puppet/chef? What are your thoughts? I've made several attempts over the years to get puppet going but never really got it off the ground. Chef I stay away from (likely due to the first demo of it I saw and how badly that went) Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. Plus puppet's language and configs get large and hard to keep track of - lots and lots of directory trees with many things mentioning other things. (Nagios has the same problem if you start keeping host, services, groups and commands in many different files) I've stumbled upon ansible, it seems much better than puppet for smallish sites with good odds I might even keep the whole thing in my head at any one time :-) Anyone care to share experiences? -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Ansible, puppet and chef
On Tuesday, September 16, 2014 10:43:18 PM Alan McKinnon wrote: Anyone here used ansible and at least one of puppet/chef? What are your thoughts? I've made several attempts over the years to get puppet going but never really got it off the ground. Chef I stay away from (likely due to the first demo of it I saw and how badly that went) Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. Plus puppet's language and configs get large and hard to keep track of - lots and lots of directory trees with many things mentioning other things. (Nagios has the same problem if you start keeping host, services, groups and commands in many different files) I've stumbled upon ansible, it seems much better than puppet for smallish sites with good odds I might even keep the whole thing in my head at any one time :-) Anyone care to share experiences? No experiences yet, but I have been looking for options to quickly and easily create (and remove) VMs lab environments. I agree with your comments on Chef and Puppet. Ansible looks nice and seems easy to manage. I miss an option to store the configuration inside a database, but I don't see an issue adding the generation of the config-files from database tables to the rest of the environment I am working on. I like that Ansible also seems to support MS Windows nodes, just too bad that requires enabling it after install. But with this, cloning VMs and changing the network configs afterwards seems easier to manage. -- Joost
Re: [gentoo-user] Ansible, puppet and chef
On 17/09/2014 09:07, Tomas Mozes wrote: On 2014-09-16 22:43, Alan McKinnon wrote: Anyone here used ansible and at least one of puppet/chef? What are your thoughts? I've made several attempts over the years to get puppet going but never really got it off the ground. Chef I stay away from (likely due to the first demo of it I saw and how badly that went) Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. Plus puppet's language and configs get large and hard to keep track of - lots and lots of directory trees with many things mentioning other things. (Nagios has the same problem if you start keeping host, services, groups and commands in many different files) I've stumbled upon ansible, it seems much better than puppet for smallish sites with good odds I might even keep the whole thing in my head at any one time :-) Anyone care to share experiences? We use ansible. I like it because you don't need any agents to install, just the ssh keys and python, which is mandatory on gentoo anyway. We use a minimalistic script that bootstraps machines (xen-domU) and then everything else is configured via ansible. Since version 1.6 there is the portage module to install software and you can do pretty stuff with replace/lineinfile/template/copy modules. The roles are a good way of keeping your systems equal. We have a common role for all gentoo machines, then roles specific for dom0 and domU machines and then the actual roles of a project (project-app for application server of a project). You can even more abstract it to have a common application server or a common database, but since you can include other playbooks, we don't use it that way (also to not get lost in too many levels of abstractions). For upgrades you either write precise playbooks (for example, before you used a specific testing package and now you want a newer testing one) where you delete the previous package.accept_keywords line and insert the new one. Or by having a small number of servers it's often faster by clusterssh. That's almost exactly the same setup I have in mind. How complex do the playbooks get in real-life? -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Ansible, puppet and chef
On 17/09/2014 09:34, J. Roeleveld wrote: On Tuesday, September 16, 2014 10:43:18 PM Alan McKinnon wrote: Anyone here used ansible and at least one of puppet/chef? What are your thoughts? I've made several attempts over the years to get puppet going but never really got it off the ground. Chef I stay away from (likely due to the first demo of it I saw and how badly that went) Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. Plus puppet's language and configs get large and hard to keep track of - lots and lots of directory trees with many things mentioning other things. (Nagios has the same problem if you start keeping host, services, groups and commands in many different files) I've stumbled upon ansible, it seems much better than puppet for smallish sites with good odds I might even keep the whole thing in my head at any one time :-) Anyone care to share experiences? No experiences yet, but I have been looking for options to quickly and easily create (and remove) VMs lab environments. Have you tried Vagrant? I haven't tried it myself, I'm just reacting to the VM keyword ;-) I agree with your comments on Chef and Puppet. Ansible looks nice and seems easy to manage. I miss an option to store the configuration inside a database, but I don't see an issue adding the generation of the config-files from database tables to the rest of the environment I am working on. Ansible has an add-on called Tower that seems to do this. The marketing blurb implies you can use almost any storage backend you like from MySQL and PostGres to LDAP I like that Ansible also seems to support MS Windows nodes, just too bad that requires enabling it after install. But with this, cloning VMs and changing the network configs afterwards seems easier to manage. I'm lucky, this is a Unix-only shop so I don't have to deal with Windows servers. The three managers who have Windows laptops for varying reasons have all been clearly told upfront they will support themselves and I ain't touching it :-) -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Ansible, puppet and chef
On Wednesday, September 17, 2014 10:12:52 AM Alan McKinnon wrote: On 17/09/2014 09:34, J. Roeleveld wrote: On Tuesday, September 16, 2014 10:43:18 PM Alan McKinnon wrote: Anyone here used ansible and at least one of puppet/chef? What are your thoughts? I've made several attempts over the years to get puppet going but never really got it off the ground. Chef I stay away from (likely due to the first demo of it I saw and how badly that went) Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. Plus puppet's language and configs get large and hard to keep track of - lots and lots of directory trees with many things mentioning other things. (Nagios has the same problem if you start keeping host, services, groups and commands in many different files) I've stumbled upon ansible, it seems much better than puppet for smallish sites with good odds I might even keep the whole thing in my head at any one time :-) Anyone care to share experiences? No experiences yet, but I have been looking for options to quickly and easily create (and remove) VMs lab environments. Have you tried Vagrant? Nope. I haven't tried it myself, I'm just reacting to the VM keyword ;-) Yes, but it doesn't have support for Xen or KVM and I'd need to write a custom provider to make that work. That basically does what I am looking into, but with the products we work with, I need more custom activities in some of the VMs then are easily organised. I agree with your comments on Chef and Puppet. Ansible looks nice and seems easy to manage. I miss an option to store the configuration inside a database, but I don't see an issue adding the generation of the config-files from database tables to the rest of the environment I am working on. Ansible has an add-on called Tower that seems to do this. The marketing blurb implies you can use almost any storage backend you like from MySQL and PostGres to LDAP Ok, from a quick scan of that page, it looked like a web frontend for some stuff. I'll definitely look into that part. The rest is more custom, so I might just generate the config files on the fly. I like that Ansible also seems to support MS Windows nodes, just too bad that requires enabling it after install. But with this, cloning VMs and changing the network configs afterwards seems easier to manage. I'm lucky, this is a Unix-only shop so I don't have to deal with Windows servers. The three managers who have Windows laptops for varying reasons have all been clearly told upfront they will support themselves and I ain't touching it :-) Not all products we deal with run on non-MS Windows systems, so we are sort-of stuck with it. They only run inside VMs that are only accessible via the LAB network. Which means, no access to the internet unless specifically allowed. (The host and port on the internet needs to be known prior to allowing access) -- Joost
Re: [gentoo-user] Ansible, puppet and chef
On Tue, Sep 16, 2014 at 10:43:18PM +0200, Alan McKinnon wrote: Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. I find that for a few machines, puppet is overkill. For a lot of machines, puppet can become unmanageable - with puppet master and security being the culprit. We have used puppet a lot but recently settled on salt (strictly speaking not my decision so cannot really compare it with ansible) and we are happy with the outcome. You might want to consider app-admin/salt as well. -- Eray
Re: [gentoo-user] Ansible, puppet and chef
On Wednesday, September 17, 2014 12:19:37 PM Eray Aslan wrote: On Tue, Sep 16, 2014 at 10:43:18PM +0200, Alan McKinnon wrote: Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. I find that for a few machines, puppet is overkill. For a lot of machines, puppet can become unmanageable - with puppet master and security being the culprit. We have used puppet a lot but recently settled on salt (strictly speaking not my decision so cannot really compare it with ansible) and we are happy with the outcome. You might want to consider app-admin/salt as well. Looks good (had a really quick look). From what I read (and please correct me if I'm wrong), a difference between salt and ansible is: Salt Requires a daemon to be installed and running on all machines and the versions need to be (mostly) in sync For Alan, this might work, but for my situation it wouldn't, as I'd need to keep various VMs in sync with the rest where I'd prefer to simply clone them and then enforce changes. Relying on SSH and powershell makes that simpler. But, it does mean that all nodes need to have incoming ports open. With Salt, all nodes connect back to the master. This allows a tighter security. -- Joost
Re: [gentoo-user] Ansible, puppet and chef
On 17/09/2014 11:34, J. Roeleveld wrote: On Wednesday, September 17, 2014 12:19:37 PM Eray Aslan wrote: On Tue, Sep 16, 2014 at 10:43:18PM +0200, Alan McKinnon wrote: Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. I find that for a few machines, puppet is overkill. For a lot of machines, puppet can become unmanageable - with puppet master and security being the culprit. We have used puppet a lot but recently settled on salt (strictly speaking not my decision so cannot really compare it with ansible) and we are happy with the outcome. You might want to consider app-admin/salt as well. Looks good (had a really quick look). From what I read (and please correct me if I'm wrong), a difference between salt and ansible is: Salt Requires a daemon to be installed and running on all machines and the versions need to be (mostly) in sync For Alan, this might work, but for my situation it wouldn't, as I'd need to keep various VMs in sync with the rest where I'd prefer to simply clone them and then enforce changes. Relying on SSH and powershell makes that simpler. But, it does mean that all nodes need to have incoming ports open. With Salt, all nodes connect back to the master. This allows a tighter security. I'm not too stressed either way. All my hosts run sshd anyway and the security is not in whether tcp22 is open or not, it's in what I put in sshd_config. With the puppet design, the puppet daemon must be running (or a cronjob) and puppet can self host that along with nrpe, munin and all the other crap that gets installled so I can do my job :-) My issue with puppet is not it's network architecture but with it's convoluted config language that I can't wrap my brains around. Plus the re-use of similar keywords to mean quite different things meaning I have to read 5 topics in the manual to get stuff working. Nagios btw has the same problem hence why I'm switching to Icinga 2 which fixes Nagios's config language once and for all. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Ansible, puppet and chef
On 2014-09-17 14:07, Alan McKinnon wrote: Nagios btw has the same problem hence why I'm switching to Icinga 2 which fixes Nagios's config language once and for all. Or you can use hostgroups/templates and have all your configuration in files and in git. Depends what you like more.
Re: [gentoo-user] Ansible, puppet and chef
On 2014-09-17 10:08, Alan McKinnon wrote: That's almost exactly the same setup I have in mind. How complex do the playbooks get in real-life? The common role has about 70 tasks. It does almost everything covered in the handbook plus installs and configures additional stuff like postfix, nrpe, etc. The dom0 role has 15 tasks including monitoring, xen, grub. The domU role basically just configures rc.conf. An actual web server with apache/php has just about 20 tasks. A load-balancer with varnish/nginx/keepalived has just about the same. A database has about 30 tasks because it also configures database replication.
Re: [gentoo-user] Ansible, puppet and chef
On 17/09/2014 14:46, Tomas Mozes wrote: On 2014-09-17 10:08, Alan McKinnon wrote: That's almost exactly the same setup I have in mind. How complex do the playbooks get in real-life? The common role has about 70 tasks. It does almost everything covered in the handbook plus installs and configures additional stuff like postfix, nrpe, etc. The dom0 role has 15 tasks including monitoring, xen, grub. The domU role basically just configures rc.conf. An actual web server with apache/php has just about 20 tasks. A load-balancer with varnish/nginx/keepalived has just about the same. A database has about 30 tasks because it also configures database replication. That doesn't seem too bad - almost manageable :-) -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Ansible, puppet and chef
We use bcfg2, and all I can say is to stay away. XML abuse runs rampant in bcfg2. From what I've heard from other professional sysadmins, Puppet is the favorite, but that's mostly conjecture. Alec On 09/16/2014 04:43 PM, Alan McKinnon wrote: Anyone here used ansible and at least one of puppet/chef? What are your thoughts? I've made several attempts over the years to get puppet going but never really got it off the ground. Chef I stay away from (likely due to the first demo of it I saw and how badly that went) Puppet seems to me a good product for a large site with 1000 hosts. Not so much for ~20 or so. Plus puppet's language and configs get large and hard to keep track of - lots and lots of directory trees with many things mentioning other things. (Nagios has the same problem if you start keeping host, services, groups and commands in many different files) I've stumbled upon ansible, it seems much better than puppet for smallish sites with good odds I might even keep the whole thing in my head at any one time :-) Anyone care to share experiences?
