Re: [Puppet Users] Automating infrastructure tests on Puppet nodes after a puppetrun
On Tuesday 19 Oct 2010 21:52:37 Nicolas Szalay wrote: - Al @ Lab42 lab42...@gmail.com a écrit : | Hi List, Hi, | I would like to discuss with whoever is interested one topic that I | suppose has general interest. | | I want to implement some kind of automatic testing on the status of a | node after a Puppet Run. | These tests involve trivial and less trivial things things like: | - A local service is running | - A local port is open | - A remote server on a remote port is reachable by the node | - An URL replies with an expected content | - Some specific function needed by the node and provided by a remote | host is working (ie: ldap acces for users authentication, ntp | sync...) | - Whatever other check that asserts that the node is correctly | working OK, so we do this slightly differently however it might help... 1) All our manifests are stored in Git 2) A Git update forces Hudson to run a build 3) The Build process performs the following steps: * Checkout the latest version of the manifest into the staging server's puppet-module-path * Start a virtual server of the defined type using cucumber-vhost[0] * Use puppet to deploy the latest staging versions of the manifests to the virtual-server * run cucumber tests (using webrat for webservices and SMTP/IMAP libraries to test sending/delivery of email) against the service/facility that is contained in the manifests we are testing * Report back on the results of those tests * Destroy the virtual server This means that all of our manifests are fully tested before they go near our production system and we can be confident (although obviously this is only as good as the tests that we write!) that when we merge from staging into master the changes that are rolled out will work correctly. I've not gone down the cucumber-puppet root as I'm not 100% sure how it works and how to write stories correctly so if anyone can point me at a good resource on this, I'd be very appreciative! Hope that helps, Matt -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Automating infrastructure tests on Puppet nodes after a puppetrun
* Matt Wallace [2010-10-20]: I've not gone down the cucumber-puppet root as I'm not 100% sure how it works and how to write stories correctly so if anyone can point me at a good resource on this, I'd be very appreciative! I have put up some documentation at http://projects.puppetlabs.com/projects/cucumber-puppet/wiki If that doesn't get you started, feel free to ask here or email me directly. I would be glad to update the documentation in case anything is unclear. cheers, Nikolay -- It's all part of my Can't-Do approach to life. Wally -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Automating infrastructure tests on Puppet nodes after a puppetrun
Hi List, I would like to discuss with whoever is interested one topic that I suppose has general interest. I want to implement some kind of automatic testing on the status of a node after a Puppet Run. These tests involve trivial and less trivial things things like: - A local service is running - A local port is open - A remote server on a remote port is reachable by the node - An URL replies with an expected content - Some specific function needed by the node and provided by a remote host is working (ie: ldap acces for users authentication, ntp sync...) - Whatever other check that asserts that the node is correctly working I want to do this directly in my modules, at least for the checks that are directly related to the resources provided by the module and build some defines to manage quickly things like check the url or check if the remote port is accessible. The point is to have a solid testing infrastructure, early notification of any problem that might take place after a Puppet run and, at the same time have a sort of monitoring logic that might be used also by other tools, like Nagios. In order to achieve something like this there are different approaches and I would like to follow what seems most sane and, mostly, what could better fit the evolution of the Puppet ecosystem. Here a pair of examples: - APPROACH 1 - CHECK TRIGGERED BY PUPPET NODE After the Puppet run a script/command is launched and makes the necessary checks (built on the node in a dinamic way, according to the modules installed). If I'm not wrong in recent Puppet versions there's an hook that makes you run custom commands after (or before? or both?) the execution of the puppet run, so this might be the way to automate the start of the checks without too many hassles. The cons are that everything is done on the node and there's not (if not implemented specifically) a centralized management of checks runs, process logic, notifications and history. - APPROACH 2 - CHECK RUN BY AN MCOLLECTIVE CLIENT ON THE PUPPET NODE This somehow intrigues me and requires the node to have a mcollective server deamon running. The automation might be triggered remotely by the mcollective client using mcollective agents available on the Puppet node. The mcollective client should be notified of the puppetrun and might not be the same PuppetMaster, and a way to do this might be via a custom report extention that reports directly to the mcollective client. The benefit is that the monitoring can be managed via mcollective and there's a central point where data are collected and commands executed. The list of checks to be done on the client should, IMHO, remain on the Puppet client (mcollective server) itself (no need to have store configs for this) and maybe a specific agent might be done to retrieve and run from the mcollective client the list of checks to perform. Another point is how to organize and define the checks' list. Cucumber seems a nice and somehow standard way to define the checks logic, but could be also a plain execution of the different checks from a sort of wrapper script. The single checks could be nrpe commands and/or mcollective agents (I love the nettest one, incidentally). AFAIK there's nothing in the above examples that is particularly difficult or can't be done with existing tools, but I would like to introduce them seamlessly in my modules (using my monitoring abstraction classes). So, I wonder if someone is already doing similar checks, what's the approach they are following and what might be the evolution of Puppet under regarding these topics. Any further or related idea is welcomed, Alessandro Franceschi -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Automating infrastructure tests on Puppet nodes after a puppetrun
- Al @ Lab42 lab42...@gmail.com a écrit : | Hi List, Hi, | I would like to discuss with whoever is interested one topic that I | suppose has general interest. | | I want to implement some kind of automatic testing on the status of a | node after a Puppet Run. | These tests involve trivial and less trivial things things like: | - A local service is running | - A local port is open | - A remote server on a remote port is reachable by the node | - An URL replies with an expected content | - Some specific function needed by the node and provided by a remote | host is working (ie: ldap acces for users authentication, ntp | sync...) | - Whatever other check that asserts that the node is correctly | working | | I want to do this directly in my modules, at least for the checks | that are directly related to the resources provided by the module | and | build some defines to manage quickly things like check the url or | check if the remote port is accessible. | | The point is to have a solid testing infrastructure, early | notification of any problem that might take place after a Puppet run | and, at the same time have a sort of monitoring logic that might be | used also by other tools, like Nagios. Do you know about puppet-cucumber ? | In order to achieve something like this there are different | approaches and I would like to follow what seems most sane and, | mostly, what could better fit the evolution of the Puppet ecosystem. | | Here a pair of examples: | | - APPROACH 1 - CHECK TRIGGERED BY PUPPET NODE This is an easy approach but how will you push information back to you ? I have not checked but I don't think that the result of post run hooks are included into reports | - APPROACH 2 - CHECK RUN BY AN MCOLLECTIVE CLIENT ON THE PUPPET NODE I would use that one, combined with nagios through the mc nrpe agent probably or something like a hudson instance to do a permanent check about this. | Another point is how to organize and define the checks' list. | Cucumber | seems a nice and somehow standard way to define the checks logic, | but could be also a plain execution of the different checks from a | sort of wrapper script. | The single checks could be nrpe commands and/or mcollective agents (I | love the nettest one, incidentally). | | | AFAIK there's nothing in the above examples that is particularly | difficult or can't be done with existing tools, but I would like to | introduce them seamlessly in my modules (using my monitoring | abstraction classes). | | So, I wonder if someone is already doing similar checks, what's the | approach they are following and what might be the evolution of Puppet | under regarding these topics. Not doing it but definitely interested. Nico. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Automating infrastructure tests on Puppet nodes after a puppetrun
- Al @ Lab42 lab42...@gmail.com wrote: Hi List, I would like to discuss with whoever is interested one topic that I suppose has general interest. I want to implement some kind of automatic testing on the status of a node after a Puppet Run. These tests involve trivial and less trivial things things like: - A local service is running - A local port is open - A remote server on a remote port is reachable by the node - An URL replies with an expected content - Some specific function needed by the node and provided by a remote host is working (ie: ldap acces for users authentication, ntp sync...) - Whatever other check that asserts that the node is correctly working sounds like things you want to monitor anyway in an ongoing manner? So assuming you have monitoring for all of this, is the problem that you want visibility of the state right now after a run and not when nagios gets round to doing its next checks which might be many minutes? I favor nrpe - cos I can deploy my check logic with puppet - but I really think you want your monitoring to cover all of this. To answer the 'now' part of it, I'd just notify via mcollective my nagios box to do a check for all services on the node post puppet run. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
