for what it's worth: for a while I was experimenting with cucumber to test scripts running on linux machines. And yes , IMHO it is the way to go .
It worked well with the behavior testing just a you guys described. still just as in regular development, bdd can be complemented with tdd. The way i did the tdd, was by using the exit codes of the shell commands. I created an abstraction in ruby for execute (local, remote) and upload, download of files. If the command did not have exit 0 (the default) then i imagined the test being failed. Using snapshots of the virtual machines, I ran the scripts against a virtual machine (used virtualbox), with the ability to do a rollback. I know this is not the way puppet normally operates, but I used the (if i recall correctly) the puppetrun command to execute it and see what happened If all of these exit codes where ok (similar to unit tests), i ran the bdd tests to see if everything was correct. By integrating the tdd, and the bdd within a CI-system, I was able to have it run continuously and rebuild the system over and over again. Another thing I want to mention is that you can easily abstract commands in a custom dsl, but I really like to see what actual commands got executed. This is slightly off topic maybe, but by logging/running the actual commands and have the whole environment build like that (including making a floppy, booting the vms) it allowed me to create a kind of install document with the actual commands instead of pointing to a newly invented DSL. So I say yes to an abstraction layer, but no to hiding the actual commands. just my 2 c On Nov 6, 3:56 pm, Lindsay Holmwood <lind...@holmwood.id.au> wrote: > 2009/11/6 Martin Englund <martin.engl...@sun.com>: > > > > > > > Folks, > > > I've been struggling a bit with how we're using puppet (at my job): > > how do you validate that puppet has done what it is supposed to, and > > even troublesome, how you validate that it has done what you intended > > it to do? > > > Since I'm the only one who is writing the puppet profiles and working > > with it on a daily basis, I'm the only one who can decipher the puppet > > logs. I often get the question: how do we know when the system is > > ready for production? > > > I've been playing around[1] with cucumber & webrat, and have pieced > > together a way to do behavior driven infrastructure testing. Puppet > > takes care of getting the system configured correctly, but there are > > often other pieces involved, like opening firewall ports, adding DNS > > entries, sendmail routing, etc. Which must be done outside of puppet > > to get the system ready for release. > > > When you write code, you always use unit testing & integration testing > > to verify that the application is working as expected, but why don't > > we use that when we install a system? > > > What are you using to verify that your system is correctly configured > > and behaves the way you want? > > > [1] <http://blogs.sun.com/martin/entry/behavior_driven_infrastructure> > > [although i've commented on the excellent blog entry, i'm posting here] > > Hi Martin, > It looks like there's a bit of crossover here with a project i've been > working on the last few months called cucumber-nagios[0]. It takes the > result of a Cucumber run and outputs it in the Nagios plugin format. > Essentially you use it to express your intentions in plain language, > and verify your intentions periodically through your monitoring > system. Just like what you've posted about. :-) > > Anyhow, I spoke about cucumber-nagios at the excellent Devopsdays in > Belgium last weekend, and I got talking with people about expanding > the library of steps to cover things like logins over SSH, file > manipulation, and mail delivery. It would be cool if we could > centralise our efforts and focus on building an awesome library of > reusable steps to test our infrastructure. > > Your point about doing behaviour driven development when writing > software is right on the mark. From an infrastructure perspective, I > like to think of Cucumber as the testing tool, and Puppet as the > programming language. > > Anyhow, i'd be interested to hear what other people think about this idea! > > Cheers, > Lindsay > > [0]http://auxesis.github.com/cucumber-nagios > > --http://holmwood.id.au/~lindsay/(me) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
