On Oct 2, 2008, at 3:11 AM, Paul Nasrat wrote: > >> ge commands. >> >> First, they tend to be slow -- slow enough that you don't really want >> them running every time you run your unit tests. This means you'd >> probably want to split out the rake tasks so that we default to only >> running the unit tests, and we have to go out of our way to run the >> integration tests. It sucks, but there's not much you can do about >> the fact that, for instance, gems sometimes take 30 seconds to do >> anything at all. > > This is a common thing I've seen in various development projects, you > want your unit tests to be fast so you can keep writing them and > running them. Integration tests are really important, but you don't > want to run them. I'd really like to see a public facing puppet > buildbot or something that runs the platform specific integration > tests and reports.
James has been working on it, but I guess he's too busy galivanting around the world to finish it up. :) I (and many others) completely agree that a buildbot is the right answer. > > I don't really see a big problem with having say different test > targets that call what you want, just default to sensible behaviour. > As every developer can't reasonably be expected to have every os > locally available having a good visible feedback cycle for integration > tests would be really useful. Okay. I'll open a ticket now. I'll plan on creating an 'integration_tests' rake task, along with an 'all_tests' task, I guess. > >> Second, you'd really like to have integration tests that actually >> install and remove packages, but to do that you need test packages. >> I've kind of got this in test/ral/providers/package.rb, but it's not >> very flexible -- you can't override it with your own packages -- and >> it just seems hard to maintain. > > We could have a separate test artifact repository to keep packages in, > and the tests could grab them via http or whatever from there. We want > controllable tests so that the packages are known and the states are > known, but I'd be hesitant to put them inline with the puppet src. I was even thinking simpler things like package names -- e.g., using apt to install a test package. You can only use packages that aren't already installed, and you're basically guaranteed that *someone* has that test package already installed. > >> I *think* we should end up with a unit and integration test for every >> provider, and each provider's integration test should at least do >> some >> minimal stuff -- install and remove a package, list all packages. If >> we do this, do we split up unit and integration tests? > > You already have a dir structure split so it's just wiring the > Rakefile and autotest config differently and setting up an environment > for integration tests. Yeah. -- Smoking is one of the leading causes of statistics. -- Fletcher Knebel --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
