On Oct 2, 2008, at 12:57 PM, Andrew Shafer wrote: > > Here are my thoughts... > > Generally: > • Unit tests should be fast (1000s in minutes) > • Unit tests should be run regularly (Dev's should be running > autotest, and the suite should ideally run every commit) > • Integration tests cannot always be fast > • Integration tests should run 'often', at least everyday if feasible > • Breaking integration tests should motivate new unit tests. > The idealize solution in my mind is the test platforms be a basic VM > without puppet and tests installation and rc scripts as part of the > process. Once you get to packages, everything is in a known state, > just use a test package per platform and take it through the > possible state transitions.
This means you need the ability to automatically create packages for every platform you're testing (otherwise you're only testing against released packages), which I think might be a bridge too far for this. > > Once you get to the networked package manager things get a little > more complicated, not conceptually, but just to maintain the > necessary infrastructure. > > And for platforms where a VM doesn't make sense, there is probably > going to be some rough edges to work out, particularly in the case > of 'failure' to get the things in a known state. This will probably > be a little clumsy and ad hoc at first, but I don't see a clear way > around that. And it might be the way all platforms start out :/ I basically agree with that, which I think is the structure we're building for the buildbot, right? I hope that platform advocates can generally maintain the individual machines or VMs, but otherwise, yeah. -- 'Tis better to be silent and be thought a fool, than to speak and remove all doubt. --Abraham Lincoln --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
