Issue #13509 has been updated by Chris Price.
awesome, thanks Justin. We would greatly appreciate any feedback or suggestions that anyone has. ---------------------------------------- Feature #13509: Establish or solidify pattern for testing modules https://projects.puppetlabs.com/issues/13509#change-58865 Author: Chris Price Status: Accepted Priority: Normal Assignee: Category: Target version: Telly Affected Puppet version: Keywords: Branch: This probably isn't actually going to happen in the Telly time frame, but I wanted to make sure that this ticket doesn't slip through the cracks. We need a consistent plan / pattern for testing modules--both for internal use and for third-party module developers. There are lots of moving pieces here... Nick L. brought a few very salient points to our attention recently: 1. How do we decide what versions of puppet to run against? There are a lot of sub-questions here as well--how do we automate the tests against the desired matrix of puppet versions? Can we somehow record the compatibility results of the tests and add them to the metadata for the module automatically? If we decide on some framework for writing and running module tests, and it relies on API in puppet core that does not exist yet, then we're guaranteeing that modules will only be able to be tested against recent versions of puppet in each series (2.7.x, master, etc.). Is this a problem? 2. The modules will probably have some dependencies on *something* in puppet core--something like a "puppet core spec api", e.g. to request setup/teardown of test state for things like puppet settings. How do we expose this somewhere where it is accessible to modules? We probably don't want to force devs to check out the actual puppet source if we can avoid it, but if they're developing against puppet from a distro (deb/rpm/etc) then it's likely that our current spec_helper and related libs will not be bundled with their distro. We may need to add something inside the main "lib" folder that provides this API so that we know it will get included in distros. (This issue came up #13439, and resulted in a hacky pull request: https://github.com/puppetlabs/puppet-grayskull/pull/94 . That one was subsequently replaced with the slightly improved: https://github.com/puppetlabs/puppet-grayskull/pull/96 , but we probably need a more general solution. As Nick pointed out, this gives us a general solution for dealing with Puppet::Util::Settings setup / teardown, but it does not deal with the rest of the stuff that is happening in spec_helper... and it would be better to have a single setup/teardown endpoint somewhere in the puppet code that could handle everything.) 3. Rubies? 4. CI integration? 5. Interface? Nick suggested maybe a face/action "puppet module test", which would be awesome... Other considerations...? -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
