> From my POV I'd like to be able to push new micro-releases to Ubuntu > stable releases. Doing so would require a Micro Release Exception [1]. > > [1]: https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions > > In order to get such an exception the following criteria need to be met: > > 1. upstream supports micro-version updates to stable releases > 2. upstream has a sufficiently high level of regression testing > for their stable releases > 3. regression tests are enabled in the package's build > > New features (even small) definitely don't comply with #1. Bug-fix-only > patches should be pushed to a micro release. Having a document outlining > what is considered a bug fix and what can be accepted in micro-releases > would be helpful here. >
As I noted previously, the line between "bug" and "feature" isn't that sharp here; the ticket could just as easily been phrased as a bug (this part of the interface doesn't work) and, as such, would have been unexceptional. > As for point #2 are there any metrics that shows code coverage of the > regression tests? What's the status of the unit test (from test/) and > spec tests (from spec/)? How often are regression tests run? Is there a > CI system setup somewhere? > Our test coverage needs a lot of work; we have recently brought on two people to work on it full time, but there is much to do. Our initial push is to get our cross platform regression testing harness more robust, fully populated, and useful. Code coverage metrics are difficult / unreliable with ruby when (as frameworks like puppet, rails, etc. do) code is generated at run time or otherwise metaprogrammed. -- M ----------------------------------------------------------- The power of accurate observation is commonly called cynicism by those who have not got it. ~George Bernard Shaw ------------------------------------------------------------ -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
