On Apr 24, 2012, at 4:11 PM, Joe Gordon wrote: > Hi All, > > I would like to propose a minimum required code coverage level per file in > Nova. Say 80%. This would mean that any new feature/file should only be > accepted if it has over 80% code coverage. Exceptions to this rule would be > allowed for code that is covered by skipped tests (as long as 80% is reached > when the tests are not skipped). >
I like the idea of looking at code coverage numbers. For any particular merge
proposal, I'd also like to know whether it increases or decreases the overall
code coverage of the project. I don't think we should gate on this, but it
would be helpful for a reviewer to see that, especially for larger proposals.
> With 193 python files in nova/tests, Nova unit tests produce 85% overall code
> coverage (calculated with ./run_test.sh -c [1]). But 23% of files (125
> files) have lower then 80% code coverage (30 tests skipped on my machine).
> Getting all files to hit the 80% code coverage mark should be one of the
> goals for Folsom.
>
I would really like to see a visualization of the code coverage distribution,
in order to help spot the outliers.
Along these lines, there's been a lot of work in the software engineering
research community about predicting which parts of the code are most likely to
contain bugs ("fault prone" is a good keyword to find this stuff, e.g.:
http://scholar.google.com/scholar?q=fault+prone, big names include Nachi
Nagappan at MS Research and Elaine Weyuker, formerly of AT&T Research). I would
*love* to see some academic researchers try to apply those techniques to
OpenStack to help guide QA activities by identifying which parts of the code
should get more rigorous testing and review.
Take care,
Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
