We haven't had a unit test policy in Pulp 3, and the community and core committers would all like one. The general desire we've heard so far is to change course and encourage developers to add unit tests to their changes to Pulp 3.
The policy we're suggesting is to add a coveralls check for Pull Requests against the pulpcore 3.0-dev branch that shows the overall coverage percentage, e.g. 12.89%. This check would pass if and only if coverage increases or remains the same with the PR. We think this will eventually get us on the path to 100% unit test coverage. We propose the coveralls check be a soft check that allow for merging if it fails. We would document the policy and try to adhere to it even though it wouldn't formally block merging. At a future point when pulp3 (maybe the GA?) we could make this a hard check. Benefits: - It's easy, simple, and automatic - It's pretty objective and there's little room to argue with a number. - Helps us raise our unit test coverage gradually over time Downsides: - Could discourage community contributions - It can be a bit strict and unforgiving at times (especially if there's a hard check) - It only provides a guarantee around quantity of unit testing and not quality *Q: What about the existing functionality? Do we need to write unit tests for it?* We're not sure about this. We'd like community feedback. Is anyone interested in writing backfill unit tests? If so, then maybe we should. *Q: Will this also affect Pulp 2?* We're not planning on it but if we like this enough, we can look at adding it to Pulp 2. *Q: Will this affect plugins?* We want to start out with just pulpcore now and see how we like it. Then we'll look at adding it to pulp_file. In the future, we can also look at ways to make it easy for plugins to set this up. *Q: Do I no longer need to write pulp-smash test plan issues in Github for Pulp 3 features?* No, you should still do that.  https://coveralls.io/
_______________________________________________ Pulp-dev mailing list Pulpfirstname.lastname@example.org https://www.redhat.com/mailman/listinfo/pulp-dev