This is a wrap-up update with the last next steps (for now) on the Pulp3 unit test discussion.
1. Here is a docs PR adopting simple but specific policy language recommending unit tests for Pulp3: https://github.com/pulp/pulp/pull/3411 2. We need to move our existing unit tests into their "forever home" so @daviddavis and I groomed this issue and put it on the sprint: https://pulp.plan.io/issues/3553 For any areas of Pulp3 untested code that you want to add unit tests for please file a Task in Redmine for that. A few of those have been filed AIUI. If there are any issues with these changes please bring them up. Any aspect of them can be rethought. David and I have tried to facilitate what we've heard from the group. On Mon, Mar 26, 2018 at 5:13 PM, Brian Bouterse <bbout...@redhat.com> wrote: > I want to summarize what I've heard to facilitate some next steps and > further discussion. > > There seems to be broad support and no -1 votes to the idea of a soft > check that tracks unit test coverage, so we wanted to get that out of the > way. Daviddavis enabled unit test coverage reporting for all Pulp3 PRs ( > https://github.com/pulp/pulp/pull/3397) and it will report on all PRs > now. Currently, it shows 54.98% for pulpcore. That number is surprisingly > high but not awesome. When looking at the report, it is mainly all import > statements and function definitions since we have few/zero unit tests but > also not much code. > > Based on feedback it sounds like leaving it at a soft check and highly > encouraging unit tests with each PR is something we could all agree on. I > want us to get to specific language that we can add into the Pulp3 docs as > a new section called "Unit Tests" here: https://docs.pulpproject.org/ > en/3.0/nightly/contributing/index.html Here is a starting point, please > send suggestions: "All new code is highly encouraged to have basic unit > tests that demonstrate its functionality. A Pull Request that has failing > unit tests cannot be merged." > > > Also from the convo on-list and on-irc, here are some questions I would > like help answering: > > * What areas in the existing codebase would really benefit from unit > testing? I think we need a classpath list of modules and classes. I made an > etherpad here: https://etherpad.net/p/Pulp_Unit_Test_Candidates > > * What are the existing unit tests and where do they live? > > * What docs need to be added to make contributing unit tests reasonable? > > > Thanks for all the discussion! > -Brian > > > On Mon, Mar 26, 2018 at 4:01 PM, Jeremy Audet <jau...@redhat.com> wrote: > >> > I'm also generally -1 against trying to pick a number (100%, 80%, 60%) >> up-front. We should unit test what makes sense to unit test, push that >> number as high as reasonable, and otherwise focus on pulp-smash, which I >> think has historically been more useful. >> >> QE is flattered by the focus on Pulp Smash. We're happy that the smoke >> tests are being executed as a pull request check. >> >> However, it's important to remember that unit tests are far faster >> than integration tests, typically by several orders of magnitude. In >> addition, Pulp Smash's smoke tests are intentionally limited. They're >> designed to execute quickly and to detect catastrophic regressions. >> They're not intended to be comprehensive. In fact, some of the >> already-written test cases may be stripped of their "smoke test" >> status for the sake of speed. Psychology is important here: it's bad >> if a developer locally fires off smoke tests, gets bored, and opens up >> a new web browser tab. >> >> Please keep this limitation in mind when deciding on policies >> regarding smoke tests. >> >> _______________________________________________ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> > >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev