Note: Descartes (or Pitest) seems be a bit lost with tests located on abstract classes. For example it pretty much react as if those 3 classes where not existing: https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-core/xwiki-commons-filter/xwiki-commons-filter-api/src/test/java/org/xwiki/filter/internal (which makes the score way lower than it should I think).
On Thu, Aug 9, 2018 at 6:10 PM, Thomas Mortagne <[email protected]> wrote: > Done: https://github.com/STAMP-project/descartes-usecases-output/pull/1 > > On Thu, Aug 9, 2018 at 3:07 PM, Thomas Mortagne > <[email protected]> wrote: >> Started looking at xwiki-commons-extension-api by the way. >> >> On Thu, Aug 9, 2018 at 11:50 AM, Thomas Mortagne >> <[email protected]> wrote: >>> This is probably because xwiki-commons-extension-api contains a lot of >>> what is closer to integration than unit tests. >>> >>> On Tue, Aug 7, 2018 at 6:07 PM, Vincent Massol <[email protected]> wrote: >>>> Note for Thomas (or whoever else is interested in improving the commons >>>> extension tests) there are plenty in >>>> >>>> ./xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/target/pit-reports/201808071749/methods.json:pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> pseudo-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> pseudo-tested >>>> partially-tested >>>> partially-tested >>>> pseudo-tested >>>> partially-tested >>>> partially-tested >>>> partially-tested >>>> partially-tested >>>> partially-tested >>>> pseudo-tested >>>> pseudo-tested >>>> >>>> Fixing those should yield a good increase in both mutation score and test >>>> coverage. >>>> >>>> Thanks >>>> -Vincent >>>> >>>>> On 7 Aug 2018, at 18:05, Vincent Massol <[email protected]> wrote: >>>>> >>>>> >>>>> >>>>>> On 30 Jul 2018, at 12:39, Vincent Massol <[email protected]> wrote: >>>>>> >>>>>> Hi devs, >>>>>> >>>>>> It would be great if you could help improve our unit tests using >>>>>> Descartes. This is needed for the STAMP research project >>>>>> (https://www.stamp-project.eu/view/main/) and will benefit XWiki by >>>>>> having 2 effects: >>>>>> * increasing the test coverage >>>>>> * improving the tests themselves (increasing their mutation score) >>>>>> >>>>>> Since 10.7 is 50% testing and 50% BFD, it would be great if you could >>>>>> spend all or a substantial part of your testing time working on this. >>>>>> >>>>>> I propose the following strategy: >>>>>> * You find a module you want to work on. >>>>>> * In that module you run: mvn clean install -Pquality >>>>>> -Dxwiki.pitest.skip=false >>>>> >>>>> You can also run that at the top of a multimodule project and then find >>>>> pseudo/partially tested methods with: >>>>> >>>>> find . -name "methods.json" -exec egrep -oH >>>>> "pseudo-tested|partially-tested" {} \; >>>>> >>>>> Thanks >>>>> -Vincent >>>>> >>>>>> * Then you check target/pit-reports/<date>/issues/index.html and verify >>>>>> if there are "pseudo tested" methods listed (when we have finished >>>>>> fixing all of those we can move to “partially tested methods”). >>>>>> * If there are some, then please record the current jacoco threshold and >>>>>> the current mutation score. >>>>>> * You can get the jacoco threshold by running "mvn clean install >>>>>> -Pquality -Dxwiki.pitest.skip=false >>>>>> -Dxwiki.pitest.mutationThreshold=100” (or by checking >>>>>> target/pit-reports/<date>/index.html, I haven’t checked yet if they are >>>>>> the same). >>>>>> * You can get the current mutation score by checking >>>>>> target/pit-reports/<date>/index.html >>>>>> * Then fix the test so that Descartes doesn’t report any pseudo tested >>>>>> or partially tested methods >>>>>> * Update the jacoco threshold and the mutation scores in the pom.xml >>>>>> * Send a PR on >>>>>> https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki >>>>>> using the format already defined there. >>>>>> >>>>>> WDYT? Doable? >>>>>> >>>>>> Thanks >>>>>> -Vincent >>>> >>> >>> >>> >>> -- >>> Thomas Mortagne >> >> >> >> -- >> Thomas Mortagne > > > > -- > Thomas Mortagne -- Thomas Mortagne
