On 11/27 20:57, Nir Soffer wrote: > Hi all, > > Thanks to Edward, we have now coverage reports in jenkins. > > The way to access the report on jenkins is to use this url: > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/<build-number>/artifact/exported-artifacts/htmlcov/index.html > > Here is an example, hopefully it will be accessible when you try: > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/index.html > > Todo: > > - Easy way to access the report from gerrit > It should be easy to add a link the coverage report in the comment added > by jenkins after a build finish.
Not really, the gerrit comments that jenkins sends can't be changed by the build, that means that you can't put put there a non-static url. There's a quite tricky way that can be done with groovy postscripts and accessing the jenkins inner objects, but that's very likely to break on any plugin upgrade and requires a bit of investigating with objects to change. > > - Store the coverage reports for longer time, maybe a week? If they are generated with the check patch, you always have the latest there (stored as long as your builds). If you want longer archiving, might be a good idea to generate them too in the check-merged. > > - We have only 45% coverage instead of the minimum, 100%. > > Note that coverage of 25% can mean *no* code was run by the tests. > The only code > running was the functions and class definitions. Here is an example: > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_devicemapper_py.html > > Modules that were never imported during the tests have 0% coverage: > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/artifact/exported-artifacts/htmlcov/_home_jenkins_workspace_vdsm_master_check-patch-fc23-x86_64_vdsm_vdsm_storage_hsm_py.html > > - coverage creates lot of useless noise in the jenkins logs, need to slicense > this output. > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/648/consoleFull > I did not find a way to do this in nosetests, may need hacking > nose coverage plugin. > > - The report includes only the tests in the tests directory. > > We have additional tests in lib/vdsm/infra/* which are not > included. We should > move these to the tests directory. > > - The report is using absolute paths, but we like shorter relative paths. > > I don't see a way to configure nosetests or coverage to generate > relative paths. > May need hacking of the generated html/json in htmlcov. If you find it please tell me, I have the same issue with other projects, I'll do if I discover a way :) > > - Add "make coverage" target for running coverage locally > > - An easy way to enable coverage for the functional tests or for running > a single test module. > > Can be done using nosetests --cover* options. Should be documented in > tests/README, and maybe automated using a script or Makefile. > When running locally, one would like to have the script open the report > in the browser automatically: > xdg-open tests/htmlcov/index.html > > - An easy way to enable coverage when testing flows in vdsm > > Petr sent a patch for enabling coverage using vdsm.conf: > https://gerrit.ovirt.org/49168 > We discussed adding vdsm-coverage package that will make it easy to setup > > Nir -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: [email protected] IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605
signature.asc
Description: PGP signature
_______________________________________________ Devel mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/devel
