Hi All, In last week's Cross Project IRC meeting [1] I sought to determine the merits and feasibility of having a central web location for code coverage of various/all OpenStack projects.
Presently a few projects add code coverage as a non-voting gate and that provides a per-commit viewing option, for example [2],[3]. I would first propose after a merge is committed, that coverage that was generated for the commit is copied to a set and fixed URL so this can be easily referenced in documentation, etc. (e.g. coverage.openstack.org/openstack-infra/cover and coverage.openstack.org/openstack/designate/cover respectively) The purpose of recording code coverage is not to down vote any code contributions due to lack of code coverage. The purpose includes: 1. Provide a central location to see current code coverage across projects, and provide known public URLs. 2. Enable an entry point for discussion of improving code coverage, especially as a means for newer developers to contribute to projects. 3. To provide analysis of project code coverage by recording coverage reports over time. fungi provided https://review.openstack.org/#/c/192253/ which is a draft spec for a much wider ci-watch effort however I see this a more specific example. I am seeking feedback from various projects on interest, concerns and recommendations before creating a spec. I see the initial steps to implement this as. 1. Agree on future domain url location for reports (e.g. coverage.openstack.org) 2. Add to the merge committing process a step similar to the existing coverage-log publisher (project-config/jenkins/jobs/macros.yaml) to agreed location. 3. Provide initially a top level index page at url for published reports These steps enable a single current occurrence of code coverage per project. It is not the objective to keep the historical HTML output. Following this, the next steps are to record coverage for historical analysis with the following. 1. Alter tox.ini [testenv:cover] to produce a text based version (aka coverage report -m). This is much easier to parse and compare different versions. I would propose we keep a weekly/daily dated file of this output, which would be automatically published as per previous publisher. 2. Use these dated files to produce basic per project analysis of lines/percentage coverage over time, publishing this to a set directory structure (e.g. /analysis/) and linked from the top level index page. Over time this can be improved to provide simple graphical output. [1] http://eavesdrop.openstack.org/meetings/crossproject/2015/crossproject.2015-08-25-21.00.html [2] http://logs.openstack.org/82/165682/8/check/nodepool-coverage/08c6340/cover/ [3] http://logs.openstack.org/38/212738/12/check/designate-coverage/bc2e329/cover/
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
