Sounds good. +1 Thanks, Marius
On Wed, Oct 3, 2018 at 11:56 AM Vincent Massol <vinc...@massol.net> wrote: > Hi devs, > > We started discussing a first global test coverage strategy in > https://markmail.org/message/grphwta63pp5p4l7 > > I’d like to propose some updates and tuning now that we have a Clover > Jenkins pipeline working (brainstormed with Simon): > > Here’s the new proposal: > > * We run the Clover Jenkins pipeline every night (between 11PM-8AM) > * The pipeline sends an email whenever the new report has modules lowering > the global TPC score compared to the baseline report (negative contribution > per module) > * The baseline report is the report generated just after each XS release. > This means that we keep the same baseline during a XS release > ** Technically it means that the pipeline will update the latest.txt file > (which contains the clover report timestamp) when it notices a version > change > * We add a step in the Release Plan Template to have the report passing > before we can release. > * The RM is in charge of a release from day 1 to the release day (already > the case), and is also in charge of making sure that the global coverage > job failures get addressed before the release day so that we’re ready on > the release day. > > Options: > * Make it easier and only fail the pipeline job when the global TPC value > is lower than the baseline (vs failing whenever a module has negative > contribution) > > WDYT? > > Thanks > -Vincent > >
