+0 with the mention that the process needs to be documented well on the Release Plan. As a RM I will need to know where and at what job / report to look at. Also our ci is very slow and the release process IMO gets harder and harder because of the long waiting times, sometimes waiting for 5 hours to have a build. I don't know how long this pipeline will take, but maybe we could improve somehow the machines the jobs are running.
Thank you, Caty On Wed, Oct 3, 2018 at 1:34 PM Marius Dumitru Florea < mariusdumitru.flo...@xwiki.com> wrote: > Sounds good. +1 > > Thanks, > Marius > > On Wed, Oct 3, 2018 at 11:56 AM Vincent Massol <vinc...@massol.net> wrote: > > > Hi devs, > > > > We started discussing a first global test coverage strategy in > > https://markmail.org/message/grphwta63pp5p4l7 > > > > I’d like to propose some updates and tuning now that we have a Clover > > Jenkins pipeline working (brainstormed with Simon): > > > > Here’s the new proposal: > > > > * We run the Clover Jenkins pipeline every night (between 11PM-8AM) > > * The pipeline sends an email whenever the new report has modules > lowering > > the global TPC score compared to the baseline report (negative > contribution > > per module) > > * The baseline report is the report generated just after each XS release. > > This means that we keep the same baseline during a XS release > > ** Technically it means that the pipeline will update the latest.txt file > > (which contains the clover report timestamp) when it notices a version > > change > > * We add a step in the Release Plan Template to have the report passing > > before we can release. > > * The RM is in charge of a release from day 1 to the release day (already > > the case), and is also in charge of making sure that the global coverage > > job failures get addressed before the release day so that we’re ready on > > the release day. > > > > Options: > > * Make it easier and only fail the pipeline job when the global TPC value > > is lower than the baseline (vs failing whenever a module has negative > > contribution) > > > > WDYT? > > > > Thanks > > -Vincent > > > > >
