I'm +0 on this and would certainly be in favour of a relaxed approach, I've seen too much time wasted on writing useless tests on setters/getters just to artificially boost a code coverage metric while real sections of code were left untested, or tests that were shallow but met the coverage guide lines went through.
However I think it is a good idea to be aware of our coverage levels and to be able to see if a module takes a big downward spike. Ian On 27 September 2015 at 07:41, Andrea Aime <[email protected]> wrote: > HI all, > the discussion kind of stopped here. Ben, are you satisfied with a more > relaxed approach > (like, case by case, or max % of reduction) to checking coverage in pull > requests? > > Cheers > Andrea > > > On Mon, Sep 21, 2015 at 9:26 AM, Simone Giannecchini < > [email protected]> wrote: > >> Ciao Ben, >> I see your point but I think we can mediate here. >> >> Coveralls allows to set up rules that make a build fail if global >> coverage goes below a certain global threshold or if the PR reduces it >> of more than x%. >> This means we can see right away trends and make sure that we are not >> below a good threshold. >> >> Since, afaik, we have a minimum goal for unit test coveralls can be >> used to enforce this upfront which is an extremely good thing. >> >> So I am +1, but I am also open to alternatives from others. >> >> Regards, >> Simone Giannecchini >> == >> GeoServer Professional Services from the experts! >> Visit http://goo.gl/it488V for more information. >> == >> Ing. Simone Giannecchini >> @simogeo >> Founder/Director >> >> GeoSolutions S.A.S. >> Via Poggio alle Viti 1187 >> 55054 Massarosa (LU) >> Italy >> phone: +39 0584 962313 >> fax: +39 0584 1660272 >> mob: +39 333 8128928 >> >> http://www.geo-solutions.it >> http://twitter.com/geosolutions_it >> >> ------------------------------------------------------- >> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 >> Le informazioni contenute in questo messaggio di posta elettronica e/o >> nel/i file/s allegato/i sono da considerarsi strettamente riservate. >> Il loro utilizzo è consentito esclusivamente al destinatario del >> messaggio, per le finalità indicate nel messaggio stesso. Qualora >> riceviate questo messaggio senza esserne il destinatario, Vi preghiamo >> cortesemente di darcene notizia via e-mail e di procedere alla >> distruzione del messaggio stesso, cancellandolo dal Vostro sistema. >> Conservare il messaggio stesso, divulgarlo anche in parte, >> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità >> diverse, costituisce comportamento contrario ai principi dettati dal >> D.Lgs. 196/2003. >> >> The information in this message and/or attachments, is intended solely >> for the attention and use of the named addressee(s) and may be >> confidential or proprietary in nature or covered by the provisions of >> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New >> Data Protection Code).Any use not in accord with its purpose, any >> disclosure, reproduction, copying, distribution, or either >> dissemination, either whole or partial, is strictly forbidden except >> previous formal approval of the named addressee(s). If you are not the >> intended recipient, please contact immediately the sender by >> telephone, fax or e-mail and delete the information in this message >> that has been received in error. The sender does not give any warranty >> or accept liability as the content, accuracy or completeness of sent >> messages and accepts no responsibility for changes made after they >> were sent or for other risks which arise as a result of e-mail >> transmission, viruses, etc. >> >> >> On Sun, Sep 20, 2015 at 9:21 PM, Ben Caradoc-Davies <[email protected]> >> wrote: >> > Andrea, >> > >> > are these rules based on coverage fraction or absolute lines covered? Is >> > coverage relative to cyclomatic complexity? >> > >> > -0 for rules that we never reduce code coverage. I can think of several >> > cases where it is quite acceptable to reduce coverage fraction, such as >> > fixing a leak by adding extra lines in a hard-to-cover catch block, or >> > disabling a failing test class or method to fix the build and preserve >> > the rest of the coverage. Given infinite time we can always add more >> > tests ... >> > >> > It might be quite useful to have a coverage change report for every pull >> > request. There is no harm trying Coverall to see what it looks like. I >> > am only against the rule that we never allow coverage to be reduced. It >> > might be good to know if a module's coverage falls below 40%. >> > >> > In my view, coverage metrics fall into the same category as formatting >> > convention compliance and lack of compiler warnings: good to have but >> > often a pain in the arse when enforced by automated tools. >> > >> > Kind regards, >> > Ben. >> > >> > On 19/09/15 19:13, Andrea Aime wrote: >> >> Hi, >> >> speaking with the OpenLayers guys this week they showed me how >> >> their pull requests check do not only include a travis build >> verification, >> >> but also a code coverage level check via Coveralls. >> >> >> >> Basically, their rule is that a pull request must never reduce the code >> >> coverage, and this can be automated via Travis and Coveralls. >> >> >> >> As far as I can see Coveralls is free for open source projects, what >> do you >> >> think of adding it to our pull requests checks? >> >> >> >> Cheers >> >> Andrea >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> >> >> >> >> >> _______________________________________________ >> >> GeoTools-Devel mailing list >> >> [email protected] >> >> https://lists.sourceforge.net/lists/listinfo/geotools-devel >> >> >> > >> > -- >> > Ben Caradoc-Davies <[email protected]> >> > Director >> > Transient Software Limited <http://transient.nz/> >> > New Zealand >> > >> > >> ------------------------------------------------------------------------------ >> > _______________________________________________ >> > GeoTools-Devel mailing list >> > [email protected] >> > https://lists.sourceforge.net/lists/listinfo/geotools-devel >> > > > > -- > == > GeoServer Professional Services from the experts! Visit > http://goo.gl/it488V for more information. > == > > Ing. Andrea Aime > @geowolf > Technical Lead > > GeoSolutions S.A.S. > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > phone: +39 0584 962313 > fax: +39 0584 1660272 > mob: +39 339 8844549 > > http://www.geo-solutions.it > http://twitter.com/geosolutions_it > > *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* > > Le informazioni contenute in questo messaggio di posta elettronica e/o > nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il > loro utilizzo è consentito esclusivamente al destinatario del messaggio, > per le finalità indicate nel messaggio stesso. Qualora riceviate questo > messaggio senza esserne il destinatario, Vi preghiamo cortesemente di > darcene notizia via e-mail e di procedere alla distruzione del messaggio > stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, > divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od > utilizzarlo per finalità diverse, costituisce comportamento contrario ai > principi dettati dal D.Lgs. 196/2003. > > > > The information in this message and/or attachments, is intended solely for > the attention and use of the named addressee(s) and may be confidential or > proprietary in nature or covered by the provisions of privacy act > (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection > Code).Any use not in accord with its purpose, any disclosure, reproduction, > copying, distribution, or either dissemination, either whole or partial, is > strictly forbidden except previous formal approval of the named > addressee(s). If you are not the intended recipient, please contact > immediately the sender by telephone, fax or e-mail and delete the > information in this message that has been received in error. The sender > does not give any warranty or accept liability as the content, accuracy or > completeness of sent messages and accepts no responsibility for changes > made after they were sent or for other risks which arise as a result of > e-mail transmission, viruses, etc. > > ------------------------------------------------------- > > > ------------------------------------------------------------------------------ > > _______________________________________________ > GeoTools-Devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geotools-devel > > -- Ian Turton
------------------------------------------------------------------------------
_______________________________________________ GeoTools-Devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-devel
