Hi Larry, On Fri, 14. Feb 2014 at 10:04:25 -0700, Larry Shaffer wrote: > I'm 100% for failing unit tests being a blocker for any release.
If all the tests were significant. That's not currently the case. All the render tests seem quite fragile. There are small differences, font mismatches across platforms and version and even tests that still require old renderers that we already removed. I don't see why those should block releases. That's my point. Currently there are a lot of constantly failing tests that make it harder to see when meaningful tests start to fail. That kind of ruins the point of running the tests in the first place. Don't get me wrong - of course these should either be fixed or removed - and once that happens, we can define failing tests as blockers. But this also has been brought up again and again, but not much action has been taken. > Another methodology would be to invest in a bunch of different CI servers on > VMs on OSGeo servers somewhere. Then, a dev could just review all failures > independent of contacting anyone else. That's a large investment in time and > resources, however. We already have that - didn't I mention that already? ;) The windows and debian/ubuntu/ubuntugis builds feed their test results into cdash[1] - since about April 2012. And your mac builds do to. So there should be enough coverage to find tests to fix. I don't think that that is the problem. Lack of time is probably the main reason for the situation and I doubt that pressure will help with that. Jürgen [1] http://dash.orfeo-toolbox.org/index.php?project=QGIS -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50 Software Engineer D-26506 Norden http://www.norbit.de QGIS PSC member (RM) Germany IRC: jef on FreeNode -- norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH Rheinstrasse 13, 26506 Norden GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502 _______________________________________________ Qgis-developer mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/qgis-developer
