Hi, While working on the test suite and trying to get all tests to pass I realized that there are a lot of tests that exist in two versions: C++ and python (e.g. composerhtml, atlas composition). The problem with that is, that they work on the same set of reference images and either one or the other fails because of slight inconsistencies in the result (nothing we should worry about, see e.g. [1]). I don't know exactly, what the source of these is, I assume that either there are some rounding/floating point differences in the python API or that meanwhile one or the other test was modified but not both synchroneously.
What can we do? (1) We could accept these deviations with anomalies or multiple reference images. (2) Or we could remove the C++ test in favor of the python test which will test the same functionality/codepath and the python binding in addition. That would have the advantage that maintainability will be easier, because not every of these tests needs to be written and adjusted twice. The only disatvantage I can see is that you then don't know if it's the python binding or the logic behind it that made a test fail. But that shouldn't make a big difference in the time one needs to debug a test. My favorite is (2). Are there any thoughts from other devs/authors of these tests? Best Matthias [1] http://dash.orfeo-toolbox.org/testDetails.php?test=26809858&build=162985 -- Help getting QGIS to the next level of quality before November 15! http://blog.vitu.ch/10102014-1046/crowdfunding-initiative-automated-testing _______________________________________________ Qgis-developer mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/qgis-developer
