> What has to be maintained about testcases ? > The only reason to change a testcase would be a change in the underlying > classes interfaces, which is one of the causes for instability, so having > testcases that break might be an aim to keep APIs stable.
I can think of several reason why the testcase need changes and the maintainance is not 0 (usually there are even more cases in practice): - The dependencies of QGIS do change (both in API and in behaviour). Most tests use Qt classes. The Qt API _and_ behaviour can change. Anybody remembers the switch from Qt3 to Qt4? It was a huge effort to port all the classes (the bigger the testsuite, the more changes). - The API of QGIS might indeed change. E.g. say Martin implements a new redesigned symbology engine -> he needs to adapt all the symbology related test (the more the better?) - The 'no change without unit test' policy means also test for internal classes (not in the public API) Marco Am Donnerstag, 7. Juli 2011, 17.53:53 schrieb Sandro Santilli: > On Thu, Jul 07, 2011 at 05:23:28PM +0200, Marco Hugentobler wrote: > > Being in favor of unit tests, we cannot hope to solve with them all the > > stability problems, as some mails here suggest. It is just one small > > piece. > > I wouldn't call it small. But it surely won't easily test each and every > possible combination of interactions. Not deterministically at least > (there are some testsuites aimed at throwing random inputs to tools, to > catch a wider set of bugs w/out tight control over the premises, postgis > garden test is one, we've used zzuf for gnash). > > > In fact, thinking about bugfixes in the past, e.g. for the 1.7 release, > > most of them occured as a complex combination of several factors and user > > interaction, very hard to detect with unit tests. > > This may be a limit on the currently available tools for testing GUIs, > but anyway nobody said it'd be an easy task. > > > Another point that needs to be considered is that the unit testing code > > needs to be maintained. Adding a test for every little change, even > > bugfix, would create a huge amount of testing code and sample datasets. > > The bigger the testsuite, the safer the codebase ! > > What has to be maintained about testcases ? > The only reason to change a testcase would be a change in the underlying > classes interfaces, which is one of the causes for instability, so having > testcases that break might be an aim to keep APIs stable. > > > What about alternatively creating a unit test base that is limited in > > size but well maintained, e.g. a good coverage of the core classes? > > I don't see any testing effort being exclusive. The more testing > means, the better. > > --strk; > > () Free GIS & Flash consultant/developer > /\ http://strk.keybit.net/services.html -- Dr. Marco Hugentobler Sourcepole - Linux & Open Source Solutions Churerstr. 22, CH-8808 Pfäffikon SZ, Switzerland [email protected] http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee _______________________________________________ Qgis-developer mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/qgis-developer
