[email protected] said: > On 04/11/2012 07:49 AM, ext [email protected]<mailto:[email protected]> > wrote: > > ** insignificant tests ** > > We still have quite some tests (around 110 in total) marked as > insignificant. This means they will be ignored by the CI system. Any help > to reduce this number would be great. > > > > I've grepped through all the modules to create a list of these insignificant > tests, and listed them below. > > Please note that a test marked as insignificant in essence provides us with > zero coverage, since all results from those test cases (although run) are > completely ignored. > > Yes, we used to have a (quite large) list of black-listed tests for Qt 4 as > well. However, back then we could mark only a test function as black-listed, > while the rest of the testcase would still provide some test coverage. That's > an important difference. > > IMO, if there are tasks we cannot repair, due to the tests being inheritly > unstable, or our infrastructure not being able to handle it, then we should > remove them. Having test cases which adds no value, but is hard to see unless > you find the "insignificant_test" keyword, is bad since you don't know > exactly what the coverage is and where it is lacking. > > How should we mark that we are working on a specific test case to fix the > instability etc, to ensure we don't do double work? Suggestions? Is a Jira > task with '<module>: insignificant <test name>' good enough? >
Tests marked as insignificant _should_ have had a task raised when they were marked as such - for those, it would make sense for the person who is working on it to assign the task to themselves and/or add a comment. In reality it seems like quite many tests were marked as insignificant with no task raised, it would probably make sense to raise a task when beginning work on it. _______________________________________________ Development mailing list [email protected] http://lists.qt-project.org/mailman/listinfo/development
