As you wrote, in the first iteration of the functionality I would go for an easy solution and I would detect just all modifications to tests/auto/* On the other hand I do not want to have the definition of stress testing in Coin code. Every one should be able to run it therefore it should be a make target. So everyone could just type: make stress-test in a test folder and after 10 min get the result.
Cheers, Jędrek ________________________________________ From: Edward Welbourne Sent: Friday, November 4, 2016 9:36 AM To: Jedrzej Nowacki Cc: [email protected] Subject: Re: [Development] Removal of some of the blacklisted (non-working) autotests? Jędrzej Nowacki contributed: > Even more, we need something that would not allow flaky, badly written > test to be merged to Qt in the first place, otherwise we would have > that discussion again in next 12 months. How practical would it be to coax CI/Gerrit into recognising when a commit adds new tests, so as to include stress-testing of the new test as part of the CI testing ? A half-way-decent start would be to notice any tests/auto/* files modified by the commit and have the testing make check in the affected directories several times, instead of just once. For bonus marks, when the test branch has several commits in it, only fail the commits that do touch those test files, if those tests show a mix of passes and fails. Eddy. _______________________________________________ Development mailing list [email protected] http://lists.qt-project.org/mailman/listinfo/development
