Re: [QGIS-Developer] Flaky tests and Travis

2019-03-12 Thread Alessandro Pasotti
I understand we haven't much of a choice: +1 for disabling. I wonder if there is a way to make it more evident for developers that they need to run disabled tests locally by hand before making a PR. It happened to me once or twice that I made a change that I knew was covered by a test case, let

Re: [QGIS-Developer] Flaky tests and Travis

2019-03-12 Thread Matthias Kuhn
On 3/12/19 12:28 AM, Nyall Dawson wrote: > On Tue, 12 Mar 2019 at 07:59, Denis Rouzaud wrote: >> I am also in favor of disabling them but maybe keep them running and have >> them as expected failure? > That sounds preferable -- but I'm not (personally) sure if it's > possible. You know the CI

Re: [QGIS-Developer] Flaky tests and Travis

2019-03-11 Thread Nyall Dawson
On Tue, 12 Mar 2019 at 07:59, Denis Rouzaud wrote: > > I am also in favor of disabling them but maybe keep them running and have > them as expected failure? That sounds preferable -- but I'm not (personally) sure if it's possible. You know the CI setup better than I do, can you see a way to do

Re: [QGIS-Developer] Flaky tests and Travis

2019-03-11 Thread Denis Rouzaud
I am also in favor of disabling them but maybe keep them running and have them as expected failure? If ones write some code related to the corresponding tests , they might still provide some valuable info. Is this possible? If not I'd still be in favor of disabling them. Cheers Denis On Mon, 11

[QGIS-Developer] Flaky tests and Travis

2019-03-11 Thread Nyall Dawson
Hi all, For a long time now we've been plagued by intermittently failing tests on Travis, which are making the whole QGIS development experience quite painful. I propose that we take an absolute hard line approach from now and disable all tests which are causing false positive failures. I've