On Wed, 2014-12-17 at 11:57 +0100, Stefan Seifert wrote: > > >> Sure, do we know what broke it? > > > >I don't know yet. In theory whoever made those breaking changes should > >have run the tests and seen the failures... > > the problem is that these integration tests run on some environments, and on > some not, on some they succeed sometime and sometimes not. this makes it so > difficult to spot the real problem, maybe its just a racing condition or > similar. thus my proposal to deactivate those tests in Jenkins for now and > create a ticket for each one. we pick one to start with and ask around who > has an environment where those tests fail every time or most times and try to > find the issue. if it then runs on those environment we have a good chance > that it will run on Jenkins as well. >
Is those tests were failing for such a long time, doesn't that mean that no one has the time to look into them when the build fails? If that's the case, no one will look into them when the build succeeds :-) We could make rule that a release of a module is conditioned of its own tests + the integration tests related to it all pass on Jenkins. Might hurt a bit initially but we should be out of this situation quickly, at the rate we're doing releases. Robert
