Public bug reported:

Some time back (5.6) the packaging was changed so failing tests during
the build did not cause the build to fail. It's not clear why, as the
test suite should at least be expected to pass on platforms that are
supported by upstream.

Failing tests might not be noticed (there are occasionally some differences 
between build test runs and dep8 test runs). Failing builds can take longer 
because they're only spotted during dep8 runs.
Passing builds pointlessly spend time on the test runs, but ignore the result.

There are multiple elements to the change:
* Test failures will cause build failures on i386 and amd64 (for now. We will 
expand this list as we verify green test runs on other platforms.)
* We run the mtr suite with --force, meaning it doesn't abort on the first 
failure. This will help us get a complete list of which tests fail on which 
* We add a simple system for skipping spurious tests until they can be fixed or 
removed upstream (dep8 already has this)
* The test datadir_permissions has been consistently failing at build-time 
since 5.7.13, so we add a fix for it.

[Test case]
* Build package (the test datadir_permission will fail, but the failure is 

[Regression potential]
* Spurious tests can cause build failures and not just dep8 failures (though 
this is technically a good thing, since they should be reported upstream and 
fixed.), so we might see more random build failures.
* --force makes builds failing due to tests take longer

** Affects: mysql-5.7 (Ubuntu)
     Importance: Undecided
         Status: New

You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.

  Test failures do not cause build failures

To manage notifications about this bug go to:

ubuntu-bugs mailing list

Reply via email to