Great to see this Benjamin!

Looking forward to seeing the parallel test runner turn green, I'll help
file tickets under the epic (I see there are a lot of test failures for me).

Once we clear the issues and turn it green, shall we make this the default?
I would be in favor of that.


On Thu, Oct 13, 2016 at 2:28 PM, Benjamin Bannier <> wrote:

> Hi,
> Since most tests in the Mesos, libprocess, and stout test suites can
> be executed in parallel (the exception being some `ROOT` tests with
> global side effects in Mesos), we recently added a parallel test
> runner `support/`. This should allow to
> potentially significantly speed up running of test suites.
> To enable automatic parallel execution of tests for test targets
> executed during `make check`, configure Mesos with the option
> `--enable-parallel-test-execution`. This will configure the test runner
> to run all tests but the `ROOT` tests in parallel; `ROOT` tests will
> be run in a separate, sequential step.
> * * *
> We use the environment variable `TEST_DRIVER` to drive parallel test
> execution. By setting this variable to an empty string you can
> temporarily disable configured parallel execution, e.g.,
>     % make check TEST_DRIVER=
> By setting this environment variable you have control over the test
> runner itself and its arguments, even without enabling parallel test
> during `./configure` time. Be aware that many `ROOT` tests cannot be
> run in parallel.
> The current settings oversubscribe the machine by running `#cores*1.5`
> parallel jobs. This was driven by the observation that currently our
> tests by and large do not make extended use of even a single core.
> The number of parallel jobs can by controlled with the `-j` flag of
> the test runner.
> Since making more use of the machine will likely increase machine load
> during test execution, running tests in parallel might expose test
> flakiness. Tests might also fail to run in parallel if testcases e.g.,
> write data to hardcoded locations or use hardcoded ports. Please file
> JIRA tickets for such tests if they do not yet exist.
> There is still some work needed to improve reporting from parallel
> tests. We currently use a very silent mode if tests are running
> without failures, and just report the logs of failed jobs in case of
> failure. MESOS-6387 sketches out possible future improvements in this
> area.
> Happy testing,
> Benjamin with help from Kevin & Till

Reply via email to