Udi Meiri created BEAM-3364:
-------------------------------
Summary: Use multiprocess nose plugin to improve unit test run time
Key: BEAM-3364
URL: https://issues.apache.org/jira/browse/BEAM-3364
Project: Beam
Issue Type: Improvement
Components: sdk-py-core, testing
Reporter: Udi Meiri
Assignee: Ahmet Altay
Priority: Minor
The multiprocess plugin is installed by default. You can try it out by running:
nosetests --processes 8 --process-timeout 60
The running time is reduced from ~3m30s to ~30s on my workstation, but with
some tests reporting errors.
Errors can be avoided by disabling multiprocess for certain classes by setting
this global in the test file:
_multiprocess_ = False # Disable nosetests multiprocess plugin.
The problem is that failures are flaky, so repeated runs are required to verify
we're not adding any new flakes by enabling multiprocess. Alternatively, we
could try fixing the tests to make them pass.
Setting up multiprocess runs is done by adding these lines to the bottom of
setup.cfg:
# Run tests in multiple processes for shorter overall running time.
processes=8
# This timeout only applies to multiprocess runs. The default is 10 seconds.
process-timeout=60
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
