IMPALA-4438: Serialize test_failpoints.py to reduce memory pressure On EC2 c3.4xlarge instances, with 8cores and 30GB RAM, this test could trigger the Linux OOM killer by running tests in parallel. This patch switches to serial execution, which makes the test take four minutes, rather than one to two minutes.
Change-Id: Iea4a588e1228d38f90387a077cbe530257636b7d Reviewed-on: http://gerrit.cloudera.org:8080/4999 Reviewed-by: Jim Apple <[email protected]> Tested-by: Internal Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/4af2ea4f Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/4af2ea4f Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/4af2ea4f Branch: refs/heads/hadoop-next Commit: 4af2ea4fe4e90588b13c4b551d0138123a743650 Parents: ae24bf2 Author: Jim Apple <[email protected]> Authored: Mon Nov 7 20:21:27 2016 -0800 Committer: Internal Jenkins <[email protected]> Committed: Wed Nov 9 01:06:33 2016 +0000 ---------------------------------------------------------------------- tests/failure/test_failpoints.py | 4 ++++ 1 file changed, 4 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/4af2ea4f/tests/failure/test_failpoints.py ---------------------------------------------------------------------- diff --git a/tests/failure/test_failpoints.py b/tests/failure/test_failpoints.py index 512d6f7..49a4699 100644 --- a/tests/failure/test_failpoints.py +++ b/tests/failure/test_failpoints.py @@ -18,6 +18,7 @@ # Injects failures at specific locations in each of the plan nodes. Currently supports # two types of failures - cancellation of the query and a failure test hook. # +import pytest import os import re from collections import defaultdict @@ -114,6 +115,9 @@ class TestFailpoints(ImpalaTestSuite): lambda v: (v.get_value('location') != 'PREPARE_SCANNER' or v.get_value('target_node')[0] == 'SCAN HDFS')) + # Run serially because this can create enough memory pressure to invoke the Linux OOM + # killer on machines with 30GB RAM. This makes the test run in 4 minutes instead of 1-2. + @pytest.mark.execute_serially def test_failpoints(self, vector): query = QUERY node_type, node_ids = vector.get_value('target_node')
