Repository: impala
Updated Branches:
  refs/heads/master 242e822ae -> 5bdce5a86


Move some test_spilling debug actions to exhaustive

Phil Z pointed out that these tests take a long time to run.
Based on experience, we only need the always/never variants
to catch the vast majority of spilling-related bugs, so
relegating the other variants to exhaustive should not cause any major
problems.

People making spilling-related changes to the spilling code should run
this test with the exhaustive dimension to flush out any problems in
advance, e.g.:

  impala-py.test tests/query_test/test_spilling.py -n2 --verbose \
    --workload_exploration_strategy=functional-query:exhaustive

Change-Id: I9ea9f6c299480f8dfc943635342e4473499cc8ad
Reviewed-on: http://gerrit.cloudera.org:8080/9976
Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com>
Reviewed-by: Philip Zeyliger <phi...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/c68da03e
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/c68da03e
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/c68da03e

Branch: refs/heads/master
Commit: c68da03e7570b9ce9460d87532a0f020da912af4
Parents: 242e822
Author: Tim Armstrong <tarmstr...@cloudera.com>
Authored: Tue Apr 10 12:01:55 2018 -0700
Committer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Committed: Thu Apr 12 01:25:36 2018 +0000

----------------------------------------------------------------------
 tests/query_test/test_spilling.py | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/impala/blob/c68da03e/tests/query_test/test_spilling.py
----------------------------------------------------------------------
diff --git a/tests/query_test/test_spilling.py 
b/tests/query_test/test_spilling.py
index 0b36429..8550f96 100644
--- a/tests/query_test/test_spilling.py
+++ b/tests/query_test/test_spilling.py
@@ -22,11 +22,17 @@ from tests.common.test_dimensions import 
(create_exec_option_dimension_from_dict
     create_parquet_dimension)
 
 # Test with denial of reservations at varying frequency.
-DEBUG_ACTION_DIMS = [None,
+# Always test with the minimal amount of spilling and running with the 
absolute minimum
+# memory requirement.
+CORE_DEBUG_ACTION_DIMS = [None,
+  '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0']
+
+# Test with different frequency of denial on exhaustive to try and exercise 
more
+# interesting code paths.
+EXHAUSTIVE_DEBUG_ACTION_DIMS = [
   '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.1',
   '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5',
-  '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.9',
-  '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0']
+  '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.9']
 
 @pytest.mark.xfail(pytest.config.option.testing_remote_cluster,
                    reason='Queries may not spill on larger clusters')
@@ -40,10 +46,13 @@ class TestSpillingDebugActionDimensions(ImpalaTestSuite):
     super(TestSpillingDebugActionDimensions, cls).add_test_dimensions()
     cls.ImpalaTestMatrix.clear_constraints()
     cls.ImpalaTestMatrix.add_dimension(create_parquet_dimension('tpch'))
+    debug_action_dims = CORE_DEBUG_ACTION_DIMS
+    if cls.exploration_strategy() == 'exhaustive':
+      debug_action_dims = CORE_DEBUG_ACTION_DIMS + EXHAUSTIVE_DEBUG_ACTION_DIMS
     # Tests are calibrated so that they can execute and spill with this page 
size.
     cls.ImpalaTestMatrix.add_dimension(
         
create_exec_option_dimension_from_dict({'default_spillable_buffer_size' : 
['256k'],
-          'debug_action' : DEBUG_ACTION_DIMS}))
+          'debug_action' : debug_action_dims}))
 
   def test_spilling(self, vector):
     self.run_test_case('QueryTest/spilling', vector)

Reply via email to