IMPALA-7699: Fix spilling test run with hdfs erasure coding turned on A spilling test when run on test build with hdfs erasure coding turned on hits an out of memory error on the hdfs scan node. This happens because the test is tuned for a regular 3 node minicluster without hdfs erasure coding. Fix is to simply increase the memory limit on the test to accommodate this difference yet keep it small enough to achieve desired spilling on the hash join node.
Testing: Ran it on an EC enabled minicluster to make sure it works Change-Id: I207569822ba7388e78936d25e2311fa09c7a1b9a Reviewed-on: http://gerrit.cloudera.org:8080/11740 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/impala/repo Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/77c56a80 Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/77c56a80 Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/77c56a80 Branch: refs/heads/master Commit: 77c56a805abf23db27db493ed12af965e515428d Parents: bad49e7 Author: Bikramjeet Vig <[email protected]> Authored: Fri Oct 19 13:28:20 2018 -0700 Committer: Impala Public Jenkins <[email protected]> Committed: Sat Oct 20 03:32:07 2018 +0000 ---------------------------------------------------------------------- .../queries/QueryTest/spilling-no-debug-action.test | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/impala/blob/77c56a80/testdata/workloads/functional-query/queries/QueryTest/spilling-no-debug-action.test ---------------------------------------------------------------------- diff --git a/testdata/workloads/functional-query/queries/QueryTest/spilling-no-debug-action.test b/testdata/workloads/functional-query/queries/QueryTest/spilling-no-debug-action.test index 539a4fd..7fe1c96 100644 --- a/testdata/workloads/functional-query/queries/QueryTest/spilling-no-debug-action.test +++ b/testdata/workloads/functional-query/queries/QueryTest/spilling-no-debug-action.test @@ -91,7 +91,7 @@ Memory limit exceeded # Hash join that will fail to repartition and therefore fail from out-of-memory because # of a large number of duplicate keys on the build side: IMPALA-4857. It does not need # any help from DEBUG_ACTION. -set mem_limit=150m; +set mem_limit=250m; select straight_join * from supplier join /* +broadcast */ lineitem on s_suppkey = l_linenumber order by l_tax desc
