This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new 6314610 IMPALA-9355:
TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory
limit
6314610 is described below
commit 63146103a76592fe7ee583f83bc1d5e0385353b7
Author: Qifan Chen <[email protected]>
AuthorDate: Sun Nov 29 15:48:51 2020 -0500
IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't
hit the memory limit
This patch reduces the memory limit for the following query in
test_exchange_mem_usage_scaling test from 170MB to 164MB
to reduce the chance of not detecting a memory allocation
failure.
set mem_limit=<limit_in_mb>
set num_scanner_threads=1;
select *
from tpch_parquet.lineitem l1
join tpch_parquet.lineitem l2 on l1.l_orderkey = l2.l_orderkey and
l1.l_partkey = l2.l_partkey and l1.l_suppkey = l2.l_suppkey
and l1.l_linenumber = l2.l_linenumber
order by l1.l_orderkey desc, l1.l_partkey, l1.l_suppkey,
l1.l_linenumber limit 5;
In a test with 500 executions of the above query with the memory
limit set to 164MB, there were 500 memory allocation failures in
total (one in each execution), and a total of 266 of them from
Exchange Node #4.
Testing:
Ran the query in question individually;
Ran TestExchangeMemUsage.test_exchange_mem_usage_scaling test;
Ran core tests.
Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4
Reviewed-on: http://gerrit.cloudera.org:8080/16791
Reviewed-by: Tim Armstrong <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
.../functional-query/queries/QueryTest/exchange-mem-scaling.test | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test
b/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test
index 4affa6e..b955bc9 100644
---
a/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test
+++
b/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test
@@ -4,7 +4,7 @@
# it hits the memory limit in the exchange node when allocating receiver-side
# buffers. It's also possible but less likely that this will hit a memory limit
# in the scan nodes.
-set mem_limit=170m;
+set mem_limit=164m;
set num_scanner_threads=1;
select *
from tpch_parquet.lineitem l1
