IMPALA-1440: test for insert mem limit This patch adds test coverage for partitioned inserts where the memory limit will be exceeded by the table writer.
Testing: Ran the test with exploration_strategy=exhaustive locally then ran an exhaustive private build. Manually inspected the memory limit report to make sure that it was behaving as expected (writer memory was being correctly tracked, etc). Change-Id: I8583c60d648af9eedc956315df5ac3c3d6608704 Reviewed-on: http://gerrit.cloudera.org:8080/3245 Reviewed-by: Matthew Jacobs <[email protected]> Tested-by: Internal Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/7ac341d4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/7ac341d4 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/7ac341d4 Branch: refs/heads/master Commit: 7ac341d4271d564ae11cb5607c05760728c99d2d Parents: 4edb8bb Author: Tim Armstrong <[email protected]> Authored: Fri May 27 17:05:29 2016 -0700 Committer: Tim Armstrong <[email protected]> Committed: Tue May 31 23:32:12 2016 -0700 ---------------------------------------------------------------------- .../functional-query/queries/QueryTest/insert.test | 14 ++++++++++++++ 1 file changed, 14 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/7ac341d4/testdata/workloads/functional-query/queries/QueryTest/insert.test ---------------------------------------------------------------------- diff --git a/testdata/workloads/functional-query/queries/QueryTest/insert.test b/testdata/workloads/functional-query/queries/QueryTest/insert.test index 157fd9a..d6359ad 100644 --- a/testdata/workloads/functional-query/queries/QueryTest/insert.test +++ b/testdata/workloads/functional-query/queries/QueryTest/insert.test @@ -844,3 +844,17 @@ where year=2009 and month>=1 and month<=4 ---- TYPES bigint ==== +---- QUERY +# Check that hdfs writers respects mem_limit. +set mem_limit=64m; +insert into table alltypesinsert +partition (year, month) +select at1.id, at1.bool_col, at1.tinyint_col, at1.smallint_col, at1.int_col, at1.bigint_col, + at1.float_col, at1.double_col, at1.date_string_col, at1.string_col, at1.timestamp_col, + at1.year, at2.id as month +from functional.alltypes at1, functional.alltypes at2; +---- SETUP +DROP PARTITIONS alltypesinsert +---- CATCH +Memory limit exceeded +====
