[
https://issues.apache.org/jira/browse/IMPALA-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659883#comment-16659883
]
Tim Armstrong commented on IMPALA-7648:
---------------------------------------
IMPALA-7648 part 1: add expected out-of-memory tests
This adds test coverage for some cases where queries are currently
expected to fail with out-of-memory:
* memory limit exceeded in exchange node
* aggregation with large var-len intermediate values
* top N with large limit
* hash join with many duplicates on right side
* analytic with a large window that needs to be buffered in-memory
Note that it's not always totally deterministic where the query hits
'memory limit exceeded' so we don't include the node ID or name in the
expected error message.
Testing:
* ran exhaustive tests
* looped modified tests locally overnight
Change-Id: Icd1a7eb97837b742a967c260cafb5a7f4f45412e
Reviewed-on: [http://gerrit.cloudera.org:8080/11564]
Reviewed-by: Impala Public Jenkins
<[[email protected]|mailto:[email protected]]>
Tested-by: Impala Public Jenkins
<[[email protected]|mailto:[email protected]]>
> Add tests for all cases where OOM is expected
> ---------------------------------------------
>
> Key: IMPALA-7648
> URL: https://issues.apache.org/jira/browse/IMPALA-7648
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: resource-management
>
> We should add tests for most or all of the subtasks of IMPALA-4834 that
> exercise the expected OOM code path. I'll add some bullet points here to
> track what the coverage is and what's missing.
> * Aggregations with large var-len string expressions
> * Top-N with large window
> * -Kudu scans- : covered by kudu-scan-mem-usage.test
> * -Nested loop join- : covered by single-node-nlj-exhaustive.test
> * Many duplicate keys on build side of hash join
> * -Large number of NULLS on build side of NAAJ- : covered by
> spilling-naaj-no-deny-reservation.test
> * -HDFS table partitioned insert- : covered insert-mem-limit.test
> * Large analytic window can't be spilled
> * Queries processing large strings (may need multiple tests to cover
> different places). large_strings.test has some coverage
> * Parquet files with large pages
> * Exchange uses a lot of memory
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
