Quanlong Huang created IMPALA-13423:
---------------------------------------
Summary: TestQueries.test_except failed by Memory limit exceeded
Key: IMPALA-13423
URL: https://issues.apache.org/jira/browse/IMPALA-13423
Project: IMPALA
Issue Type: Bug
Reporter: Quanlong Huang
Assignee: Quanlong Huang
TestQueries.test_except failed in a exhaustive build due to Memory limit
exceeded
{code:java}
query_test/test_queries.py:146: in test_except
self.run_test_case('QueryTest/except', vector)
common/impala_test_suite.py:789: in run_test_case
user=test_section.get('USER', '').strip() or None)
common/impala_test_suite.py:708: in __exec_in_impala
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:1079: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:378: in execute
self.__cursor.execute(sql_stmt, configuration=self.__query_options)
/data/jenkins/workspace/impala-cdwh-2024.0.18.3-exhaustive/repos/Impala/infra/python/env-gcc10.4.0/lib/python2.7/site-packages/impala/hiveserver2.py:343:
in execute
self._wait_to_finish() # make execute synchronous
/data/jenkins/workspace/impala-cdwh-2024.0.18.3-exhaustive/repos/Impala/infra/python/env-gcc10.4.0/lib/python2.7/site-packages/impala/hiveserver2.py:428:
in _wait_to_finish
raise OperationalError(resp.errorMessage)
E OperationalError: Failed to get minimum memory reservation of 284.18 MB on
daemon impala-ec2-centos79-m6i-4xlarge-xldisk-17ac.vpc.cloudera.com:27000 for
query bd468801592e634f:1550644600000000 due to following error: Memory limit
exceeded: Could not allocate memory while trying to increase reservation.
E Query(bd468801592e634f:1550644600000000) could not allocate 284.18 MB
without exceeding limit.
E Error occurred on backend
impala-ec2-centos79-m6i-4xlarge-xldisk-17ac.vpc.cloudera.com:27000
E Memory left in process limit: 1.30 GB
E Query(bd468801592e634f:1550644600000000): Reservation=0
ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
E Memory is likely oversubscribed. Reducing query concurrency or configuring
admission control may help avoid this error.{code}
The queries:
{code:java}
-- Executing select id, bool_col, tinyint_col, smallint_col, int_col,
bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col,
year, month from alltypestiny where year=2009 and month in (1,2,3)
except
(select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col,
float_col, double_col, date_string_col, string_col, timestamp_col, year, month
from alltypestiny where year=2009 and month=1
except
(select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col,
float_col, double_col, date_string_col, string_col, timestamp_col, year, month
from alltypestiny where year=2009 and month=2
except
(select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col,
float_col, double_col, date_string_col, string_col, timestamp_col, year, month
from alltypestiny where year=2009 and month=2
except
select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col,
float_col, double_col, date_string_col, string_col, timestamp_col, year, month
from alltypestiny where year=2009 and month=3))){code}
In another exhaustive build, 4 runs of this test failed:
{code:java}
query_test.test_queries.TestQueries.test_except[protocol: hs2 | exec_option:
{'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'debug_action':
'BEFORE_CODEGEN_IN_ASYNC_CODEGEN_THREAD:JITTER@1000|AFTER_STARTING_ASYNC_CODEGEN_IN_FRAGMENT_THREAD:JITTER@1000',
'ASYNC_CODEGEN': 1, 'exec_single_node_rows_threshold': 0} | table_format:
parquet/none]
query_test.test_queries.TestQueries.test_except[protocol: beeswax |
exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'test_replan': 1, 'exec_single_node_rows_threshold':
'100', 'batch_size': 0, 'num_nodes': 0} | table_format: text/def/block]
query_test.test_queries.TestQueries.test_except[protocol: hs2 | exec_option:
{'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
'abort_on_error': 1, 'test_replan': 1, 'exec_single_node_rows_threshold':
'100', 'batch_size': 0, 'num_nodes': 0} | table_format: parquet/none]
query_test.test_queries.TestQueries.test_except[protocol: hs2-http |
exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]{code}
Here are the failed queries. Test1:
{code}
-- 2024-10-03 12:12:36,258 INFO MainThread: Closing active operation
-- Executing select id, bool_col, tinyint_col, smallint_col, int_col,
bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col,
year, month from alltypestiny where year=2009 and month=1
union distinct
select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col,
double_col, date_string_col, string_col, timestamp_col, year, month from
alltypestiny where year=2009 and month=1
except
select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col,
double_col, date_string_col, string_col, timestamp_col, year, month from
alltypestiny where year=2009 and month=2{code}
Test2:
{code}
select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col,
double_col, date_string_col, string_col, timestamp_col, year, month from
alltypestiny;
-- 2024-10-03 12:12:39,088 INFO MainThread: Started query
9a451a9233b46f49:95dd375800000000{code}
Test3:
{code}
-- 2024-10-03 12:12:39,237 INFO MainThread: Loading query test file:
/data/jenkins/workspace/impala-cdwh-2024.0.18.3-exhaustive-release/repos/Impala/testdata/workloads/functional-query/queries/QueryTest/except.test
-- Executing select id, bool_col, tinyint_col, smallint_col, int_col,
bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col,
year, month from alltypestiny{code}
Test4:
{code}
-- 2024-10-03 12:12:41,728 INFO MainThread: Loading query test file:
/data/jenkins/workspace/impala-cdwh-2024.0.18.3-exhaustive-release/repos/Impala/testdata/workloads/functional-query/queries/QueryTest/except.test
-- Executing select id, bool_col, tinyint_col, smallint_col, int_col,
bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col,
year, month from alltypestiny{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
