Arnab Karmakar created IMPALA-15513:
---------------------------------------
Summary: TestHBO.test_except_cardinality seems flaky
Key: IMPALA-15513
URL: https://issues.apache.org/jira/browse/IMPALA-15513
Project: IMPALA
Issue Type: Bug
Reporter: Arnab Karmakar
Assignee: Quanlong Huang
The TestHBO.test_except_cardinality test is failing with the error
“AssertionError: EXPLAIN output mismatch for QueryTest/hbo-except”.
Stack Trace:
{code:java}
query_test/test_hbo.py:441: in test_except_cardinality
self._run_hbo_explains('QueryTest/hbo-except')
self = <test_hbo.TestHBO object at 0x7f5c41371350>
query_test/test_hbo.py:71: in _run_hbo_explains
assert actual_cardinality_lines == expected_cardinality_lines, (
E AssertionError: EXPLAIN output mismatch for QueryTest/hbo-except.
E Expected:
E # Calcite uses UNION+AGG instead of ANTI JOIN for EXCEPT
E PLAN-ROOT SINK
E |
E 06:EXCHANGE [UNPARTITIONED]
E |
E 05:AGGREGATE [FINALIZE]
E | output: count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1, 1,
NULL))
E | group by: ID
E | having: count(if($f1 = 0, 1, NULL)) > 0, count(if($f1 = 1, 1, NULL)) = 0
E | row-size=20B cardinality=7.29K (from HBO)
E |
E 04:EXCHANGE [HASH(ID)]
E |
E 03:AGGREGATE [STREAMING]
E | output: count(if($f1 = 0, 1, NULL)), count(if($f1 = 1, 1, NULL))
E | group by: ID
E | row-size=20B cardinality=7.30K (from HBO)
E |
E 00:UNION
E | row-size=5B cardinality=7.31K (from HBO)
E |
E |--02:SCAN HDFS [functional.alltypestiny]
E | HDFS partitions=4/4 files=4 size=460B
E | predicates: functional.alltypestiny.string_col != 'test_except'
E | row-size=17B cardinality=8 (from HBO)
E |
E 01:SCAN HDFS [functional.alltypes]
E HDFS partitions=24/24 files=24 size=478.45KB
E predicates: functional.alltypes.string_col != 'test_except'
E row-size=17B cardinality=7.30K (from HBO)
E
E
E Actual:
E PLAN-ROOT SINK
E |
E 06:EXCHANGE [UNPARTITIONED]
E |
E 05:AGGREGATE [FINALIZE]
E | output: count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1, 1,
NULL))
E | group by: ID
E | having: count(if($f1 = 0, 1, NULL)) > 0, count(if($f1 = 1, 1, NULL)) = 0
E | row-size=20B cardinality=7.29K (from HBO)
E |
E 04:EXCHANGE [HASH(ID)]
E |
E 03:AGGREGATE [STREAMING]
E | output: count(if($f1 = 0, 1, NULL)), count(if($f1 = 1, 1, NULL))
E | group by: ID
E | row-size=20B cardinality=7.31K (from HBO)
E |
E 00:UNION
E | row-size=5B cardinality=7.31K (from HBO)
E |
E |--02:SCAN HDFS [functional.alltypestiny]
E | HDFS partitions=4/4 files=4 size=460B
E | predicates: functional.alltypestiny.string_col != 'test_except'
E | row-size=17B cardinality=8 (from HBO)
E |
E 01:SCAN HDFS [functional.alltypes]
E HDFS partitions=24/24 files=24 size=478.45KB
E predicates: functional.alltypes.string_col != 'test_except'
E row-size=17B cardinality=7.30K (from HBO)
E assert ['| row-size...K (from HBO)'] == ['| row-size...K (from HBO)']
E At index 1 diff: '| row-size=20B cardinality=7.31K (from HBO)' != '|
row-size=20B cardinality=7.30K (from HBO)'
E Full diff:
E [
E '| row-size=20B cardinality=7.29K (from HBO)',
E - '| row-size=20B cardinality=7.30K (from HBO)',
E ? ^
E + '| row-size=20B cardinality=7.31K (from HBO)',
E ? ^
E '| row-size=5B cardinality=7.31K (from HBO)',
E '| row-size=17B cardinality=8 (from HBO)',
E ' row-size=17B cardinality=7.30K (from HBO)',
E ]
actual_cardinality_lines = ['| row-size=20B cardinality=7.29K (from
HBO)', '| row-size=20B cardinality=7.31K (from HBO)', '| row-size=5B
cardinality=7.31K (from HBO)', '| row-size=17B cardinality=8 (from HBO)', '
row-size=17B cardinality=7.30K (from HBO)']
actual_plan_lines = ['PLAN-ROOT SINK', '|', '06:EXCHANGE
[UNPARTITIONED]', '|', '05:AGGREGATE [FINALIZE]', '| output:
count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1, 1, NULL))', ...]
expected_cardinality_lines = ['| row-size=20B cardinality=7.29K (from
HBO)', '| row-size=20B cardinality=7.30K (from HBO)', '| row-size=5B
cardinality=7.31K (from HBO)', '| row-size=17B cardinality=8 (from HBO)', '
row-size=17B cardinality=7.30K (from HBO)']
expected_plan_lines = ['# Calcite uses UNION+AGG instead of ANTI JOIN
for EXCEPT', 'PLAN-ROOT SINK', '|', '06:EXCHANGE [UNPARTITIONED]', '|',
'05:AGGREGATE [FINALIZE]', ...]
plan_section_name = 'CALCITE_PLANNER_PLAN'
query = "EXPLAIN select id from functional.alltypes where
string_col != 'test_except'\n except\n select id from functional.alltypestiny
where string_col != 'test_except'"
result = <tests.common.impala_connection.ImpylaHS2ResultSet object
at 0x7f5c414be7d0>
section = defaultdict(<class 'str'>, {'QUERY': "# EXPLAIN the same
query that has been executed\nEXPLAIN select id from function...45KB\n
predicates: functional.alltypes.string_col != 'test_except'\n row-size=17B
cardinality=7.30K (from HBO)\n"})
self = <test_hbo.TestHBO object at 0x7f5c41371350>
test_cases = [defaultdict(<class 'str'>, {'QUERY': "# EXPLAIN the same
query that has been executed\nEXPLAIN select id from functio...B\n
predicates: functional.alltypes_0.string_col != 'test_except'\n row-size=17B
cardinality=7.30K (from HBO)\n"})]
test_file = 'QueryTest/hbo-except'{code}
stderr:
{code:java}
--------------------------------- Captured Err ---------------------------------
-- 2026-10-06 19:41:17,897 INFO MainThread: hs2:
set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
-- 2026-10-06 19:41:17,897 INFO MainThread: hs2-http:
set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
-- 2026-10-06 19:41:17,899 INFO MainThread: hs2:
set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
-- 2026-10-06 19:41:17,899 INFO MainThread: hs2: set_configuration:
set use_hbo_stats=True;
set store_hbo_stats=True;
-- 2026-10-06 19:41:17,899 INFO MainThread: hs2: executing against Impala
at localhost:21050. session: 5d4a5724266456e8:12e78bdab0d8fc8b main_cursor:
True user: None
select id from functional.alltypes where string_col != 'test_except'
except
select id from functional.alltypestiny where string_col != 'test_except'
;
-- 2026-10-06 19:41:17,927 INFO MainThread:
af483a6aa5c5b599:4276713500000000: query started
-- 2026-10-06 19:41:17,935 INFO MainThread:
af483a6aa5c5b599:4276713500000000: getting log for operation
-- 2026-10-06 19:41:17,935 INFO MainThread:
af483a6aa5c5b599:4276713500000000: getting runtime profile operation
-- 2026-10-06 19:41:17,946 INFO MainThread:
af483a6aa5c5b599:4276713500000000: closing query for operation
-- 2026-10-06 19:41:18,947 INFO MainThread: Loading query test file:
/data/jenkins/workspace/impala-asf-master-core-calcite/repos/Impala/testdata/workloads/functional-query/queries/QueryTest/hbo-except.test
-- 2026-10-06 19:41:18,948 INFO MainThread: hs2:
set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
-- 2026-10-06 19:41:18,948 INFO MainThread: hs2: set_configuration:
set use_hbo_stats=True;
-- 2026-10-06 19:41:18,948 INFO MainThread: hs2: executing against Impala
at localhost:21050. session: 5d4a5724266456e8:12e78bdab0d8fc8b main_cursor:
True user: None
EXPLAIN select id from functional.alltypes where string_col != 'test_except'
except
select id from functional.alltypestiny where string_col != 'test_except';
-- 2026-10-06 19:41:18,958 INFO MainThread:
5243e19b35c5c195:dcd6f24900000000: query started
-- 2026-10-06 19:41:18,958 INFO MainThread:
5243e19b35c5c195:dcd6f24900000000: getting log for operation
-- 2026-10-06 19:41:18,958 INFO MainThread:
5243e19b35c5c195:dcd6f24900000000: getting runtime profile operation
-- 2026-10-06 19:41:18,959 INFO MainThread:
5243e19b35c5c195:dcd6f24900000000: closing query for operation{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
