[
https://issues.apache.org/jira/browse/IMPALA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-15513.
-------------------------------------
Fix Version/s: Impala 5.1.0
Resolution: Fixed
> TestHBO.test_except_cardinality seems flaky
> -------------------------------------------
>
> Key: IMPALA-15513
> URL: https://issues.apache.org/jira/browse/IMPALA-15513
> Project: IMPALA
> Issue Type: Bug
> Reporter: Arnab Karmakar
> Assignee: Quanlong Huang
> Priority: Major
> Fix For: Impala 5.1.0
>
> Attachments: expected_profile.txt, failed_profile.txt
>
>
> The TestHBO.test_except_cardinality test is failing with the error
> “AssertionError: EXPLAIN output mismatch for QueryTest/hbo-except”.
> Stack Trace:
> {code:java}
> query_test/test_hbo.py:441: in test_except_cardinality
> self._run_hbo_explains('QueryTest/hbo-except')
> self = <test_hbo.TestHBO object at 0x7f5c41371350>
> query_test/test_hbo.py:71: in _run_hbo_explains
> assert actual_cardinality_lines == expected_cardinality_lines, (
> E AssertionError: EXPLAIN output mismatch for QueryTest/hbo-except.
> E Expected:
> E # Calcite uses UNION+AGG instead of ANTI JOIN for EXCEPT
> E PLAN-ROOT SINK
> E |
> E 06:EXCHANGE [UNPARTITIONED]
> E |
> E 05:AGGREGATE [FINALIZE]
> E | output: count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1,
> 1, NULL))
> E | group by: ID
> E | having: count(if($f1 = 0, 1, NULL)) > 0, count(if($f1 = 1, 1, NULL))
> = 0
> E | row-size=20B cardinality=7.29K (from HBO)
> E |
> E 04:EXCHANGE [HASH(ID)]
> E |
> E 03:AGGREGATE [STREAMING]
> E | output: count(if($f1 = 0, 1, NULL)), count(if($f1 = 1, 1, NULL))
> E | group by: ID
> E | row-size=20B cardinality=7.30K (from HBO)
> E |
> E 00:UNION
> E | row-size=5B cardinality=7.31K (from HBO)
> E |
> E |--02:SCAN HDFS [functional.alltypestiny]
> E | HDFS partitions=4/4 files=4 size=460B
> E | predicates: functional.alltypestiny.string_col != 'test_except'
> E | row-size=17B cardinality=8 (from HBO)
> E |
> E 01:SCAN HDFS [functional.alltypes]
> E HDFS partitions=24/24 files=24 size=478.45KB
> E predicates: functional.alltypes.string_col != 'test_except'
> E row-size=17B cardinality=7.30K (from HBO)
> E
> E
> E Actual:
> E PLAN-ROOT SINK
> E |
> E 06:EXCHANGE [UNPARTITIONED]
> E |
> E 05:AGGREGATE [FINALIZE]
> E | output: count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1,
> 1, NULL))
> E | group by: ID
> E | having: count(if($f1 = 0, 1, NULL)) > 0, count(if($f1 = 1, 1, NULL))
> = 0
> E | row-size=20B cardinality=7.29K (from HBO)
> E |
> E 04:EXCHANGE [HASH(ID)]
> E |
> E 03:AGGREGATE [STREAMING]
> E | output: count(if($f1 = 0, 1, NULL)), count(if($f1 = 1, 1, NULL))
> E | group by: ID
> E | row-size=20B cardinality=7.31K (from HBO)
> E |
> E 00:UNION
> E | row-size=5B cardinality=7.31K (from HBO)
> E |
> E |--02:SCAN HDFS [functional.alltypestiny]
> E | HDFS partitions=4/4 files=4 size=460B
> E | predicates: functional.alltypestiny.string_col != 'test_except'
> E | row-size=17B cardinality=8 (from HBO)
> E |
> E 01:SCAN HDFS [functional.alltypes]
> E HDFS partitions=24/24 files=24 size=478.45KB
> E predicates: functional.alltypes.string_col != 'test_except'
> E row-size=17B cardinality=7.30K (from HBO)
> E assert ['| row-size...K (from HBO)'] == ['| row-size...K (from HBO)']
> E At index 1 diff: '| row-size=20B cardinality=7.31K (from HBO)' != '|
> row-size=20B cardinality=7.30K (from HBO)'
> E Full diff:
> E [
> E '| row-size=20B cardinality=7.29K (from HBO)',
> E - '| row-size=20B cardinality=7.30K (from HBO)',
> E ? ^
> E + '| row-size=20B cardinality=7.31K (from HBO)',
> E ? ^
> E '| row-size=5B cardinality=7.31K (from HBO)',
> E '| row-size=17B cardinality=8 (from HBO)',
> E ' row-size=17B cardinality=7.30K (from HBO)',
> E ]
> actual_cardinality_lines = ['| row-size=20B cardinality=7.29K (from
> HBO)', '| row-size=20B cardinality=7.31K (from HBO)', '| row-size=5B
> cardinality=7.31K (from HBO)', '| row-size=17B cardinality=8 (from HBO)',
> ' row-size=17B cardinality=7.30K (from HBO)']
> actual_plan_lines = ['PLAN-ROOT SINK', '|', '06:EXCHANGE
> [UNPARTITIONED]', '|', '05:AGGREGATE [FINALIZE]', '| output:
> count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1, 1, NULL))', ...]
> expected_cardinality_lines = ['| row-size=20B cardinality=7.29K
> (from HBO)', '| row-size=20B cardinality=7.30K (from HBO)', '| row-size=5B
> cardinality=7.31K (from HBO)', '| row-size=17B cardinality=8 (from HBO)',
> ' row-size=17B cardinality=7.30K (from HBO)']
> expected_plan_lines = ['# Calcite uses UNION+AGG instead of ANTI JOIN
> for EXCEPT', 'PLAN-ROOT SINK', '|', '06:EXCHANGE [UNPARTITIONED]', '|',
> '05:AGGREGATE [FINALIZE]', ...]
> plan_section_name = 'CALCITE_PLANNER_PLAN'
> query = "EXPLAIN select id from functional.alltypes where
> string_col != 'test_except'\n except\n select id from
> functional.alltypestiny where string_col != 'test_except'"
> result = <tests.common.impala_connection.ImpylaHS2ResultSet
> object at 0x7f5c414be7d0>
> section = defaultdict(<class 'str'>, {'QUERY': "# EXPLAIN the same
> query that has been executed\nEXPLAIN select id from function...45KB\n
> predicates: functional.alltypes.string_col != 'test_except'\n row-size=17B
> cardinality=7.30K (from HBO)\n"})
> self = <test_hbo.TestHBO object at 0x7f5c41371350>
> test_cases = [defaultdict(<class 'str'>, {'QUERY': "# EXPLAIN the
> same query that has been executed\nEXPLAIN select id from functio...B\n
> predicates: functional.alltypes_0.string_col != 'test_except'\n
> row-size=17B cardinality=7.30K (from HBO)\n"})]
> test_file = 'QueryTest/hbo-except'{code}
> stderr:
> {code:java}
> --------------------------------- Captured Err
> ---------------------------------
> -- 2026-10-06 19:41:17,897 INFO MainThread: hs2:
> set
> client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:17,897 INFO MainThread: hs2-http:
> set
> client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:17,899 INFO MainThread: hs2:
> set
> client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:17,899 INFO MainThread: hs2: set_configuration:
> set use_hbo_stats=True;
> set store_hbo_stats=True;
> -- 2026-10-06 19:41:17,899 INFO MainThread: hs2: executing against Impala
> at localhost:21050. session: 5d4a5724266456e8:12e78bdab0d8fc8b main_cursor:
> True user: None
> select id from functional.alltypes where string_col != 'test_except'
> except
> select id from functional.alltypestiny where string_col !=
> 'test_except'
> ;
> -- 2026-10-06 19:41:17,927 INFO MainThread:
> af483a6aa5c5b599:4276713500000000: query started
> -- 2026-10-06 19:41:17,935 INFO MainThread:
> af483a6aa5c5b599:4276713500000000: getting log for operation
> -- 2026-10-06 19:41:17,935 INFO MainThread:
> af483a6aa5c5b599:4276713500000000: getting runtime profile operation
> -- 2026-10-06 19:41:17,946 INFO MainThread:
> af483a6aa5c5b599:4276713500000000: closing query for operation
> -- 2026-10-06 19:41:18,947 INFO MainThread: Loading query test file:
> /data/jenkins/workspace/impala-asf-master-core-calcite/repos/Impala/testdata/workloads/functional-query/queries/QueryTest/hbo-except.test
> -- 2026-10-06 19:41:18,948 INFO MainThread: hs2:
> set
> client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:18,948 INFO MainThread: hs2: set_configuration:
> set use_hbo_stats=True;
> -- 2026-10-06 19:41:18,948 INFO MainThread: hs2: executing against Impala
> at localhost:21050. session: 5d4a5724266456e8:12e78bdab0d8fc8b main_cursor:
> True user: None
> EXPLAIN select id from functional.alltypes where string_col != 'test_except'
> except
> select id from functional.alltypestiny where string_col != 'test_except';
> -- 2026-10-06 19:41:18,958 INFO MainThread:
> 5243e19b35c5c195:dcd6f24900000000: query started
> -- 2026-10-06 19:41:18,958 INFO MainThread:
> 5243e19b35c5c195:dcd6f24900000000: getting log for operation
> -- 2026-10-06 19:41:18,958 INFO MainThread:
> 5243e19b35c5c195:dcd6f24900000000: getting runtime profile operation
> -- 2026-10-06 19:41:18,959 INFO MainThread:
> 5243e19b35c5c195:dcd6f24900000000: closing query for operation{code}
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
