Quanlong Huang has uploaded this change for review. (
http://gerrit.cloudera.org:8080/24306
Change subject: IMPALA-14992: Deflake test_cancelled_nodes_in_exec_summary
......................................................................
IMPALA-14992: Deflake test_cancelled_nodes_in_exec_summary
The failed query has 3 scan nodes:
with l as (
select * from tpch.lineitem UNION ALL select * from tpch.lineitem
)
select STRAIGHT_JOIN count(*) from
(select * from tpch.lineitem a LIMIT 1) a
join
(select * from l LIMIT 125000) b
on a.l_orderkey = -b.l_orderkey;
The test expects the two scan nodes under UNION have the CANCELLED
marker and the other scan node on "tpch.lineitem a LIMIT 1" can complete
so won't have the CANCELLED marker. However, "LIMIT 1" is also applied
to the ExchangeNode on top of that scan, and the ExchangeNode is the
leaf of the coordinator fragment:
F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
...
06:EXCHANGE [UNPARTITIONED]
| limit: 1
| in pipelines: 00(GETNEXT)
|
F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
00:SCAN HDFS [tpch.lineitem a, RANDOM]
HDFS partitions=1/1 files=1 size=718.94MB
...
limit: 1
There are 3 scan node instances for this HdfsScanNode. Any of them
returns a row leads the coordinator to mark the query as finished and
cancels all running fragment instances, which might cancel the other 2
scan node instances. Usually they finish quickly since they have limit=1
so won't be cancelled. The test is flaky when the cluster is busy so
those scan node instances run slow.
This patch deflakes the test by not using "LIMIT 1" on the scan to
ensure it can finish.
Testing
- Ran the test 1000 times.
Change-Id: I00d0b81051c864f76060ae7ebcafb635da37ab1e
---
M tests/query_test/test_observability.py
1 file changed, 18 insertions(+), 15 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/24306/1
--
To view, visit http://gerrit.cloudera.org:8080/24306
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I00d0b81051c864f76060ae7ebcafb635da37ab1e
Gerrit-Change-Number: 24306
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <[email protected]>
