[
https://issues.apache.org/jira/browse/IMPALA-15003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18081718#comment-18081718
]
Quanlong Huang commented on IMPALA-15003:
-----------------------------------------
The exec summary on Ozone erasure-coding builds:
{noformat}
Operator #Hosts #Inst Avg Time Max Time #Rows Est.
#Rows Peak Mem Est. Peak Mem Detail
-------------------------------------------------------------------------------------------------------------------------------------
F05:ROOT 1 1 17.742us 17.742us
0 0
09:AGGREGATE 1 1 167.687us 167.687us 1
1 16.00 KB 16.00 KB FINALIZE
08:EXCHANGE 1 1 14.608us 14.608us 1
1 16.00 KB 16.00 KB UNPARTITIONED
F00:EXCHANGE SENDER 1 1 66.645us 66.645us
0 48.00 KB
05:AGGREGATE 1 1 0.000ns 0.000ns 1
1 51.00 KB 16.00 KB
04:HASH JOIN 1 1 15.675ms 15.675ms 0
4 9.07 MB 4.75 MB INNER JOIN, BROADCAST
|--07:EXCHANGE 1 1 507.527us 507.527us 125.00K
125.00K 1.22 MB 988.56 KB BROADCAST
| F04:EXCHANGE SENDER 1 1 4.078ms 4.078ms
18.00 KB 48.00 KB
| 06:EXCHANGE 1 1 583.811us 583.811us 125.00K
125.00K 40.00 KB 988.56 KB UNPARTITIONED
| F03:EXCHANGE SENDER 1 1 4.607ms 4.607ms
18.00 KB 48.00 KB
| 01:UNION 1 1 85.025us 85.025us 125.00K
125.00K 0 0
| |--03:SCAN OZONE 1 1 53.531us 53.531us 0
6.00M 0 176.00 MB tpch.lineitem, CANCELLED
| 02:SCAN OZONE 1 1 122.560ms 122.560ms 125.95K
6.00M 24.04 MB 176.00 MB tpch.lineitem, CANCELLED
00:SCAN OZONE 1 1 3s322ms 3s322ms 6
4 24.05 MB 176.00 MB tpch.lineitem a{noformat}
It has fewer rows comparing to the exec summary on HDFS builds:
{noformat}
Operator #Hosts #Inst Avg Time Max Time #Rows Est.
#Rows Peak Mem Est. Peak Mem Detail
--------------------------------------------------------------------------------------------------------------------------------------
F06:ROOT 1 1 3.155us 3.155us
0 0
10:AGGREGATE 1 1 58.261us 58.261us 1
1 16.00 KB 16.00 KB FINALIZE
09:EXCHANGE 1 1 10.573us 10.573us 3
3 32.00 KB 16.00 KB UNPARTITIONED
F05:EXCHANGE SENDER 3 3 26.395us 29.837us
31.00 B 48.00 KB
05:AGGREGATE 3 3 0.000ns 0.000ns 3
3 24.00 KB 16.00 KB
04:HASH JOIN 3 3 451.339us 532.024us 0
4 4.06 MB 2.88 MB INNER JOIN, PARTITIONED
|--08:EXCHANGE 3 3 89.086us 123.057us 125.00K
125.00K 752.00 KB 337.52 KB HASH(-1 * l_orderkey)
| F04:EXCHANGE SENDER 1 1 2.664ms 2.664ms
215.08 KB 144.00 KB
| 06:EXCHANGE 1 1 217.258us 217.258us 125.00K
125.00K 8.62 MB 361.52 KB UNPARTITIONED
| F03:EXCHANGE SENDER 3 3 2.972ms 3.549ms
54.05 KB 48.00 KB
| 01:UNION 3 3 62.785us 79.635us 375.00K
125.00K 0 0
| |--03:SCAN HDFS 3 3 35.180us 41.639us 0
6.00M 0 264.00 MB tpch.lineitem, CANCELLED
| 02:SCAN HDFS 3 3 7.651ms 11.532ms 377.86K
6.00M 48.67 MB 264.00 MB tpch.lineitem, CANCELLED
07:EXCHANGE 3 3 7.931us 11.162us 6
4 16.00 KB 16.00 KB HASH(a.l_orderkey)
F00:EXCHANGE SENDER 3 3 46.500us 55.732us
132.00 B 144.00 KB
00:SCAN HDFS 3 3 258.935ms 278.645ms 6
4 48.09 MB 264.00 MB tpch.lineitem a{noformat}
We just want to verify the details of specific operators. So the test shouldn't
hard code the row index.
> test_cancelled_nodes_in_exec_summary fails on list index out of range
> ---------------------------------------------------------------------
>
> Key: IMPALA-15003
> URL: https://issues.apache.org/jira/browse/IMPALA-15003
> Project: IMPALA
> Issue Type: Bug
> Components: Test
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
>
> test_cancelled_nodes_in_exec_summary fails on Ozone erasure-coding builds:
> *Error Message*
> IndexError: list index out of range
> *Stacktrace*
> {code:python}
> query_test/test_observability.py:223: in test_cancelled_nodes_in_exec_summary
> assert result.exec_summary[15]['operator'] == '00:' + scan_op,
> result.runtime_profile
> E IndexError: list index out of range
> query = '\n with l as (select * from tpch.lineitem UNION
> ALL select * from tpch.lineitem)\n select STRAIGHT_JOIN...e l_orderkey
> = 1) a\n join\n (select * from l LIMIT 125000) b\n on
> a.l_orderkey = -b.l_orderkey'
> result = <tests.common.impala_connection.ImpylaHS2ResultSet
> object at 0x7f1b465302d0>
> scan_op = 'SCAN OZONE'
> self = <test_observability.TestObservability object at
> 0x7f1b4707f410>{code}
> The query is
> {code:sql}
> with l as (select * from tpch.lineitem UNION ALL select * from
> tpch.lineitem)
> select STRAIGHT_JOIN count(*) from
> (select * from tpch.lineitem a where l_orderkey = 1) a
> join
> (select * from l LIMIT 125000) b
> on a.l_orderkey = -b.l_orderkey;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
