[
https://issues.apache.org/jira/browse/IMPALA-7148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678707#comment-16678707
]
Michael Ho commented on IMPALA-7148:
------------------------------------
All the tests have been re-enabled now that IMPALA-4063 has been merged.
However, there seems to be some flakiness within the test itself which led to
intermittent failure. In particular, {{test_profile_fragment_instances}} seems
to make assumption such as there will be 3 instances of each {{HDFS_SCAN_NODE}}
etc. This seems flaky as the profile below shows that we could have only 2
instances of {{HDFS_SCAN_NODE}}.
{noformat}
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem
Est. Peak Mem Detail
----------------------------------------------------------------------------------------------------------------------
05:AGGREGATE 1 0.000ns 0.000ns 1 1 24.00 KB
10.00 MB FINALIZE
04:HASH JOIN 1 9.000ms 9.000ms 0 1 141.06 MB
17.00 MB INNER JOIN, BROADCAST
|--08:EXCHANGE 1 304.006ms 304.006ms 2.00M 2.00M 8.18 MB
7.64 MB UNPARTITIONED
| 07:EXCHANGE 1 263.005ms 263.005ms 2.00M 2.00M 1.09 MB
7.65 MB UNPARTITIONED
| 01:UNION 2 11.000ms 11.000ms 4.00M 2.00M 0
0
| |--03:SCAN HDFS 2 0.000ns 0.000ns 0 6.00M 0
176.00 MB tpch.lineitem
| 02:SCAN HDFS 2 1s361ms 1s411ms 4.00M 6.00M 24.29 MB
176.00 MB tpch.lineitem
06:EXCHANGE 1 0.000ns 0.000ns 1 1 16.00 KB
16.00 KB UNPARTITIONED
00:SCAN HDFS 2 1s601ms 1s616ms 2 1 24.05 MB
176.00 MB tpch.lineitem a
{noformat}
This led to test failure like below:
{noformat}
Stacktrace
query_test/test_observability.py:161: in test_profile_fragment_instances
assert results.runtime_profile.count("HDFS_SCAN_NODE") == 12
E assert 9 == 12
E + where 9 = <built-in method count of str object at
0x69f5a80>('HDFS_SCAN_NODE')
E + where <built-in method count of str object at 0x69f5a80> = 'Query
(id=86445c4beb185e78:bf1b7fc300000000):\n DEBUG MODE WARNING: Query profile
created while running a DEBUG buil... - OptimizationTime: 79.001ms\n
- PeakMemoryUsage: 281.00 KB (287744)\n - PrepareTime:
23.000ms\n'.count
E + where 'Query (id=86445c4beb185e78:bf1b7fc300000000):\n DEBUG MODE
WARNING: Query profile created while running a DEBUG buil... -
OptimizationTime: 79.001ms\n - PeakMemoryUsage: 281.00 KB (287744)\n
- PrepareTime: 23.000ms\n' =
<tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at
0x69bd690>.runtime_profile
Standard Error
-- executing against localhost:21000
with l as (select * from tpch.lineitem UNION ALL select * from
tpch.lineitem)
select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT
1) a
join (select * from l LIMIT 2000000) b on a.l_orderkey = -b.l_orderkey;;
-- 2018-11-06 18:18:22,759 INFO MainThread: Started query
86445c4beb185e78:bf1b7fc300000000
{noformat}
> Re-enable tests disabled in IMPALA-6338
> ---------------------------------------
>
> Key: IMPALA-7148
> URL: https://issues.apache.org/jira/browse/IMPALA-7148
> Project: IMPALA
> Issue Type: Test
> Affects Versions: Impala 2.13.0, Impala 3.1.0
> Reporter: Thomas Tauber-Marshall
> Assignee: Thomas Tauber-Marshall
> Priority: Critical
>
> The underlying bug in IMPALA-6338 was causing a large number of flaky tests
> and was tricky to fix given our present coordinator runtime profile handling.
> We decided to temporarily disable these tests pending work on the coordinator
> (MPALA-2990/IMPALA-4063). Once that work is done, we should re-enable these
> tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
