[
https://issues.apache.org/jira/browse/IMPALA-15150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-15150.
-------------------------------------
Resolution: Fixed
> test_runtime_filter_annotation_with_hbo_cardinality is flaky
> ------------------------------------------------------------
>
> Key: IMPALA-15150
> URL: https://issues.apache.org/jira/browse/IMPALA-15150
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> test_runtime_filter_annotation_with_hbo_cardinality could fail due to the HBO
> stats on "01:SCAN HDFS [tpch_parquet.orders]" is missing:
> {noformat}
> AssertionError: Max Per-Host Resource Reservation: Memory=28.88MB Threads=6
> Per-Host Resource Estimates: Memory=409MB
>
> PLAN-ROOT SINK
> |
> 07:AGGREGATE [FINALIZE]
> | output: count:merge(l_orderkey)
> | row-size=8B cardinality=1
> |
> 06:EXCHANGE [UNPARTITIONED]
> |
> 03:AGGREGATE
> | output: count(l_orderkey)
> | row-size=8B cardinality=3
> |
> 02:HASH JOIN [INNER JOIN, PARTITIONED]
> | hash predicates: l_orderkey = o_orderkey
> | runtime filters: RF000 <- o_orderkey
> | row-size=62B cardinality=10.95K
> |
> |--05:EXCHANGE [HASH(o_orderkey)]
> | |
> | 01:SCAN HDFS [tpch_parquet.orders]
> | HDFS partitions=1/1 files=2 size=54.21MB
> | predicates: o_custkey < 1000
> | row-size=16B cardinality=150.00K
> |
> 04:EXCHANGE [HASH(l_orderkey)]
> |
> 00:SCAN HDFS [tpch_parquet.lineitem]
> HDFS partitions=1/1 files=3 size=193.97MB
> predicates: l_discount <= 0.07, l_discount >= 0.05, l_quantity < 24,
> l_shipdate < '1995-01-01', l_shipdate >= '1994-01-01'
> runtime filters: RF000 -> l_orderkey
> row-size=46B cardinality=114.16K (from HBO)
> assert 'cardinality=37.88K(filtered from 114.16K from HBO)' in "Max Per-Host
> Resource Reservation: Memory=28.88MB Threads=6\nPer-Host Resource Estimates:
> Memory=409MB\n\nPLAN-ROOT S...', l_shipdate >= '1994-01-01'\n runtime
> filters: RF000 -> l_orderkey\n row-size=46B cardinality=114.16K (from HBO)"
> + where "Max Per-Host Resource Reservation: Memory=28.88MB
> Threads=6\nPer-Host Resource Estimates: Memory=409MB\n\nPLAN-ROOT S...',
> l_shipdate >= '1994-01-01'\n runtime filters: RF000 -> l_orderkey\n
> row-size=46B cardinality=114.16K (from HBO)" = <built-in method join of str
> object at 0x882a98>(['Max Per-Host Resource Reservation: Memory=28.88MB
> Threads=6', 'Per-Host Resource Estimates: Memory=409MB', '', 'PLAN-ROOT
> SINK', '|', '07:AGGREGATE [FINALIZE]', ...])
> + where <built-in method join of str object at 0x882a98> = '\n'.join
> + and ['Max Per-Host Resource Reservation: Memory=28.88MB Threads=6',
> 'Per-Host Resource Estimates: Memory=409MB', '', 'PLAN-ROOT SINK', '|',
> '07:AGGREGATE [FINALIZE]', ...] =
> <tests.common.impala_connection.ImpylaHS2ResultSet object at
> 0x7f9029583950>.data{noformat}
> The cause is that the test doesn't wait for 1s for previous queries to flush
> HBO stats which is asychronous. The other HBO tests all have such a sleep.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
