[
https://issues.apache.org/jira/browse/IMPALA-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-9814.
-----------------------------------
Fix Version/s: Impala 4.0
Resolution: Fixed
> Analytic planner can under-parallelise with mt_dop
> --------------------------------------------------
>
> Key: IMPALA-9814
> URL: https://issues.apache.org/jira/browse/IMPALA-9814
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: multithreading
> Fix For: Impala 4.0
>
>
> This is a TODO-MT in AnalyticPlanner.java. Here is an example of a query that
> is under-parallelised
> {noformat}
>
> set mt_dop=3;
> explain select count(*) over (partition by ss_addr_sk, ss_store_sk),
> count(*) over (partition by ss_sold_date_sk, ss_st
> ore_sk)
> from tpcds_parquet.store_sales;
>
>
> +---------------------------------------------------------------------------+
> | Explain String
> |
>
> +---------------------------------------------------------------------------+
> | Max Per-Host Resource Reservation: Memory=61.50MB Threads=7
> |
> | Per-Host Resource Estimates: Memory=148MB
> |
> |
> |
> | PLAN-ROOT SINK
> |
> | |
> |
> | 06:EXCHANGE [UNPARTITIONED]
> |
> | |
> |
> | 04:ANALYTIC
> |
> | | functions: count(*)
> |
> | | partition by: ss_sold_date_sk, ss_store_sk
> |
> | | row-size=28B cardinality=2.88M
> |
> | |
> |
> | 03:SORT
> |
> | | order by: ss_sold_date_sk ASC NULLS FIRST, ss_store_sk ASC NULLS
> FIRST |
> | | row-size=20B cardinality=2.88M
> |
> | |
> |
> | 02:ANALYTIC
> |
> | | functions: count(*)
> |
> | | partition by: ss_addr_sk, ss_store_sk
> |
> | | row-size=20B cardinality=2.88M
> |
> | |
> |
> | 01:SORT
> |
> | | order by: ss_addr_sk ASC NULLS FIRST, ss_store_sk ASC NULLS FIRST
> |
> | | row-size=12B cardinality=2.88M
> |
> | |
> |
> | 05:EXCHANGE [HASH(ss_store_sk)]
> |
> | |
> |
> | 00:SCAN HDFS [tpcds_parquet.store_sales]
> |
> | HDFS partitions=1824/1824 files=1824 size=196.96MB
> |
> | row-size=12B cardinality=2.88M
> |
>
> +---------------------------------------------------------------------------+
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
