[
https://issues.apache.org/jira/browse/IMPALA-9956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-9956.
-----------------------------------
Fix Version/s: Impala 4.0
Resolution: Fixed
> Inlining functions in Sorter::Partition() gives a significant speedup.
> ----------------------------------------------------------------------
>
> Key: IMPALA-9956
> URL: https://issues.apache.org/jira/browse/IMPALA-9956
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: performance
> Fix For: Impala 4.0
>
>
> In TPC-H q67, the sort gets ~9% faster just from this small change.
> {noformat}
> AFTER
> Fetched 100 row(s) in 3.27s
> [localhost:21000] tpcds_parquet> summary;
> +-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
> | Operator | #Hosts | #Inst | Avg Time | Max Time | #Rows | Est.
> #Rows | Peak Mem | Est. Peak Mem | Detail |
> +-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
> | F00:ROOT | 1 | 1 | 402.93us | 402.93us | |
> | 0 B | 0 B | |
> | 11:TOP-N | 1 | 1 | 9.85ms | 9.85ms | 100 | 100
> | 7.04 MB | 9.38 KB | |
> | 10:ANALYTIC | 1 | 1 | 102.50ms | 102.50ms | 514.59K | 15.09M
> | 11.02 MB | 4.00 MB | |
> | 09:SORT | 1 | 1 | 304.39ms | 304.39ms | 514.59K | 15.09M
> | 80.02 MB | 38.00 MB | |
> | 08:AGGREGATE | 1 | 1 | 178.87ms | 178.87ms | 514.59K | 15.09M
> | 114.16 MB | 1.42 GB | FINALIZE |
> | 07:AGGREGATE | 1 | 1 | 794.68ms | 794.68ms | 514.59K | 15.09M
> | 257.67 MB | 16.98 GB | FINALIZE |
> | 06:HASH JOIN | 1 | 1 | 33.95ms | 33.95ms | 535.97K | 2.88M
> | 3.33 MB | 2.88 MB | INNER JOIN |
> | |--03:SCAN HDFS | 1 | 1 | 2.77ms | 2.77ms | 18.00K | 18.00K
> | 2.69 MB | 80.00 MB | tpcds_parquet.item |
> | 05:HASH JOIN | 1 | 1 | 8.02ms | 8.02ms | 535.97K | 2.88M
> | 2.05 MB | 1.94 MB | INNER JOIN |
> | |--02:SCAN HDFS | 1 | 1 | 1.18ms | 1.18ms | 12 | 12
> | 69.00 KB | 32.00 MB | tpcds_parquet.store |
> | 04:HASH JOIN | 1 | 1 | 10.22ms | 10.22ms | 535.97K | 2.88M
> | 2.01 MB | 1.94 MB | INNER JOIN |
> | |--01:SCAN HDFS | 1 | 1 | 2.62ms | 2.62ms | 365 | 7.30K
> | 1.33 MB | 80.00 MB | tpcds_parquet.date_dim |
> | 00:SCAN HDFS | 1 | 1 | 9.06ms | 9.06ms | 535.97K | 2.88M
> | 14.37 MB | 64.00 MB | tpcds_parquet.store_sales |
> +-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
> BEFORE
> Fetched 100 row(s) in 3.33s
> [localhost:21000] tpcds_parquet> summary;
> +-----------------+--------+-------+----------+----------+---su------+------------+-----------+---------------+---------------------------+
> | Operator | #Hosts | #Inst | Avg Time | Max Time | #Rows | Est.
> #Rows | Peak Mem | Est. Peak Mem | Detail |
> +-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
> | F00:ROOT | 1 | 1 | 414.22us | 414.22us | |
> | 0 B | 0 B | |
> | 11:TOP-N | 1 | 1 | 10.03ms | 10.03ms | 100 | 100
> | 7.04 MB | 9.38 KB | |
> | 10:ANALYTIC | 1 | 1 | 101.26ms | 101.26ms | 514.59K | 15.09M
> | 11.02 MB | 4.00 MB | |
> | 09:SORT | 1 | 1 | 334.83ms | 334.83ms | 514.59K | 15.09M
> | 80.02 MB | 38.00 MB | |
> | 08:AGGREGATE | 1 | 1 | 221.69ms | 221.69ms | 514.59K | 15.09M
> | 114.16 MB | 1.42 GB | FINALIZE |
> | 07:AGGREGATE | 1 | 1 | 811.38ms | 811.38ms | 514.59K | 15.09M
> | 257.68 MB | 16.98 GB | FINALIZE |
> | 06:HASH JOIN | 1 | 1 | 30.35ms | 30.35ms | 535.97K | 2.88M
> | 3.33 MB | 2.88 MB | INNER JOIN |
> | |--03:SCAN HDFS | 1 | 1 | 2.71ms | 2.71ms | 18.00K | 18.00K
> | 2.63 MB | 80.00 MB | tpcds_parquet.item |
> | 05:HASH JOIN | 1 | 1 | 7.48ms | 7.48ms | 535.97K | 2.88M
> | 2.07 MB | 1.94 MB | INNER JOIN |
> | |--02:SCAN HDFS | 1 | 1 | 869.04us | 869.04us | 12 | 12
> | 69.00 KB | 32.00 MB | tpcds_parquet.store |
> | 04:HASH JOIN | 1 | 1 | 9.77ms | 9.77ms | 535.97K | 2.88M
> | 2.01 MB | 1.94 MB | INNER JOIN |
> | |--01:SCAN HDFS | 1 | 1 | 2.83ms | 2.83ms | 365 | 7.30K
> | 1.33 MB | 80.00 MB | tpcds_parquet.date_dim |
> | 00:SCAN HDFS | 1 | 1 | 8.31ms | 8.31ms | 535.97K | 2.88M
> | 14.35 MB | 64.00 MB | tpcds_parquet.store_sales |
> +-----------------+--------+-------+----------+----------+---------+------------+-----------+---------------+---------------------------+
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
