Michael Smith has posted comments on this change. (
http://gerrit.cloudera.org:8080/20014 )
Change subject: PROTOTYPE: Use new LLVM Pass Manager
......................................................................
Patch Set 1:
(1 comment)
With codegen cache disabled: Report Generated on 2023-06-06
Run Description: "3346d070ad47b49ccbfa087e27b6ecb137b36a0a vs
e8c15c650c7eadb9023fba9a8656cbd6ee88acd3"
Cluster Name: UNKNOWN
Lab Run Info: UNKNOWN
Impala Version: impalad version 4.3.0-SNAPSHOT RELEASE ()
Baseline Impala Version: impalad version 4.3.0-SNAPSHOT RELEASE (2023-06-04)
+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) |
Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(42) | parquet / none / none | 4.24 | -1.15% | 2.93 | -2.62%
|
+----------+-----------------------+---------+------------+------------+----------------+
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+--------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) |
Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval |
Tval |
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+--------+
| TPCH(42) | TPCH-Q18 | parquet / none / none | 10.31 | 10.11 | +1.96%
| 1.04% | 1.03% | 50 | +1.95% | 7.14 | 9.40 |
| TPCH(42) | TPCH-Q9 | parquet / none / none | 11.61 | 11.45 | +1.35%
| 1.17% | 0.96% | 50 | +1.31% | 5.11 | 6.26 |
| TPCH(42) | TPCH-Q7 | parquet / none / none | 3.29 | 3.28 | +0.27%
| 2.89% | 2.77% | 50 | +0.07% | 0.40 | 0.47 |
| TPCH(42) | TPCH-Q13 | parquet / none / none | 7.02 | 7.00 | +0.20%
| 0.75% | 0.58% | 50 | +0.01% | 0.87 | 1.49 |
| TPCH(42) | TPCH-Q3 | parquet / none / none | 4.24 | 4.25 | -0.23%
| 1.55% | 1.87% | 50 | -0.01% | -0.34 | -0.68 |
| TPCH(42) | TPCH-Q15 | parquet / none / none | 2.50 | 2.52 | -0.48%
| 1.89% | 1.62% | 50 | -0.12% | -0.83 | -1.37 |
| TPCH(42) | TPCH-Q10 | parquet / none / none | 3.61 | 3.63 | -0.60%
| 2.43% | 1.68% | 50 | -0.18% | -1.18 | -1.46 |
| TPCH(42) | TPCH-Q6 | parquet / none / none | 0.76 | 0.76 | -0.96%
| 2.35% | 1.36% | 50 | -0.05% | -1.15 | -2.51 |
| TPCH(42) | TPCH-Q21 | parquet / none / none | 21.05 | 21.29 | -1.11%
| 0.69% | 0.74% | 50 | -1.11% | -6.15 | -7.77 |
| TPCH(42) | TPCH-Q1 | parquet / none / none | 5.06 | 5.15 | -1.84%
| 1.30% | 6.88% | 50 | -0.94% | -2.31 | -1.86 |
| TPCH(42) | TPCH-Q8 | parquet / none / none | 3.23 | 3.28 | -1.74%
| 1.70% | 1.62% | 50 | -1.64% | -4.60 | -5.29 |
| TPCH(42) | TPCH-Q4 | parquet / none / none | 2.24 | 2.29 | -2.04%
| 1.90% | 1.48% | 50 | -2.27% | -5.51 | -6.07 |
| TPCH(42) | TPCH-Q22 | parquet / none / none | 2.06 | 2.10 | -2.22%
| 2.63% | 2.33% | 50 | -2.48% | -4.12 | -4.53 |
| TPCH(42) | TPCH-Q16 | parquet / none / none | 1.87 | 1.92 | -2.44%
| 1.83% | 1.74% | 50 | -2.64% | -5.20 | -6.92 |
| TPCH(42) | TPCH-Q17 | parquet / none / none | 2.40 | 2.47 | -2.90%
| 4.69% | 5.38% | 50 | -2.40% | -2.69 | -2.91 |
| TPCH(42) | TPCH-Q14 | parquet / none / none | 1.93 | 1.99 | -3.30%
| 2.05% | 7.51% | 50 | -2.57% | -4.54 | -3.01 |
| TPCH(42) | TPCH-Q20 | parquet / none / none | 1.87 | 1.96 | -4.52%
| 2.64% | 2.82% | 50 | -5.26% | -6.60 | -8.46 |
| TPCH(42) | TPCH-Q5 | parquet / none / none | 2.62 | 2.75 | -4.61%
| 1.52% | 1.25% | 50 | -5.48% | -8.29 | -17.04 |
| TPCH(42) | TPCH-Q12 | parquet / none / none | 1.72 | 1.82 | I -5.70%
| 2.04% | 3.29% | 50 | I -5.85% | -7.76 | -10.58 |
| TPCH(42) | TPCH-Q2 | parquet / none / none | 1.12 | 1.19 | I -6.48%
| 2.87% | 2.88% | 50 | I -8.44% | -7.16 | -11.64 |
| TPCH(42) | TPCH-Q11 | parquet / none / none | 0.89 | 0.96 | I -7.61%
| 5.31% | 4.00% | 50 | I -7.65% | -7.24 | -8.50 |
| TPCH(42) | TPCH-Q19 | parquet / none / none | 2.01 | 2.29 | I
-11.89% | 1.22% | 1.16% | 50 | I -12.93% | -8.53 |
-53.16 |
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------+----------------+---------+--------+
(I) Improvement: TPCH(42) TPCH-Q12 [parquet / none / none] (1.82s -> 1.72s
[-5.70%])
+---------------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+-------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Inst | #Rows | Est
#Rows |
+---------------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+-------+-----------+
| 02:HASH JOIN | 4.63% | 109.33ms | 108.73ms | +0.55% | 9.93%
| 156.00ms | 172.00ms | -9.30% | 3 | 3 | 1.31M | 14.10M |
| F01:EXCHANGE SENDER | 9.47% | 223.81ms | 264.16ms | -15.28% | *
19.67% * | 424.00ms | 459.99ms | -7.83% | 3 | 3 | -1 | -1
|
| 01:SCAN HDFS | 30.29% | 715.91ms | 696.48ms | +2.79% | *
10.87% * | 963.99ms | 975.99ms | -1.23% | 3 | 3 | 1.31M | 13.47M
|
| F00:EXCHANGE SENDER | 5.93% | 140.27ms | 148.65ms | -5.64% | 8.71%
| 188.00ms | 204.00ms | -7.84% | 3 | 3 | -1 | -1 |
| 00:SCAN HDFS | 48.76% | 1.15s | 1.20s | -3.66% | 1.72%
| 1.25s | 1.39s | -10.06% | 3 | 3 | 2.34M | 63.00M |
+---------------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+-------+-----------+
(I) Improvement: TPCH(42) TPCH-Q2 [parquet / none / none] (1.19s -> 1.12s
[-6.48%])
+--------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+--------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) | StdDev(%) |
Max | Base Max | Delta(Max) | #Hosts | #Inst | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+--------+-----------+
| 03:SCAN HDFS | 14.40% | 380.73ms | 445.38ms | -14.52% | 8.56% |
451.99ms | 531.99ms | -15.04% | 1 | 1 | 5 | 25 |
| 01:SCAN HDFS | 14.22% | 376.16ms | 450.93ms | -16.58% | 9.47% |
455.99ms | 515.99ms | -11.63% | 1 | 1 | 84.23K | 420.00K |
| 00:SCAN HDFS | 3.71% | 98.12ms | 99.97ms | -1.85% | 8.01% |
152.00ms | 160.00ms | -5.00% | 3 | 3 | 33.42K | 53.13K |
| 02:SCAN HDFS | 17.96% | 474.85ms | 519.15ms | -8.53% | 5.79% |
623.99ms | 655.99ms | -4.88% | 3 | 3 | 26.62K | 33.60M |
| 07:SCAN HDFS | 13.40% | 354.36ms | 434.04ms | -18.36% | * 11.69% * |
447.99ms | 539.99ms | -17.04% | 1 | 1 | 5 | 25 |
| 06:SCAN HDFS | 13.89% | 367.42ms | 436.73ms | -15.87% | * 12.57% * |
479.99ms | 527.99ms | -9.09% | 1 | 1 | 84.23K | 420.00K |
| 05:SCAN HDFS | 16.79% | 444.02ms | 443.80ms | +0.05% | 8.86% |
583.99ms | 611.99ms | -4.58% | 3 | 3 | 26.60K | 33.60M |
+--------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+--------+-----------+
(I) Improvement: TPCH(42) TPCH-Q11 [parquet / none / none] (0.96s -> 0.89s
[-7.61%])
+---------------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+--------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Inst | #Rows | Est
#Rows |
+---------------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+--------+-----------+
| 07:SCAN HDFS | 20.66% | 259.67ms | 304.32ms | -14.67% | 7.07%
| 348.00ms | 427.99ms | -18.69% | 1 | 1 | 16.88K | 420.00K |
| 06:SCAN HDFS | 18.01% | 226.34ms | 224.08ms | +1.01% | *
14.42% * | 460.00ms | 483.99ms | -4.96% | 3 | 3 | 1.35M | 33.60M
|
| 17:AGGREGATE | 5.22% | 65.60ms | 67.02ms | -2.11% | *
11.13% * | 120.00ms | 116.00ms | +3.45% | 3 | 3 | 1.27M | 1.32M
|
| F00:EXCHANGE SENDER | 4.78% | 60.08ms | 56.03ms | +7.24% | *
16.06% * | 124.00ms | 128.00ms | -3.12% | 3 | 3 | -1 | -1
|
| 05:AGGREGATE | 5.03% | 63.18ms | 64.57ms | -2.15% | *
15.44% * | 132.00ms | 128.00ms | +3.13% | 3 | 3 | 1.32M | 1.32M
|
| 01:SCAN HDFS | 22.83% | 286.85ms | 344.08ms | -16.63% | 7.04%
| 348.00ms | 403.99ms | -13.86% | 1 | 1 | 16.88K | 420.00K |
| 00:SCAN HDFS | 13.32% | 167.37ms | 160.05ms | +4.57% | *
28.84% * | 504.00ms | 495.99ms | +1.61% | 3 | 3 | 1.35M | 33.60M
|
+---------------------+------------+----------+----------+------------+------------+----------+----------+------------+--------+-------+--------+-----------+
(I) Improvement: TPCH(42) TPCH-Q19 [parquet / none / none] (2.29s -> 2.01s
[-11.89%])
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+--------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) | StdDev(%) |
Max | Base Max | Delta(Max) | #Hosts | #Inst | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+--------+-----------+
| 01:SCAN HDFS | 16.06% | 178.91ms | 178.26ms | +0.37% | 2.91% |
232.00ms | 232.00ms | -0.00% | 3 | 3 | 20.33K | 60.14K |
| 00:SCAN HDFS | 80.39% | 895.25ms | 899.94ms | -0.52% | 1.52% |
963.99ms | 967.99ms | -0.41% | 3 | 3 | 13.07K | 8.30M |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+--------+-----------+
Significant perf change detected.
http://gerrit.cloudera.org:8080/#/c/20014/1/be/src/codegen/llvm-codegen.cc
File be/src/codegen/llvm-codegen.cc:
http://gerrit.cloudera.org:8080/#/c/20014/1/be/src/codegen/llvm-codegen.cc@1248
PS1, Line 1248: llvm::ModuleAnalysisManager module_analysis_manager;
There's room to share this with OptimizeModule.
--
To view, visit http://gerrit.cloudera.org:8080/20014
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I947a5b067da50c18f62c3f9af9876463e542f58a
Gerrit-Change-Number: 20014
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: Abhishek Rawat <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Comment-Date: Tue, 06 Jun 2023 16:10:46 +0000
Gerrit-HasComments: Yes
