Tim Armstrong has posted comments on this change.
Change subject: Upgrade LLVM to 3.8.0
......................................................................
Patch Set 7:
Here is the cluster perf results against a recent baseline
(0d7189c545962b6774500275d34068d27d9936b2).
There are a few apparent regressions. Digging into them I didn't find anything
directly connected to codegen. primitive_filter_bigint_non_selective looks like
random noise (due to the high stddev). For TPCH-Q14 and
primitive_exchange_broadcast
the non-codegened hash table build took more CPU time. All other measures of
CPU time
were the same or lower.
Report Generated on 2016-04-11
Cluster Name: UNKNOWN
Lab Run Info: UNKNOWN
Impala Version: impalad version 2.6.0-cdh5-INTERNAL RELEASE ()
Baseline Impala Version: impalad version 2.6.0-cdh5-INTERNAL RELEASE
(2016-04-10)
+---------------------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) |
GeoMean(s) | Delta(GeoMean) |
+---------------------+-----------------------+---------+------------+------------+----------------+
| TPCH_NESTED(_300) | parquet / none / none | 43.86 | -0.35% |
18.77 | -0.19% |
| TPCH(_300) | parquet / none / none | 39.87 | -1.42% |
22.69 | -1.13% |
| TPCDS(_1000) | parquet / none / none | 9.23 | -3.94% |
3.33 | -1.41% |
| TARGETED-PERF(_300) | parquet / none / none | 43.85 | -2.19% |
13.76 | -3.00% |
+---------------------+-----------------------+---------+------------+------------+----------------+
+---------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload | Query
| File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)
| Base StdDev(%) | Num Clients | Iters |
+---------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TPCH(_300) | TPCH-Q14
| parquet / none / none | 8.18 | 6.15 | R +33.01% | 0.61%
| 1.24% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_exchange_broadcast
| parquet / none / none | 111.99 | 85.48 | R +31.01% | 0.82%
| 3.50% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_filter_bigint_non_selective
| parquet / none / none | 2.17 | 1.76 | R +23.12% | * 18.45% *
| 0.02% | 1 | 3 |
| TPCH(_300) | TPCH-Q16
| parquet / none / none | 10.66 | 9.76 | +9.28% | 0.96%
| 0.99% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q61
| parquet / none / none | 2.91 | 2.69 | +8.38% | 3.10%
| 6.81% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q10
| parquet / none / none | 9.94 | 9.39 | +5.85% | 0.48%
| 2.65% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q52
| parquet / none / none | 0.99 | 0.94 | +5.62% | 7.53%
| 1.70% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_filter_decimal_selective
| parquet / none / none | 1.69 | 1.61 | +4.83% | 2.99%
| 1.31% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q53
| parquet / none / none | 1.61 | 1.54 | +4.32% | 6.36%
| 1.38% | 1 | 3 |
| TPCH(_300) | TPCH-Q12
| parquet / none / none | 16.95 | 16.30 | +4.02% | 0.79%
| 4.79% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q27
| parquet / none / none | 2.76 | 2.66 | +3.81% | 2.72%
| 0.78% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q5
| parquet / none / none | 19.23 | 18.54 | +3.76% | 0.05%
| 0.29% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q7
| parquet / none / none | 3.03 | 2.92 | +3.54% | 1.81%
| 0.13% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q4
| parquet / none / none | 7.12 | 6.89 | +3.35% | 0.31%
| 2.21% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q18
| parquet / none / none | 79.96 | 77.37 | +3.35% | 2.36%
| 0.46% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_filter_bigint_selective
| parquet / none / none | 1.49 | 1.44 | +3.15% | 0.73%
| 4.99% | 1 | 3 |
| TPCH(_300) | TPCH-Q17
| parquet / none / none | 130.53 | 126.99 | +2.78% | 1.12%
| 2.91% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q11
| parquet / none / none | 3.68 | 3.59 | +2.47% | 2.17%
| 2.42% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q3
| parquet / none / none | 8.12 | 7.93 | +2.41% | 1.29%
| 0.34% | 1 | 3 |
| TPCH(_300) | TPCH-Q19
| parquet / none / none | 162.88 | 159.09 | +2.38% | 4.79%
| 0.47% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_topn_bigint
| parquet / none / none | 12.91 | 12.64 | +2.16% | 1.34%
| 0.30% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q55
| parquet / none / none | 0.97 | 0.95 | +2.16% | 4.93%
| 3.14% | 1 | 3 |
| TPCH(_300) | TPCH-Q3
| parquet / none / none | 33.51 | 32.82 | +2.11% | 1.14%
| 0.30% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q8
| parquet / none / none | 1.68 | 1.65 | +2.00% | 0.22%
| 1.21% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q63
| parquet / none / none | 1.54 | 1.51 | +1.95% | 2.17%
| 3.12% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q96
| parquet / none / none | 3.86 | 3.79 | +1.84% | 2.42%
| 0.68% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q21
| parquet / none / none | 68.59 | 67.39 | +1.78% | 0.60%
| 4.60% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q13
| parquet / none / none | 10.78 | 10.66 | +1.12% | 1.38%
| 0.16% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_top-n_all
| parquet / none / none | 82.08 | 81.38 | +0.87% | 1.56%
| 0.59% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q59
| parquet / none / none | 21.49 | 21.34 | +0.71% | 0.19%
| 2.22% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q43
| parquet / none / none | 3.83 | 3.80 | +0.67% | 1.21%
| 1.92% | 1 | 3 |
| TPCH(_300) | TPCH-Q2
| parquet / none / none | 8.67 | 8.63 | +0.48% | 0.67%
| 1.28% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q2
| parquet / none / none | 19.84 | 19.82 | +0.10% | 1.29%
| 0.03% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q17
| parquet / none / none | 133.17 | 133.20 | -0.02% | 2.48%
| 2.44% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q15
| parquet / none / none | 6.75 | 6.75 | -0.07% | 1.48%
| 2.35% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q9
| parquet / none / none | 273.67 | 273.93 | -0.09% | 0.39%
| 4.31% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q89
| parquet / none / none | 1.98 | 1.98 | -0.10% | 0.23%
| 2.88% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q8
| parquet / none / none | 35.80 | 35.89 | -0.27% | 0.03%
| 0.23% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_filter_string_non_selective
| parquet / none / none | 2.00 | 2.01 | -0.45% | 2.88%
| 3.83% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q22
| parquet / none / none | 2.17 | 2.18 | -0.52% | 0.75%
| 2.35% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_orderby_bigint
| parquet / none / none | 31.84 | 32.01 | -0.52% | 2.24%
| 0.72% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q88
| parquet / none / none | 13.28 | 13.35 | -0.52% | 1.80%
| 0.22% | 1 | 3 |
| TPCH(_300) | TPCH-Q1
| parquet / none / none | 19.16 | 19.29 | -0.67% | 0.67%
| 0.74% | 1 | 3 |
| TARGETED-PERF(_300) |
primitive_shuffle_join_one_to_many_string_with_groupby | parquet / none / none
| 239.36 | 241.38 | -0.83% | 0.09% | 0.08% | 1
| 3 |
| TPCH(_300) | TPCH-Q13
| parquet / none / none | 33.97 | 34.35 | -1.10% | 2.83%
| 0.37% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_filter_string_selective
| parquet / none / none | 1.88 | 1.90 | -1.11% | 1.19%
| 0.13% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q73
| parquet / none / none | 1.33 | 1.35 | -1.12% | 1.38%
| 0.41% | 1 | 3 |
| TPCH(_300) | TPCH-Q11
| parquet / none / none | 6.68 | 6.77 | -1.35% | 0.94%
| 1.47% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q1
| parquet / none / none | 19.72 | 20.02 | -1.47% | 0.88%
| 0.13% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q34
| parquet / none / none | 2.86 | 2.90 | -1.57% | 0.67%
| 0.82% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q6
| parquet / none / none | 80.55 | 81.86 | -1.59% | 0.44%
| 1.51% | 1 | 3 |
| TPCH(_300) | TPCH-Q22
| parquet / none / none | 8.02 | 8.16 | -1.64% | 3.09%
| 0.13% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q19
| parquet / none / none | 1.17 | 1.19 | -1.79% | 0.26%
| 1.84% | 1 | 3 |
| TPCH(_300) | TPCH-Q18
| parquet / none / none | 87.85 | 89.51 | -1.86% | 0.22%
| 0.70% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q12
| parquet / none / none | 7.04 | 7.19 | -2.00% | 1.41%
| 2.47% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q7
| parquet / none / none | 58.35 | 59.56 | -2.03% | 1.54%
| 0.17% | 1 | 3 |
| TPCH(_300) | TPCH-Q21
| parquet / none / none | 124.27 | 126.87 | -2.05% | 2.81%
| 0.76% | 1 | 3 |
| TPCH(_300) | TPCH-Q4
| parquet / none / none | 33.56 | 34.35 | -2.29% | 3.33%
| 2.82% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_orderby_all
| parquet / none / none | 115.69 | 118.42 | -2.31% | 0.33%
| 0.59% | 1 | 3 |
| TPCH(_300) | TPCH-Q8
| parquet / none / none | 9.09 | 9.31 | -2.37% | 2.33%
| 2.42% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q16
| parquet / none / none | 16.54 | 16.95 | -2.41% | 0.52%
| 3.46% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_filter_string_like
| parquet / none / none | 6.77 | 6.94 | -2.43% | 0.75%
| 2.80% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_groupby_bigint_lowndv
| parquet / none / none | 4.20 | 4.31 | -2.46% | 1.24%
| 1.21% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q46
| parquet / none / none | 3.01 | 3.09 | -2.50% | 0.14%
| 0.29% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q14
| parquet / none / none | 6.83 | 7.02 | -2.67% | 0.27%
| 2.10% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_broadcast_join_1
| parquet / none / none | 1.87 | 1.92 | -2.77% | 1.97%
| * 10.86% * | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q19
| parquet / none / none | 156.05 | 160.63 | -2.85% | 0.68%
| 0.38% | 1 | 3 |
| TPCH(_300) | TPCH-Q20
| parquet / none / none | 16.07 | 16.57 | -3.03% | 0.57%
| 0.96% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q47
| parquet / none / none | 40.94 | 42.34 | -3.29% | 1.28%
| 2.43% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q68
| parquet / none / none | 2.54 | 2.63 | -3.35% | 0.29%
| 1.24% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_groupby_decimal_lowndv.test
| parquet / none / none | 4.10 | 4.25 | -3.48% | 2.42%
| 1.09% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q3
| parquet / none / none | 1.08 | 1.12 | -3.50% | 0.28%
| 4.20% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_broadcast_join_3
| parquet / none / none | 53.17 | 55.15 | -3.59% | 1.37%
| 1.12% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q6
| parquet / none / none | 4.69 | 4.92 | -4.54% | 0.08%
| 1.65% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_exchange_shuffle
| parquet / none / none | 87.46 | 93.04 | -6.00% | 0.12%
| 1.29% | 1 | 3 |
| TPCH(_300) | TPCH-Q15
| parquet / none / none | 5.77 | 6.18 | -6.52% | 0.49%
| 4.42% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q42
| parquet / none / none | 0.89 | 0.96 | -7.52% | 1.74%
| 5.06% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_groupby_bigint_pk
| parquet / none / none | 98.71 | 107.67 | -8.31% | 3.52%
| 1.08% | 1 | 3 |
| TPCH(_300) | TPCH-Q7
| parquet / none / none | 47.26 | 51.70 | -8.59% | 1.10%
| 0.00% | 1 | 3 |
| TPCH_NESTED(_300) | TPCH-Q20
| parquet / none / none | 16.95 | 18.57 | -8.73% | 0.11%
| 3.85% | 1 | 3 |
| TPCH(_300) | TPCH-Q9
| parquet / none / none | 40.13 | 44.25 | -9.30% | 1.09%
| 1.73% | 1 | 3 |
| TPCH(_300) | TPCH-Q6
| parquet / none / none | 3.49 | 3.85 | -9.31% | 0.57%
| 1.90% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q98
| parquet / none / none | 1.38 | 1.53 | -10.17% | 2.21%
| 1.46% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_broadcast_join_2
| parquet / none / none | 6.26 | 6.99 | -10.35% | 3.09%
| 2.49% | 1 | 3 |
| TPCH(_300) | TPCH-Q10
| parquet / none / none | 20.92 | 23.33 | -10.35% | 0.85%
| 2.23% | 1 | 3 |
| TPCH(_300) | TPCH-Q5
| parquet / none / none | 49.56 | 55.57 | -10.81% | 0.19%
| 5.68% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_shuffle_join_union_all_with_groupby
| parquet / none / none | 86.70 | 97.53 | -11.11% | 0.26%
| 0.67% | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q79
| parquet / none / none | 2.97 | 3.38 | -11.99% | 1.22%
| 2.03% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_groupby_bigint_highndv
| parquet / none / none | 26.61 | 31.10 | -14.43% | 0.00%
| 3.90% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_filter_decimal_non_selective
| parquet / none / none | 1.85 | 2.23 | -16.92% | 1.44%
| * 14.83% * | 1 | 3 |
| TPCDS(_1000) | TPCDS-Q65
| parquet / none / none | 32.12 | 38.79 | -17.19% | 0.43%
| 5.75% | 1 | 3 |
| TARGETED-PERF(_300) | primitive_groupby_decimal_highndv
| parquet / none / none | 27.70 | 39.99 | I -30.73% | 5.08%
| 0.86% | 1 | 3 |
+---------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
(R) Regression: TPCH(_300) TPCH-Q14 [parquet / none / none] (6.15s ->
8.18s [+33.01%])
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+--------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+--------+-----------+
| 07:AGGREGATE | 3.76% | 180.52ms | 183.73ms | -1.75% |
0.86% | 182.07ms | 184.73ms | -1.44% | 1 | 1 | 1 |
| 03:AGGREGATE | 6.24% | 299.29ms | 301.28ms | -0.66% |
0.50% | 338.70ms | 361.01ms | -6.18% | 15 | 15 | 1 |
| 02:HASH JOIN | 35.83% | 1.72s | 1.66s | +3.50% |
0.53% | 2.01s | 1.98s | +1.51% | 15 | 22.45M | 180.00M |
| 05:EXCHANGE | 3.86% | 185.08ms | 183.97ms | +0.60% |
1.18% | 229.54ms | 233.73ms | -1.79% | 15 | 60.00M | 60.00M |
| 00:SCAN HDFS | 48.09% | 2.31s | 2.33s | -0.80% |
0.67% | 2.59s | 2.62s | -1.01% | 15 | 22.45M | 180.00M |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+--------+-----------+
(R) Regression: TARGETED-PERF(_300) primitive_exchange_broadcast
[parquet / none / none] (85.48s -> 111.99s [+31.01%])
+--------------+------------+--------+----------+------------+-----------+--------+----------+------------+--------+---------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Rows | Est #Rows |
+--------------+------------+--------+----------+------------+-----------+--------+----------+------------+--------+---------+-----------+
| 02:HASH JOIN | 81.11% | 87.04s | 68.53s | +27.02% | 0.58%
| 87.55s | 70.53s | +24.12% | 1 | 0 | 3 |
| 04:EXCHANGE | 17.92% | 19.23s | 12.93s | +48.74% | 0.11%
| 19.25s | 13.89s | +38.61% | 1 | 350.01M | 180.00M |
+--------------+------------+--------+----------+------------+-----------+--------+----------+------------+--------+---------+-----------+
(R) Regression: TARGETED-PERF(_300)
primitive_filter_bigint_non_selective [parquet / none / none] (1.76s -> 2.17s
[+23.12%])
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+-----------+
| 03:AGGREGATE | 19.73% | 280.50ms | 276.55ms | +1.43% |
0.62% | 282.26ms | 280.83ms | +0.51% | 1 | 1 | 1 |
| 01:AGGREGATE | 47.88% | 680.71ms | 644.38ms | +5.64% |
4.36% | 1.47s | 721.60ms | +103.86% | 15 | 15 | 1 |
| 00:SCAN HDFS | 32.37% | 460.26ms | 490.06ms | -6.08% |
2.07% | 590.43ms | 598.85ms | -1.41% | 15 | 1.80B | 180.00M |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+-----------+
(I) Improvement: TARGETED-PERF(_300) primitive_groupby_decimal_highndv
[parquet / none / none] (39.99s -> 27.70s [-30.73%])
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+--------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+--------+-----------+
| 03:AGGREGATE | 2.66% | 604.51ms | 606.86ms | -0.39% |
1.03% | 646.11ms | 698.87ms | -7.55% | 15 | 0 | 365.14K |
| 01:AGGREGATE | 94.31% | 21.43s | 24.44s | -12.33% |
0.32% | 27.45s | 38.53s | -28.77% | 15 | 56.48M | 3.65M |
| 00:SCAN HDFS | 2.28% | 518.32ms | 530.63ms | -2.32% |
1.03% | 695.83ms | 663.29ms | +4.91% | 15 | 1.80B | 1.80B |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+--------+-----------+
(V) Significant Variability: TARGETED-PERF(_300)
primitive_filter_bigint_non_selective [parquet / none / none] (0.02% -> 18.45%)
No Nodes with significant StdDev %
Significant perf change detected
--
To view, visit http://gerrit.cloudera.org:8080/2486
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I17d7afd05ad3b472a0bfe035bfc3daada5597b2d
Gerrit-PatchSet: 7
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Dan Hecht <[email protected]>
Gerrit-Reviewer: Michael Ho <[email protected]>
Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-HasComments: No
