Hello Dan Hecht,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7974
to look at the new patch set (#5).
Change subject: IMPALA-5860: upgrade to LLVM 3.9.1
......................................................................
IMPALA-5860: upgrade to LLVM 3.9.1
LLVM made a few API changes:
* Misc minor changes to function and type signatures
* The CloneFunction() API changed semantics (http://reviews.llvm.org/D18628)
TODO: need to rebase and update toolchain to a later version. It has
gotten behind with the frequent Kudu version bumps.
Testing:
Ran core and ASAN tests.
Perf:
Ran single node TPC-H and targeted perf with scale factor 60. Both
improved on average.
+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) |
Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(60) | parquet / none / none | 17.82 | -5.01% | 11.64 | -4.23%
|
+----------+-----------------------+---------+------------+------------+----------------+
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) |
Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients | Iters |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TPCH(60) | TPCH-Q1 | parquet / none / none | 27.97 | 27.59 | +1.36%
| 0.39% | 0.41% | 1 | 5 |
| TPCH(60) | TPCH-Q20 | parquet / none / none | 5.81 | 5.78 | +0.44%
| 0.73% | 0.21% | 1 | 5 |
| TPCH(60) | TPCH-Q21 | parquet / none / none | 62.98 | 62.98 | +0.01%
| 5.56% | 1.07% | 1 | 5 |
| TPCH(60) | TPCH-Q15 | parquet / none / none | 8.45 | 8.46 | -0.20%
| 0.40% | 0.38% | 1 | 5 |
| TPCH(60) | TPCH-Q4 | parquet / none / none | 5.57 | 5.59 | -0.41%
| 0.43% | 0.80% | 1 | 5 |
| TPCH(60) | TPCH-Q6 | parquet / none / none | 3.16 | 3.17 | -0.45%
| 0.78% | 1.70% | 1 | 5 |
| TPCH(60) | TPCH-Q5 | parquet / none / none | 7.41 | 7.47 | -0.92%
| 0.71% | 1.06% | 1 | 5 |
| TPCH(60) | TPCH-Q9 | parquet / none / none | 33.45 | 33.78 | -0.99%
| 1.15% | 0.85% | 1 | 5 |
| TPCH(60) | TPCH-Q11 | parquet / none / none | 2.00 | 2.03 | -1.34%
| 1.71% | 2.24% | 1 | 5 |
| TPCH(60) | TPCH-Q2 | parquet / none / none | 4.71 | 4.79 | -1.60%
| 1.49% | 1.95% | 1 | 5 |
| TPCH(60) | TPCH-Q18 | parquet / none / none | 46.48 | 47.71 | -2.58%
| 1.04% | 0.38% | 1 | 5 |
| TPCH(60) | TPCH-Q14 | parquet / none / none | 5.85 | 6.02 | -2.84%
| 0.44% | 0.70% | 1 | 5 |
| TPCH(60) | TPCH-Q22 | parquet / none / none | 6.51 | 6.76 | -3.71%
| 2.29% | 2.42% | 1 | 5 |
| TPCH(60) | TPCH-Q19 | parquet / none / none | 7.27 | 7.63 | -4.69%
| 1.33% | 0.78% | 1 | 5 |
| TPCH(60) | TPCH-Q10 | parquet / none / none | 13.19 | 13.84 | -4.73%
| 0.42% | 1.44% | 1 | 5 |
| TPCH(60) | TPCH-Q13 | parquet / none / none | 21.95 | 23.12 | -5.03%
| 0.25% | 1.19% | 1 | 5 |
| TPCH(60) | TPCH-Q16 | parquet / none / none | 5.29 | 5.57 | -5.04%
| 0.85% | 0.78% | 1 | 5 |
| TPCH(60) | TPCH-Q7 | parquet / none / none | 42.05 | 44.33 | -5.16%
| 2.07% | 2.28% | 1 | 5 |
| TPCH(60) | TPCH-Q12 | parquet / none / none | 19.77 | 21.00 | -5.87%
| 8.14% | 5.09% | 1 | 5 |
| TPCH(60) | TPCH-Q3 | parquet / none / none | 11.46 | 12.32 | -6.94%
| 0.76% | 0.53% | 1 | 5 |
| TPCH(60) | TPCH-Q17 | parquet / none / none | 40.09 | 49.28 |
-18.64% | 2.09% | 0.67% | 1 | 5 |
| TPCH(60) | TPCH-Q8 | parquet / none / none | 10.63 | 13.47 | I
-21.08% | * 12.34% * | * 21.09% * | 1 | 5 |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
+-------------------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s)
| Delta(GeoMean) |
+-------------------+-----------------------+---------+------------+------------+----------------+
| TARGETED-PERF(60) | parquet / none / none | 22.38 | -1.24% | 4.17
| +0.81% |
+-------------------+-----------------------+---------+------------+------------+----------------+
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload | Query |
File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base
StdDev(%) | Num Clients | Iters |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TARGETED-PERF(60) | primitive_conjunct_ordering_1 |
parquet / none / none | 0.12 | 0.10 | R +22.38% | 0.81% | *
27.26% * | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_decimal_highndv |
parquet / none / none | 29.86 | 25.46 | +17.31% | 6.18% |
3.83% | 1 | 5 |
| TARGETED-PERF(60) | PERF_LIMIT-Q1 |
parquet / none / none | 0.01 | 0.01 | +13.41% | * 15.35% * |
2.95% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_bigint_non_selective |
parquet / none / none | 0.88 | 0.82 | +7.17% | 9.52% |
3.59% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_decimal_non_selective |
parquet / none / none | 1.48 | 1.41 | +4.94% | 4.23% |
1.86% | 1 | 5 |
| TARGETED-PERF(60) | primitive_small_join_1 |
parquet / none / none | 0.18 | 0.18 | +4.26% | * 11.92% * |
2.43% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_3 |
parquet / none / none | 7.29 | 7.03 | +3.77% | 5.98% |
9.35% | 1 | 5 |
| TARGETED-PERF(60) | primitive_exchange_broadcast |
parquet / none / none | 38.41 | 37.02 | +3.77% | 8.59% |
1.31% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q6 |
parquet / none / none | 1.93 | 1.89 | +2.14% | 2.22% |
1.75% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_2 |
parquet / none / none | 7.26 | 7.17 | +1.29% | 2.28% |
4.54% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q1 |
parquet / none / none | 2.79 | 2.75 | +1.28% | 0.52% |
0.76% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q3 |
parquet / none / none | 3.51 | 3.47 | +1.01% | 0.63% |
0.57% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_selective |
parquet / none / none | 1.05 | 1.04 | +0.76% | 3.03% |
2.40% | 1 | 5 |
| TARGETED-PERF(60) | primitive_orderby_bigint |
parquet / none / none | 4.88 | 4.84 | +0.75% | 0.58% |
0.97% | 1 | 5 |
| TARGETED-PERF(60) | primitive_top-n_all |
parquet / none / none | 38.56 | 38.28 | +0.73% | 0.20% |
0.24% | 1 | 5 |
| TARGETED-PERF(60) | primitive_orderby_all |
parquet / none / none | 25.68 | 25.54 | +0.55% | 0.27% |
0.40% | 1 | 5 |
| TARGETED-PERF(60) | primitive_shuffle_join_union_all_with_groupby |
parquet / none / none | 54.02 | 53.74 | +0.53% | 0.35% |
0.23% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q5 |
parquet / none / none | 4.28 | 4.26 | +0.43% | 0.68% |
0.47% | 1 | 5 |
| TARGETED-PERF(60) | primitive_empty_build_join_1 |
parquet / none / none | 16.25 | 16.19 | +0.42% | 0.33% |
0.42% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_highndv |
parquet / none / none | 32.49 | 32.36 | +0.42% | 0.23% |
0.88% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q1 |
parquet / none / none | 2.22 | 2.21 | +0.34% | 1.82% |
1.88% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_pk |
parquet / none / none | 112.73 | 112.50 | +0.21% | 0.75% |
0.99% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q4 |
parquet / none / none | 3.52 | 3.51 | +0.13% | 0.58% |
0.65% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q2 |
parquet / none / none | 3.06 | 3.06 | +0.03% | 0.69% |
0.76% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_decimal_selective |
parquet / none / none | 1.20 | 1.20 | -0.01% | 2.35% |
1.24% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_2 |
parquet / none / none | 4.27 | 4.27 | -0.03% | 0.52% |
0.48% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_decimal_lowndv.test |
parquet / none / none | 3.87 | 3.87 | -0.07% | 1.69% |
1.63% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q7 |
parquet / none / none | 1.92 | 1.93 | -0.28% | 2.33% |
1.94% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q5 |
parquet / none / none | 0.48 | 0.48 | -0.28% | 0.59% |
0.53% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q4 |
parquet / none / none | 17.48 | 17.53 | -0.30% | 0.43% |
0.58% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q7 |
parquet / none / none | 7.87 | 7.90 | -0.35% | 0.67% |
0.55% | 1 | 5 |
| TARGETED-PERF(60) | primitive_exchange_shuffle |
parquet / none / none | 74.25 | 74.53 | -0.37% | 0.57% |
0.36% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_lowndv |
parquet / none / none | 3.81 | 3.82 | -0.42% | 1.51% |
1.10% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q2 |
parquet / none / none | 9.93 | 10.00 | -0.67% | 0.77% |
0.67% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_like |
parquet / none / none | 14.63 | 14.74 | -0.72% | 0.24% |
0.02% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_4 |
parquet / none / none | 0.23 | 0.23 | -0.82% | 0.59% |
1.31% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q6 |
parquet / none / none | 9.87 | 10.03 | -1.55% | 0.39% |
0.22% | 1 | 5 |
| TARGETED-PERF(60) | primitive_shuffle_join_one_to_many_string_with_groupby |
parquet / none / none | 262.13 | 268.18 | -2.26% | 0.31% |
0.27% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_non_selective |
parquet / none / none | 1.23 | 1.26 | -2.26% | 1.72% |
2.15% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_1 |
parquet / none / none | 2.04 | 2.09 | -2.54% | 0.31% |
2.88% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_3 |
parquet / none / none | 0.13 | 0.13 | -3.13% | 0.73% |
2.50% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_bigint_selective |
parquet / none / none | 0.12 | 0.12 | -3.15% | 1.03% |
1.73% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_5 |
parquet / none / none | 14.11 | 14.60 | -3.33% | 2.03% |
2.43% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q3 |
parquet / none / none | 8.28 | 8.64 | -4.17% | 0.79% |
1.08% | 1 | 5 |
| TARGETED-PERF(60) | primitive_long_predicate |
parquet / none / none | 215.27 | 227.90 | -5.54% | 0.06% |
0.08% | 1 | 5 |
| TARGETED-PERF(60) | primitive_topn_bigint |
parquet / none / none | 4.48 | 4.81 | -6.90% | 8.50% | *
15.79% * | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_in_predicate |
parquet / none / none | 1.84 | 1.99 | -7.51% | 3.98% |
5.29% | 1 | 5 |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
Change-Id: Ida873ddb15e393b0bd37486db24add8a32f43ad0
---
M be/src/codegen/codegen-symbol-emitter.cc
M be/src/codegen/instruction-counter-test.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/common/compiler-util.h
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
7 files changed, 31 insertions(+), 25 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/7974/5
--
To view, visit http://gerrit.cloudera.org:8080/7974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ida873ddb15e393b0bd37486db24add8a32f43ad0
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Dan Hecht <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
