Hello Tim Armstrong, Dan Hecht, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9584
to look at the new patch set (#5).
Change subject: IMPALA-5980: Upgrade to LLVM 5.0.1
......................................................................
IMPALA-5980: Upgrade to LLVM 5.0.1
Highlighting a few changes in LLVM:
- Minor changes to some function signatures
- Minor changes to error handling
- Split Bitcode/ReaderWriter.h - https://reviews.llvm.org/D26502
- Introduced an optional new GVN optimization pass.
Needed to fix a bunch of new clang-tidy warnings.
Testing:
Ran core and ASAN tests successfully.
Performance:
Ran single node TPC-H and targeted perf with scale factor 60. Both
improved on average. Identified regression in
"primitive_filter_in_predicate" which will be addressed by IMPALA-6621.
+-------------------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s)
| Delta(GeoMean) |
+-------------------+-----------------------+---------+------------+------------+----------------+
| TARGETED-PERF(60) | parquet / none / none | 22.29 | -0.12% | 3.90
| +3.16% |
| TPCH(60) | parquet / none / none | 15.97 | -3.64% | 10.14
| -4.92% |
+-------------------+-----------------------+---------+------------+------------+----------------+
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload | Query |
File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base
StdDev(%) | Num Clients | Iters |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TARGETED-PERF(60) | PERF_LIMIT-Q1 |
parquet / none / none | 0.01 | 0.00 | R +156.43% | * 25.80% * | *
17.14% * | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_in_predicate |
parquet / none / none | 3.39 | 1.92 | R +76.33% | 3.23% |
4.37% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_non_selective |
parquet / none / none | 1.25 | 1.11 | +12.46% | 3.41% |
5.36% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_decimal_selective |
parquet / none / none | 1.40 | 1.25 | +12.25% | 3.57% |
3.44% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_like |
parquet / none / none | 16.87 | 15.65 | +7.78% | 5.05% |
0.37% | 1 | 5 |
| TARGETED-PERF(60) | primitive_min_max_runtime_filter |
parquet / none / none | 1.79 | 1.71 | +4.77% | 0.71% |
1.73% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_2 |
parquet / none / none | 0.60 | 0.58 | +3.64% | 3.19% |
3.81% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_string_selective |
parquet / none / none | 0.95 | 0.93 | +2.91% | 5.23% |
5.85% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_3 |
parquet / none / none | 4.33 | 4.21 | +2.83% | 5.46% |
3.25% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_lowndv |
parquet / none / none | 4.59 | 4.47 | +2.82% | 3.73% |
1.14% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_3 |
parquet / none / none | 0.20 | 0.19 | +2.65% | 4.76% |
2.24% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q1 |
parquet / none / none | 2.49 | 2.43 | +2.31% | 1.06% |
1.93% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q6 |
parquet / none / none | 2.04 | 2.00 | +2.09% | 3.51% |
2.80% | 1 | 5 |
| TPCH(60) | TPCH-Q3 |
parquet / none / none | 12.37 | 12.17 | +1.62% | 0.80% |
2.45% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q5 |
parquet / none / none | 4.52 | 4.45 | +1.54% | 1.23% |
1.08% | 1 | 5 |
| TPCH(60) | TPCH-Q6 |
parquet / none / none | 2.95 | 2.91 | +1.33% | 1.92% |
1.67% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q4 |
parquet / none / none | 3.71 | 3.66 | +1.26% | 0.34% |
0.53% | 1 | 5 |
| TPCH(60) | TPCH-Q1 |
parquet / none / none | 18.69 | 18.47 | +1.19% | 0.75% |
0.31% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q7 |
parquet / none / none | 8.15 | 8.07 | +0.99% | 3.92% |
1.58% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_decimal_highndv |
parquet / none / none | 31.31 | 31.01 | +0.97% | 1.74% |
1.14% | 1 | 5 |
| TPCH(60) | TPCH-Q5 |
parquet / none / none | 7.59 | 7.53 | +0.78% | 0.38% |
0.99% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q4 |
parquet / none / none | 21.25 | 21.09 | +0.76% | 0.76% |
0.75% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_4 |
parquet / none / none | 0.24 | 0.24 | +0.75% | 3.14% |
4.76% | 1 | 5 |
| TPCH(60) | TPCH-Q19 |
parquet / none / none | 7.88 | 7.82 | +0.74% | 2.39% |
2.64% | 1 | 5 |
| TARGETED-PERF(60) | primitive_orderby_bigint |
parquet / none / none | 5.10 | 5.07 | +0.61% | 0.74% |
0.54% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q3 |
parquet / none / none | 3.61 | 3.59 | +0.60% | 1.45% |
0.90% | 1 | 5 |
| TARGETED-PERF(60) | primitive_orderby_all |
parquet / none / none | 27.63 | 27.48 | +0.55% | 0.85% |
0.10% | 1 | 5 |
| TPCH(60) | TPCH-Q4 |
parquet / none / none | 5.81 | 5.79 | +0.45% | 1.65% |
2.16% | 1 | 5 |
| TPCH(60) | TPCH-Q13 |
parquet / none / none | 23.49 | 23.43 | +0.27% | 0.83% |
0.63% | 1 | 5 |
| TPCH(60) | TPCH-Q21 |
parquet / none / none | 68.88 | 68.76 | +0.18% | 0.22% |
0.19% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_decimal_lowndv.test |
parquet / none / none | 4.38 | 4.37 | +0.09% | 2.45% |
0.45% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_5 |
parquet / none / none | 10.40 | 10.40 | +0.07% | 0.77% |
0.50% | 1 | 5 |
| TARGETED-PERF(60) | primitive_long_predicate |
parquet / none / none | 222.37 | 222.23 | +0.06% | 0.25% |
0.25% | 1 | 5 |
| TPCH(60) | TPCH-Q8 |
parquet / none / none | 10.65 | 10.65 | +0.03% | 0.55% |
1.40% | 1 | 5 |
| TARGETED-PERF(60) | primitive_shuffle_join_one_to_many_string_with_groupby |
parquet / none / none | 261.84 | 261.87 | -0.01% | 0.91% |
0.74% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q3 |
parquet / none / none | 9.44 | 9.45 | -0.02% | 0.92% |
1.33% | 1 | 5 |
| TPCH(60) | TPCH-Q16 |
parquet / none / none | 5.21 | 5.21 | -0.02% | 1.46% |
1.64% | 1 | 5 |
| TARGETED-PERF(60) | primitive_top-n_all |
parquet / none / none | 34.58 | 34.62 | -0.11% | 0.22% |
0.19% | 1 | 5 |
| TARGETED-PERF(60) | primitive_topn_bigint |
parquet / none / none | 4.24 | 4.25 | -0.13% | 6.66% |
2.03% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q2 |
parquet / none / none | 3.23 | 3.24 | -0.34% | 2.03% |
0.32% | 1 | 5 |
| TARGETED-PERF(60) | primitive_broadcast_join_1 |
parquet / none / none | 0.18 | 0.18 | -0.40% | 6.16% |
2.45% | 1 | 5 |
| TARGETED-PERF(60) | primitive_exchange_broadcast |
parquet / none / none | 46.27 | 46.51 | -0.52% | 7.83% | *
15.60% * | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_pk |
parquet / none / none | 114.32 | 114.92 | -0.52% | 0.24% |
0.61% | 1 | 5 |
| TPCH(60) | TPCH-Q22 |
parquet / none / none | 6.66 | 6.70 | -0.53% | 1.39% |
0.84% | 1 | 5 |
| TPCH(60) | TPCH-Q20 |
parquet / none / none | 5.78 | 5.81 | -0.62% | 1.25% |
0.67% | 1 | 5 |
| TPCH(60) | TPCH-Q2 |
parquet / none / none | 2.53 | 2.55 | -0.64% | 3.86% |
3.72% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q5 |
parquet / none / none | 0.58 | 0.58 | -0.75% | 0.99% |
6.89% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q7 |
parquet / none / none | 2.05 | 2.07 | -0.86% | 2.16% |
4.73% | 1 | 5 |
| TARGETED-PERF(60) | primitive_shuffle_join_union_all_with_groupby |
parquet / none / none | 54.86 | 55.34 | -0.87% | 0.25% |
0.66% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_2 |
parquet / none / none | 7.52 | 7.59 | -0.98% | 1.53% |
1.73% | 1 | 5 |
| TPCH(60) | TPCH-Q9 |
parquet / none / none | 36.43 | 36.79 | -1.00% | 1.60% |
7.39% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q1 |
parquet / none / none | 2.79 | 2.82 | -1.10% | 1.15% |
2.25% | 1 | 5 |
| TPCH(60) | TPCH-Q11 |
parquet / none / none | 1.95 | 1.97 | -1.18% | 3.14% |
2.24% | 1 | 5 |
| TARGETED-PERF(60) | PERF_AGG-Q2 |
parquet / none / none | 10.98 | 11.11 | -1.24% | 0.77% |
1.45% | 1 | 5 |
| TARGETED-PERF(60) | primitive_small_join_1 |
parquet / none / none | 0.22 | 0.22 | -1.34% | * 13.03% * | *
12.31% * | 1 | 5 |
| TPCH(60) | TPCH-Q7 |
parquet / none / none | 42.82 | 43.41 | -1.37% | 1.63% |
1.51% | 1 | 5 |
| TARGETED-PERF(60) | primitive_empty_build_join_1 |
parquet / none / none | 3.30 | 3.35 | -1.54% | 2.15% |
1.27% | 1 | 5 |
| TARGETED-PERF(60) | PERF_STRING-Q6 |
parquet / none / none | 10.34 | 10.54 | -1.81% | 0.24% |
2.02% | 1 | 5 |
| TARGETED-PERF(60) | primitive_groupby_bigint_highndv |
parquet / none / none | 32.80 | 33.46 | -1.98% | 1.29% |
0.61% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_decimal_non_selective |
parquet / none / none | 1.62 | 1.67 | -3.01% | 0.79% |
1.65% | 1 | 5 |
| TARGETED-PERF(60) | primitive_conjunct_ordering_1 |
parquet / none / none | 0.13 | 0.14 | -3.36% | 8.66% | *
12.66% * | 1 | 5 |
| TARGETED-PERF(60) | primitive_exchange_shuffle |
parquet / none / none | 84.92 | 87.96 | -3.46% | 1.46% |
1.50% | 1 | 5 |
| TPCH(60) | TPCH-Q12 |
parquet / none / none | 6.98 | 7.31 | -4.57% | 1.03% |
7.13% | 1 | 5 |
| TPCH(60) | TPCH-Q18 |
parquet / none / none | 47.54 | 50.39 | -5.64% | 5.70% |
5.53% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_bigint_non_selective |
parquet / none / none | 0.88 | 0.96 | -7.81% | 4.27% |
5.97% | 1 | 5 |
| TPCH(60) | TPCH-Q15 |
parquet / none / none | 8.14 | 9.15 | -11.09% | 0.63% | *
10.44% * | 1 | 5 |
| TPCH(60) | TPCH-Q10 |
parquet / none / none | 12.66 | 14.28 | -11.34% | 4.32% |
1.14% | 1 | 5 |
| TPCH(60) | TPCH-Q17 |
parquet / none / none | 10.31 | 12.59 | -18.14% | 0.65% |
3.72% | 1 | 5 |
| TARGETED-PERF(60) | primitive_filter_bigint_selective |
parquet / none / none | 0.14 | 0.19 | I -27.60% | * 32.55% * | *
39.78% * | 1 | 5 |
| TPCH(60) | TPCH-Q14 |
parquet / none / none | 6.10 | 11.00 | I -44.55% | 4.06% |
3.84% | 1 | 5 |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
Change-Id: Ib0a15cb53feab89e7b35a56b67b3b30eb3e62c6b
---
M be/CMakeLists.txt
M be/src/benchmarks/row-batch-serialize-benchmark.cc
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/instruction-counter-test.cc
M be/src/codegen/llvm-codegen.cc
M be/src/exprs/aggregate-functions-ir.cc
M be/src/rpc/impala-service-pool.h
M be/src/runtime/bufferpool/buffer-pool-test.cc
M be/src/runtime/data-stream-sender.cc
M be/src/runtime/data-stream-sender.h
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/krpc-data-stream-sender.h
M be/src/runtime/row-batch-serialize-test.cc
M be/src/runtime/row-batch.cc
M be/src/runtime/row-batch.h
M be/src/runtime/types.cc
M be/src/service/fe-support.cc
M be/src/thirdparty/squeasel/squeasel.c
M be/src/transport/TSasl.cpp
M be/src/transport/TSaslClientTransport.cpp
M be/src/transport/TSaslTransport.cpp
M be/src/udf/udf-test.cc
M be/src/udf/udf.h
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/workloads/functional-query/queries/QueryTest/udf-errors.test
26 files changed, 131 insertions(+), 123 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/9584/5
--
To view, visit http://gerrit.cloudera.org:8080/9584
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib0a15cb53feab89e7b35a56b67b3b30eb3e62c6b
Gerrit-Change-Number: 9584
Gerrit-PatchSet: 5
Gerrit-Owner: Bikramjeet Vig <bikramjeet....@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
