Michael Ho has posted comments on this change.
Change subject: IMPALA-3115: Hoist some variables out of loops in
cross-compiled code.
......................................................................
Patch Set 2:
(3 comments)
+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) |
Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(15) | parquet / none / none | 15.82 | -0.44% | 9.84 | -0.47%
|
+----------+-----------------------+---------+------------+------------+----------------+
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) |
Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients | Iters |
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| TPCH(15) | TPCH-Q12 | parquet / none / none | 3.63 | 3.57 | +1.73%
| 1.40% | 1.17% | 1 | 5 |
| TPCH(15) | TPCH-Q6 | parquet / none / none | 2.55 | 2.54 | +0.47%
| 2.93% | 3.15% | 1 | 5 |
| TPCH(15) | TPCH-Q22 | parquet / none / none | 5.89 | 5.88 | +0.29%
| 1.53% | 0.71% | 1 | 5 |
| TPCH(15) | TPCH-Q7 | parquet / none / none | 28.10 | 28.02 | +0.28%
| 1.92% | 0.69% | 1 | 5 |
| TPCH(15) | TPCH-Q10 | parquet / none / none | 6.85 | 6.84 | +0.18%
| 0.89% | 0.93% | 1 | 5 |
| TPCH(15) | TPCH-Q11 | parquet / none / none | 1.86 | 1.86 | +0.12%
| 1.27% | 2.54% | 1 | 5 |
| TPCH(15) | TPCH-Q17 | parquet / none / none | 30.68 | 30.67 | +0.01%
| 0.56% | 0.56% | 1 | 5 |
| TPCH(15) | TPCH-Q9 | parquet / none / none | 40.85 | 40.85 | -0.00%
| 0.92% | 0.79% | 1 | 5 |
| TPCH(15) | TPCH-Q5 | parquet / none / none | 19.37 | 19.42 | -0.23%
| 0.65% | 0.65% | 1 | 5 |
| TPCH(15) | TPCH-Q14 | parquet / none / none | 3.31 | 3.32 | -0.37%
| 2.45% | 3.42% | 1 | 5 |
| TPCH(15) | TPCH-Q19 | parquet / none / none | 37.52 | 37.67 | -0.40%
| 1.17% | 1.93% | 1 | 5 |
| TPCH(15) | TPCH-Q21 | parquet / none / none | 32.12 | 32.28 | -0.50%
| 4.91% | 2.99% | 1 | 5 |
| TPCH(15) | TPCH-Q13 | parquet / none / none | 15.59 | 15.67 | -0.54%
| 0.39% | 0.14% | 1 | 5 |
| TPCH(15) | TPCH-Q18 | parquet / none / none | 35.04 | 35.26 | -0.61%
| 0.83% | 0.71% | 1 | 5 |
| TPCH(15) | TPCH-Q2 | parquet / none / none | 2.68 | 2.70 | -0.93%
| 1.64% | 2.03% | 1 | 5 |
| TPCH(15) | TPCH-Q15 | parquet / none / none | 5.04 | 5.09 | -0.97%
| 1.30% | 3.00% | 1 | 5 |
| TPCH(15) | TPCH-Q8 | parquet / none / none | 34.95 | 35.32 | -1.05%
| 0.70% | 0.41% | 1 | 5 |
| TPCH(15) | TPCH-Q3 | parquet / none / none | 12.05 | 12.19 | -1.14%
| 0.36% | 0.80% | 1 | 5 |
| TPCH(15) | TPCH-Q4 | parquet / none / none | 14.08 | 14.24 | -1.14%
| 0.37% | 0.93% | 1 | 5 |
| TPCH(15) | TPCH-Q20 | parquet / none / none | 6.55 | 6.65 | -1.50%
| 1.38% | 1.11% | 1 | 5 |
| TPCH(15) | TPCH-Q16 | parquet / none / none | 1.68 | 1.71 | -1.55%
| 2.15% | 1.39% | 1 | 5 |
| TPCH(15) | TPCH-Q1 | parquet / none / none | 7.73 | 7.92 | -2.39%
| 0.73% | 1.37% | 1 | 5 |
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
+-------------------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s)
| Delta(GeoMean) |
+-------------------+-----------------------+---------+------------+------------+----------------+
| TARGETED-PERF(15) | parquet / none / none | 7.68 | -1.10% | 2.94
| -1.13% |
+-------------------+-----------------------+---------+------------+------------+----------------+
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| Workload | Query |
File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base
StdDev(%) | Num Clients | Iters |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| TARGETED-PERF(15) | PERF_STRING-Q5 |
parquet / none / none | 2.05 | 2.00 | +2.92% | 3.64% | 1.08%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_filter_bigint_non_selective |
parquet / none / none | 1.08 | 1.05 | +2.71% | 2.04% | 1.93%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_STRING-Q1 |
parquet / none / none | 1.61 | 1.58 | +1.85% | 2.22% | 2.78%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_STRING-Q7 |
parquet / none / none | 3.06 | 3.01 | +1.66% | 2.44% | 1.69%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_orderby_all |
parquet / none / none | 10.42 | 10.29 | +1.21% | 1.46% | 0.83%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_AGG-Q1 |
parquet / none / none | 1.42 | 1.40 | +1.11% | 4.51% | 1.70%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_AGG-Q7 |
parquet / none / none | 1.19 | 1.17 | +0.99% | 4.28% | 2.77%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_shuffle_join_one_to_many_string_with_groupby |
parquet / none / none | 73.02 | 72.31 | +0.98% | 0.18% | 0.46%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_STRING-Q3 |
parquet / none / none | 1.80 | 1.79 | +0.89% | 2.38% | 1.39%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_broadcast_join_3 |
parquet / none / none | 3.24 | 3.21 | +0.85% | 1.26% | 2.06%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_LIMIT-Q1 |
parquet / none / none | 0.01 | 0.01 | +0.45% | 3.89% | 1.12%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_STRING-Q2 |
parquet / none / none | 1.71 | 1.71 | +0.33% | 2.03% | 3.58%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_broadcast_join_2 |
parquet / none / none | 2.08 | 2.08 | +0.16% | 2.03% | 1.06%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_filter_decimal_selective |
parquet / none / none | 1.17 | 1.17 | +0.14% | 2.11% | 2.14%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_broadcast_join_1 |
parquet / none / none | 1.11 | 1.11 | +0.13% | 3.80% | 5.04%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_filter_string_non_selective |
parquet / none / none | 1.49 | 1.49 | -0.07% | 4.10% | 9.80%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_exchange_broadcast |
parquet / none / none | 14.81 | 14.83 | -0.09% | 0.62% | 0.69%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_orderby_bigint |
parquet / none / none | 2.50 | 2.51 | -0.21% | 3.08% | 3.73%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_AGG-Q5 |
parquet / none / none | 2.45 | 2.47 | -0.59% | 2.45% | 2.20%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_groupby_bigint_pk |
parquet / none / none | 29.22 | 29.44 | -0.75% | 0.87% | 1.05%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_STRING-Q4 |
parquet / none / none | 1.79 | 1.81 | -0.94% | 3.16% | 3.40%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_filter_bigint_selective |
parquet / none / none | 0.99 | 1.00 | -0.94% | 3.73% | 2.13%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_filter_string_selective |
parquet / none / none | 1.17 | 1.19 | -1.04% | 3.63% | 6.77%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_AGG-Q6 |
parquet / none / none | 1.12 | 1.14 | -1.11% | 3.77% | 0.11%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_filter_string_like |
parquet / none / none | 5.18 | 5.24 | -1.15% | 1.04% | 1.46%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_topn_bigint |
parquet / none / none | 5.51 | 5.57 | -1.15% | 6.09% | 9.13%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_top-n_all |
parquet / none / none | 31.54 | 32.08 | -1.68% | 0.82% | 0.64%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_shuffle_join_union_all_with_groupby |
parquet / none / none | 19.42 | 19.77 | -1.77% | 0.66% | 0.61%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_filter_decimal_non_selective |
parquet / none / none | 1.23 | 1.25 | -1.88% | 1.89% | 1.70%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_groupby_bigint_highndv |
parquet / none / none | 8.70 | 8.95 | -2.80% | 0.92% | 0.40%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_STRING-Q6 |
parquet / none / none | 3.71 | 3.82 | -2.96% | 1.16% | 3.03%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_groupby_decimal_highndv |
parquet / none / none | 14.92 | 15.40 | -3.09% | 0.28% | 0.74%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_AGG-Q3 |
parquet / none / none | 3.86 | 4.00 | -3.45% | 0.56% | 0.62%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_exchange_shuffle |
parquet / none / none | 25.77 | 26.70 | -3.49% | 0.29% | 0.26%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_AGG-Q2 |
parquet / none / none | 2.44 | 2.56 | -4.86% | 1.02% | 0.02%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_groupby_bigint_lowndv |
parquet / none / none | 1.49 | 1.59 | -6.26% | 3.68% | 2.74%
| 1 | 5 |
| TARGETED-PERF(15) | primitive_groupby_decimal_lowndv.test |
parquet / none / none | 1.44 | 1.55 | -7.27% | 3.01% | 1.41%
| 1 | 5 |
| TARGETED-PERF(15) | PERF_AGG-Q4 |
parquet / none / none | 6.24 | 6.98 | -10.63% | 7.27% | 5.67%
| 1 | 5 |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
http://gerrit.cloudera.org:8080/#/c/2661/2/be/src/exec/partitioned-aggregation-node-ir.cc
File be/src/exec/partitioned-aggregation-node-ir.cc:
Line 195: const HashTableCtx* __restrict__ ht_ctx, Partition* __restrict__
partition,
> I think these restrict qualifiers may be discarded when TryAddToHashTable i
Yes. I verified the IR is the same without the qualifier.
http://gerrit.cloudera.org:8080/#/c/2661/2/be/src/exec/partitioned-hash-join-node-ir.cc
File be/src/exec/partitioned-hash-join-node-ir.cc:
Line 274: buildStatus_
> while you're here, make it build_status_?
Done
http://gerrit.cloudera.org:8080/#/c/2661/2/be/src/runtime/row-batch.h
File be/src/runtime/row-batch.h:
Line 145: /// For row batches with zero capacity, Next() will always return
NULL.
> Maybe we should just DCHECK if Next() is called on an empty batch? The code
DCHECK_LT(row_idx, capacity) was actually triggered in GetRow() during testing
but I couldn't recover a proper backtrace as I couldn't find the binary in the
AWS VM. The DCHECK message suggested that both num_rows_ and capacity_ are zero
so that seems to be a legitimate case.
--
To view, visit http://gerrit.cloudera.org:8080/2661
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7152b1fb094b3c3574d203e3774f4297f2225dc
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Ho <[email protected]>
Gerrit-Reviewer: Michael Ho <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-HasComments: Yes
