Tim Armstrong has posted comments on this change.
Change subject: Inject precision and scale constants for all decimal builtins
......................................................................
Patch Set 3:
Perf results look good, there are no regressions and a lot of improvements.
Report Generated on 2016-03-15
Run Description: "Base: 274f20d4b28f5fc71ddb5c79850d3e6741a7105b vs
Ref: e5f009867baa336e4e4fad6b48494c39004bb84c"
Cluster Name: UNKNOWN
Lab Run Info: UNKNOWN
Impala Version: impalad version 2.5.0-cdh5.7.0 RELEASE
(2016-02-24)
Baseline Impala Version: impalad version 2.5.0-cdh5.7.0 RELEASE
(2016-02-24)
+--------------------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) |
GeoMean(s) | Delta(GeoMean) |
+--------------------+-----------------------+---------+------------+------------+----------------+
| TARGETED-PERF(_20) | parquet / none / none | 10.10 | -2.07% |
3.54 | -3.55% |
+--------------------+-----------------------+---------+------------+------------+----------------+
+--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| Workload | Query
| File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) |
Base StdDev(%) | Num Clients | Iters |
+--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| TARGETED-PERF(_20) | PERF_STRING-Q7
| parquet / none / none | 3.87 | 3.73 | +3.74% | 1.51% |
1.78% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_AGG-Q2
| parquet / none / none | 3.42 | 3.36 | +1.95% | 5.27% |
1.59% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_topn_bigint
| parquet / none / none | 6.91 | 6.80 | +1.64% | 2.92% |
0.74% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_AGG-Q6
| parquet / none / none | 1.39 | 1.38 | +0.58% | 1.15% |
1.53% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_shuffle_join_union_all_with_groupby
| parquet / none / none | 20.45 | 20.34 | +0.55% | 0.84% |
0.62% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_STRING-Q1
| parquet / none / none | 1.64 | 1.64 | +0.48% | 1.76% |
1.57% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_AGG-Q7
| parquet / none / none | 1.39 | 1.39 | +0.30% | 0.88% |
2.22% | 1 | 10 |
| TARGETED-PERF(_20) |
primitive_shuffle_join_one_to_many_string_with_groupby | parquet / none / none
| 97.85 | 97.80 | +0.05% | 0.70% | 0.52% | 1
| 10 |
| TARGETED-PERF(_20) | primitive_orderby_bigint
| parquet / none / none | 3.20 | 3.20 | +0.01% | 2.12% |
2.42% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_AGG-Q3
| parquet / none / none | 5.29 | 5.29 | -0.02% | 4.37% |
0.70% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_STRING-Q6
| parquet / none / none | 4.74 | 4.75 | -0.09% | 1.43% |
1.72% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_orderby_all
| parquet / none / none | 13.56 | 13.58 | -0.17% | 1.33% |
1.10% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_exchange_shuffle
| parquet / none / none | 36.32 | 36.43 | -0.30% | 0.90% |
1.37% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_top-n_all
| parquet / none / none | 40.83 | 41.04 | -0.50% | 2.46% |
1.64% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_filter_bigint_non_selective
| parquet / none / none | 1.07 | 1.08 | -0.62% | 8.78% |
0.07% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_STRING-Q3
| parquet / none / none | 1.86 | 1.88 | -0.77% | 2.41% |
1.69% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_exchange_broadcast
| parquet / none / none | 20.91 | 21.10 | -0.88% | 1.76% |
2.10% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_filter_string_selective
| parquet / none / none | 1.44 | 1.45 | -1.17% | 4.89% |
4.90% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_AGG-Q5
| parquet / none / none | 2.93 | 2.97 | -1.26% | 1.64% |
2.20% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_filter_string_like
| parquet / none / none | 6.41 | 6.51 | -1.60% | 0.88% |
1.25% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_groupby_bigint_pk
| parquet / none / none | 41.12 | 41.84 | -1.72% | 0.64% |
1.19% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_broadcast_join_3
| parquet / none / none | 3.90 | 3.98 | -2.08% | 1.47% |
2.61% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_groupby_bigint_lowndv
| parquet / none / none | 2.04 | 2.10 | -2.64% | 1.67% |
5.41% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_groupby_decimal_lowndv.test
| parquet / none / none | 2.05 | 2.11 | -2.91% | 2.24% |
3.17% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_LIMIT-Q1
| parquet / none / none | 0.01 | 0.01 | -2.94% | 3.11% |
5.39% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_groupby_bigint_highndv
| parquet / none / none | 11.11 | 11.48 | -3.25% | 1.39% |
1.29% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_STRING-Q4
| parquet / none / none | 1.89 | 1.96 | -3.28% | 1.53% |
1.30% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_filter_string_non_selective
| parquet / none / none | 1.57 | 1.64 | -3.98% | 4.82% |
5.64% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_broadcast_join_2
| parquet / none / none | 2.36 | 2.47 | -4.22% | 0.95% |
2.27% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_filter_bigint_selective
| parquet / none / none | 0.93 | 0.98 | -4.73% | 0.21% |
0.13% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_broadcast_join_1
| parquet / none / none | 1.26 | 1.32 | -5.18% | 2.37% |
2.35% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_STRING-Q5
| parquet / none / none | 2.09 | 2.22 | -5.89% | 1.16% |
1.88% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_STRING-Q2
| parquet / none / none | 1.73 | 1.84 | -6.03% | 1.05% |
2.07% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_AGG-Q1
| parquet / none / none | 1.70 | 1.81 | -6.17% | 0.36% |
1.78% | 1 | 10 |
| TARGETED-PERF(_20) | PERF_AGG-Q4
| parquet / none / none | 13.68 | 14.69 | -6.86% | 2.70% |
3.26% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_groupby_decimal_highndv
| parquet / none / none | 18.37 | 22.48 | -18.26% | 6.83% |
1.08% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_filter_decimal_non_selective
| parquet / none / none | 1.30 | 1.71 | I -23.59% | 1.75% |
2.14% | 1 | 10 |
| TARGETED-PERF(_20) | primitive_filter_decimal_selective
| parquet / none / none | 1.16 | 1.55 | I -24.68% | 2.24% |
1.37% | 1 | 10 |
+--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
(I) Improvement: TARGETED-PERF(_20)
primitive_filter_decimal_non_selective [parquet / none / none] (1.71s -> 1.30s
[-23.59%])
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+---------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+---------+-----------+
| 01:AGGREGATE | 29.03% | 344.76ms | 346.51ms | -0.50% |
1.76% | 353.73ms | 375.53ms | -5.81% | 1 | 1 | 1 |
| 00:SCAN HDFS | 64.73% | 768.78ms | 1.13s | -31.70% |
1.55% | 788.14ms | 1.16s | -31.88% | 1 | 119.99M | 12.00M |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+---------+-----------+
(I) Improvement: TARGETED-PERF(_20) primitive_filter_decimal_selective
[parquet / none / none] (1.55s -> 1.16s [-24.68%])
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+-----------+
| 00:SCAN HDFS | 85.43% | 898.16ms | 1.24s | -27.33% |
2.30% | 932.27ms | 1.28s | -26.90% | 1 | 113 | 12.00M |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+-------+-----------+
Significant perf change detected
Report Generated on 2016-03-15
Run Description: "Base: 274f20d4b28f5fc71ddb5c79850d3e6741a7105b vs
Ref: e5f009867baa336e4e4fad6b48494c39004bb84c"
Cluster Name: UNKNOWN
Lab Run Info: UNKNOWN
Impala Version: impalad version 2.5.0-cdh5.7.0 RELEASE
(2016-02-24)
Baseline Impala Version: impalad version 2.5.0-cdh5.7.0 RELEASE
(2016-02-24)
+-----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s)
| Delta(GeoMean) |
+-----------+-----------------------+---------+------------+------------+----------------+
| TPCH(_20) | parquet / none / none | 11.75 | -5.41% | 8.26
| -3.59% |
+-----------+-----------------------+---------+------------+------------+----------------+
+-----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) |
Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients | Iters |
+-----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| TPCH(_20) | TPCH-Q13 | parquet / none / none | 10.76 | 10.20 |
+5.48% | 2.54% | 1.32% | 1 | 10 |
| TPCH(_20) | TPCH-Q11 | parquet / none / none | 2.55 | 2.47 |
+3.50% | 4.85% | 3.36% | 1 | 10 |
| TPCH(_20) | TPCH-Q17 | parquet / none / none | 22.17 | 21.70 |
+2.18% | 0.89% | 0.52% | 1 | 10 |
| TPCH(_20) | TPCH-Q20 | parquet / none / none | 7.38 | 7.22 |
+2.17% | 2.18% | 1.81% | 1 | 10 |
| TPCH(_20) | TPCH-Q7 | parquet / none / none | 18.06 | 17.74 |
+1.80% | 2.22% | 1.05% | 1 | 10 |
| TPCH(_20) | TPCH-Q5 | parquet / none / none | 11.44 | 11.24 |
+1.74% | 2.78% | 2.18% | 1 | 10 |
| TPCH(_20) | TPCH-Q3 | parquet / none / none | 8.08 | 7.99 |
+1.14% | 2.11% | 1.70% | 1 | 10 |
| TPCH(_20) | TPCH-Q12 | parquet / none / none | 4.66 | 4.62 |
+0.98% | 0.80% | 2.22% | 1 | 10 |
| TPCH(_20) | TPCH-Q21 | parquet / none / none | 35.94 | 35.66 |
+0.79% | 1.68% | 1.16% | 1 | 10 |
| TPCH(_20) | TPCH-Q9 | parquet / none / none | 14.83 | 14.85 |
-0.13% | 0.89% | 1.20% | 1 | 10 |
| TPCH(_20) | TPCH-Q2 | parquet / none / none | 3.23 | 3.24 |
-0.44% | 1.88% | 2.90% | 1 | 10 |
| TPCH(_20) | TPCH-Q10 | parquet / none / none | 6.40 | 6.48 |
-1.33% | 2.68% | 1.93% | 1 | 10 |
| TPCH(_20) | TPCH-Q16 | parquet / none / none | 2.13 | 2.16 |
-1.68% | 2.48% | 1.61% | 1 | 10 |
| TPCH(_20) | TPCH-Q22 | parquet / none / none | 3.14 | 3.24 |
-2.83% | 1.25% | 1.45% | 1 | 10 |
| TPCH(_20) | TPCH-Q15 | parquet / none / none | 6.52 | 6.72 |
-2.95% | 3.06% | 2.07% | 1 | 10 |
| TPCH(_20) | TPCH-Q14 | parquet / none / none | 4.13 | 4.27 |
-3.11% | 2.10% | 1.65% | 1 | 10 |
| TPCH(_20) | TPCH-Q8 | parquet / none / none | 6.46 | 6.75 |
-4.27% | 1.15% | 1.79% | 1 | 10 |
| TPCH(_20) | TPCH-Q4 | parquet / none / none | 12.74 | 13.43 |
-5.10% | 2.66% | 3.13% | 1 | 10 |
| TPCH(_20) | TPCH-Q18 | parquet / none / none | 15.68 | 16.63 |
-5.75% | 3.02% | 3.84% | 1 | 10 |
| TPCH(_20) | TPCH-Q6 | parquet / none / none | 3.27 | 3.52 |
-7.21% | 4.19% | 3.57% | 1 | 10 |
| TPCH(_20) | TPCH-Q19 | parquet / none / none | 48.10 | 55.31 |
-13.02% | 2.25% | 1.59% | 1 | 10 |
| TPCH(_20) | TPCH-Q1 | parquet / none / none | 10.76 | 17.78 |
I -39.49% | 2.07% | 1.63% | 1 | 10 |
+-----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
(I) Improvement: TPCH(_20) TPCH-Q1 [parquet / none / none] (17.78s ->
10.76s [-39.49%])
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+---------+-----------+
| Operator | % of Query | Avg | Base Avg | Delta(Avg) |
StdDev(%) | Max | Base Max | Delta(Max) | #Hosts | #Rows | Est #Rows |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+---------+-----------+
| 01:AGGREGATE | 96.99% | 10.28s | 17.10s | -39.90% |
1.93% | 10.75s | 17.70s | -39.28% | 1 | 4 | 6 |
| 00:SCAN HDFS | 2.24% | 237.54ms | 227.02ms | +4.64% |
7.65% | 280.05ms | 245.34ms | +14.15% | 1 | 118.31M | 12.00M |
+--------------+------------+----------+----------+------------+-----------+----------+----------+------------+--------+---------+-----------+
Significant perf change detected
Report Generated on 2016-03-15
Run Description: "Base: 274f20d4b28f5fc71ddb5c79850d3e6741a7105b vs
Ref: e5f009867baa336e4e4fad6b48494c39004bb84c"
Cluster Name: UNKNOWN
Lab Run Info: UNKNOWN
Impala Version: impalad version 2.5.0-cdh5.7.0 RELEASE
(2016-02-24)
Baseline Impala Version: impalad version 2.5.0-cdh5.7.0 RELEASE
(2016-02-24)
+-------------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) |
GeoMean(s) | Delta(GeoMean) |
+-------------+-----------------------+---------+------------+------------+----------------+
| TPCDS(_100) | parquet / none / none | 6.24 | -1.68% | 1.43
| -0.80% |
+-------------+-----------------------+---------+------------+------------+----------------+
+-------------+------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| Workload | Query | File Format
| Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Num
Clients | Iters |
+-------------+------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| TPCDS(_100) | TPCDS-Q19 | parquet / none /
none | 0.97 | 0.93 | +4.07% | 3.77% | 2.95% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-PROMOTION | parquet / none /
none | 0.28 | 0.28 | +2.12% | 0.72% | 0.46% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-CUSTOMER | parquet / none /
none | 0.28 | 0.27 | +1.87% | 0.69% | 0.28% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-HOUSEHOLD_DEMOGRAPHICS | parquet / none /
none | 0.28 | 0.28 | +1.70% | 0.59% | 0.20% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-ITEM | parquet / none /
none | 0.28 | 0.28 | +1.60% | 0.61% | 0.17% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-TIME_DIM | parquet / none /
none | 0.28 | 0.28 | +1.60% | 0.40% | 0.41% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-CUSTOMER_ADDRESS | parquet / none /
none | 0.28 | 0.28 | +1.50% | 0.48% | 0.50% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-DATE_DIM | parquet / none /
none | 0.28 | 0.28 | +1.50% | 0.23% | 0.45% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-STORE | parquet / none /
none | 0.28 | 0.28 | +1.41% | 0.24% | 0.45% | 1
| 10 |
| TPCDS(_100) | TPCDS-COUNT-CUSTOMER_DEMOGRAPHICS | parquet / none /
none | 0.28 | 0.28 | +1.16% | 0.50% | 0.41% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q79 | parquet / none /
none | 1.66 | 1.64 | +1.10% | 2.03% | 3.41% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q89 | parquet / none /
none | 2.04 | 2.03 | +0.78% | 2.17% | 2.82% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q65 | parquet / none /
none | 24.15 | 24.00 | +0.63% | 1.08% | 1.34% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q55 | parquet / none /
none | 0.42 | 0.42 | +0.33% | 3.87% | 2.72% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q6 | parquet / none /
none | 56.13 | 56.05 | +0.15% | 1.74% | 0.53% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q8 | parquet / none /
none | 1.19 | 1.19 | +0.10% | 3.33% | 2.75% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q42 | parquet / none /
none | 0.42 | 0.42 | -0.23% | 0.15% | 0.38% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q47 | parquet / none /
none | 33.62 | 33.71 | -0.27% | 1.65% | 2.06% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q68 | parquet / none /
none | 1.26 | 1.27 | -0.38% | 3.93% | 2.55% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q52 | parquet / none /
none | 0.42 | 0.43 | -0.55% | 3.67% | 4.16% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q34 | parquet / none /
none | 1.61 | 1.62 | -0.64% | 1.97% | 2.22% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q27 | parquet / none /
none | 2.93 | 2.95 | -0.66% | 3.04% | 3.68% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q7 | parquet / none /
none | 3.13 | 3.16 | -1.06% | 0.87% | 1.25% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q98 | parquet / none /
none | 0.84 | 0.86 | -2.08% | 2.88% | 3.89% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q3 | parquet / none /
none | 0.93 | 0.95 | -2.23% | 2.55% | 4.44% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q63 | parquet / none /
none | 1.58 | 1.61 | -2.26% | 2.01% | 2.57% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q73 | parquet / none /
none | 0.72 | 0.74 | -2.27% | 2.21% | 4.25% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q96 | parquet / none /
none | 5.05 | 5.22 | -3.27% | 0.95% | 0.77% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q46 | parquet / none /
none | 2.83 | 2.93 | -3.27% | 1.09% | 1.17% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q53 | parquet / none /
none | 1.56 | 1.61 | -3.36% | 1.58% | 1.52% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q88 | parquet / none /
none | 42.31 | 44.05 | -3.94% | 1.25% | 1.36% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q43 | parquet / none /
none | 3.24 | 3.39 | -4.60% | 2.05% | 2.04% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q59 | parquet / none /
none | 19.19 | 20.54 | -6.56% | 1.46% | 0.94% | 1
| 10 |
| TPCDS(_100) | TPCDS-Q61 | parquet / none /
none | 1.54 | 1.71 | -10.06% | 2.49% | 6.60% | 1
| 10 |
+-------------+------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
--
To view, visit http://gerrit.cloudera.org:8080/2535
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7b7d1485f357ba88517ca15c5c7428c0ffe25dfd
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Dan Hecht <[email protected]>
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-HasComments: No
