Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/16073 )
Change subject: IMPALA-7020: fix costing of non-trivial CAST expressions
......................................................................
IMPALA-7020: fix costing of non-trivial CAST expressions
Some cast operations are quite expensive to evaluate,
which was not reflected in the uniform costing of CAST
expresions.
We fix this by increasing the cost of non-trivial
casts to be the same as an arbitrary function call.
Testing:
Ran exhaustive tests.
Add planner tests to check that CAST expressions are
materialized or not based on the input and output
types - the planner output lists 'materialized:'
expressions for the SORT operator.
A few existing planner tests had changes in predicate
ordering. I checked manually that these changes made
sense.
Perf:
I sanity-checked that this actually helped (a variant of)
the example query from IMPALA-7020. The following query
went from ~8s to ~2s in my dev environment:
select *
FROM
(
SELECT
o.*,
ROW_NUMBER() OVER(ORDER BY evt_ts DESC) AS rn
FROM
(
SELECT
l_orderkey,l_partkey,l_linenumber,l_quantity, cast (l_shipdate as
date) evt_ts
FROM
tpch_parquet.lineitem
) o
) r
WHERE
rn BETWEEN 1 AND 101
ORDER BY rn;
Change-Id: I3f1a16fc45191a2eedf38cc243c70173d44806c6
Reviewed-on: http://gerrit.cloudera.org:8080/16073
Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M
testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test
M
testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test
M
testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
M
testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test
6 files changed, 212 insertions(+), 10 deletions(-)
Approvals:
Tim Armstrong: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/16073
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3f1a16fc45191a2eedf38cc243c70173d44806c6
Gerrit-Change-Number: 16073
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Shant Hovsepian <sh...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
