Hello Vincent Tran, Impala Public Jenkins, Sailesh Mukil,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7521
to look at the new patch set (#3).
Change subject: IMPALA-5336: Fix partition pruning when column is cast
......................................................................
IMPALA-5336: Fix partition pruning when column is cast
Partition pruning has two mechanisms:
1) Simple predicates (e.g. binary predicates of the form
<SlotRef> <op> <LiteralExpr>) can be used to derive lists
of matching partition ids directly from the
partition key values. This is handled directly in the FE
and is very efficient for supported simple predicates.
2) General expr evaluation of predicates using the BE (via
FeSupport). This works for all predicates, so is the
mechanism used for predicates not supported by (1).
The issue was that (1) was being used when a binary
predicate contained an implicit cast on the SlotRef. While
this is OK when being evaluated by the BE, the simple
mechanism in (1) would not be able to match the partition
key values with the predicate literal because the partition
key values cannot be cast in the FE.
The fix is to force binary predicates involving a cast to be
evaluated in the BE.
Testing: A planner test was added to demonstrate the
expected partition pruning occurs.
Some modifications were made to the functional schema table
stringpartitionkey, so it will be necessary to reload those
tables:
load-data.py -w functional-query --table_names=stringpartitionkey
Change-Id: I94f597a6589f5e34d2b74abcd29be77c4161cd99
---
M fe/src/main/java/org/apache/impala/planner/HdfsPartitionPruner.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/bin/compute-table-stats.sh
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
A
testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test
6 files changed, 35 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/7521/3
--
To view, visit http://gerrit.cloudera.org:8080/7521
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I94f597a6589f5e34d2b74abcd29be77c4161cd99
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Matthew Jacobs <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Matthew Jacobs <[email protected]>
Gerrit-Reviewer: Sailesh Mukil <[email protected]>
Gerrit-Reviewer: Vincent Tran <[email protected]>
