-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/489/
-----------------------------------------------------------
(Updated 2011-03-16 16:50:15.347839)
Review request for hive.
Changes
-------
Add more changes to suite the current implementation of JDO filtering. Some
major changes are:
- ExpressionTree.makeFilterForEquals(): previous '=' are translated to JDO
method matches(). This will introduce false positives if the partition value
contains regex special characters (e.g., dot). I changed this function to use
startWith(), endsWith(), and indexOf() in the case of whether the partition
column is at the beginning, end or middle of the partition spec string. Two
unit tests files (ppr_pushdown*.q) are added to test these cases.
- ObjectStore.listMPartitions(): added a query.setOrder() to return
partitions ordered by their partition names. This is to be backward compatible
with the old partition pruning behavior.
- PartitionPruner.prune(): check if the partition pruning expression contains
non-partition columns. If so add the resulting partitions to unkn_parts,
otherwise to true_parts. This is required by down stream optimizations.
- Utilities.checkJDOPushDown(): return true only if the partition column type
is string and constant type is string. This is required by the current
implementation of JDO filter (see ExpressionTree.java and Filter.g).
Passed all unit tests.
Summary
-------
* expose HiveMetaStoreClient.listPartitionsByFilter() to Hive.java so that
PartitionPruner can use that for certain partition predicates.
* only allows {=, AND, OR} in the partition predicates that can be pushed
down to JDO filtering.
Diffs (updated)
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
1081948
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
1081948
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java
1081948
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1081948
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1081948
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java
1081948
trunk/ql/src/test/queries/clientpositive/ppr_pushdown.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ppr_pushdown2.q PRE-CREATION
trunk/ql/src/test/results/clientpositive/ppr_pushdown.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ppr_pushdown2.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/489/diff
Testing
-------
Thanks,
Ning
