[
https://issues.apache.org/jira/browse/DRILL-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venki Korukanti updated DRILL-3746:
-----------------------------------
Description:
If Hive contains a table which has external partitions, Drill fails in
partition pruning code, which causes the query to fail.
{code}
CREATE TABLE external_partition_test (boolean_field BOOLEAN) PARTITIONED BY
(boolean_part BOOLEAN);
ALTER TABLE external_partition_test ADD PARTITION (boolean_part='true')
LOCATION '/some/path';
ALTER TABLE external_partition_test ADD PARTITION (boolean_part='false')
LOCATION '/some/path';
{code}
Query:
{code}
SELECT * FROM hive.`default`.external_partition_test where boolean_part = false
{code}
Exception:
{code}
java.lang.StringIndexOutOfBoundsException
String index out of range: -14
at java.lang.String.substring(String.java:1875) ~[na:1.7.0_45]
at
org.apache.drill.exec.planner.sql.HivePartitionLocation.<init>(HivePartitionLocation.java:31)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.HivePartitionDescriptor.getPartitions(HivePartitionDescriptor.java:117)
~[classes/:na]
at
org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:185)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.logical.HivePushPartitionFilterIntoScan$2.onMatch(HivePushPartitionFilterIntoScan.java:92)
~[classes/:na]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
~[calcite-core-1.4.0-drill-r0.jar:1.4.0-drill-r0]
{code}
Looking at {{HivePartitionLocation}}, it looks like we are depending on the
organization of files on FileSystem to get the partition values. We should get
the partition values from MetaStore.
was:
If Hive contains a table which has external partitions, Drill fails in
partition pruning code, which causes the query to fail.
{code}
CREATE TABLE external_partition_test (boolean_field BOOLEAN) PARTITIONED BY
(boolean_part BOOLEAN);
ALTER TABLE external_partition_test ADD PARTITION (boolean_part='true')
LOCATION '/some/path';
ALTER TABLE external_partition_test ADD PARTITION (boolean_part='false')
LOCATION '/some/path';
{code}
Exception:
{code}
java.lang.StringIndexOutOfBoundsException
String index out of range: -14
at java.lang.String.substring(String.java:1875) ~[na:1.7.0_45]
at
org.apache.drill.exec.planner.sql.HivePartitionLocation.<init>(HivePartitionLocation.java:31)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.HivePartitionDescriptor.getPartitions(HivePartitionDescriptor.java:117)
~[classes/:na]
at
org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:185)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.logical.HivePushPartitionFilterIntoScan$2.onMatch(HivePushPartitionFilterIntoScan.java:92)
~[classes/:na]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
~[calcite-core-1.4.0-drill-r0.jar:1.4.0-drill-r0]
{code}
Looking at the {{HivePartitionLocation}}, it looks like we are depending on the
structure of the files on FileSystem to get the partition values. We should get
these partition from MetaStore.
> Hive query fails if the table contains external partitions
> ----------------------------------------------------------
>
> Key: DRILL-3746
> URL: https://issues.apache.org/jira/browse/DRILL-3746
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Venki Korukanti
>
> If Hive contains a table which has external partitions, Drill fails in
> partition pruning code, which causes the query to fail.
> {code}
> CREATE TABLE external_partition_test (boolean_field BOOLEAN) PARTITIONED BY
> (boolean_part BOOLEAN);
> ALTER TABLE external_partition_test ADD PARTITION (boolean_part='true')
> LOCATION '/some/path';
> ALTER TABLE external_partition_test ADD PARTITION (boolean_part='false')
> LOCATION '/some/path';
> {code}
> Query:
> {code}
> SELECT * FROM hive.`default`.external_partition_test where boolean_part =
> false
> {code}
> Exception:
> {code}
> java.lang.StringIndexOutOfBoundsException
> String index out of range: -14
> at java.lang.String.substring(String.java:1875) ~[na:1.7.0_45]
> at
> org.apache.drill.exec.planner.sql.HivePartitionLocation.<init>(HivePartitionLocation.java:31)
> ~[classes/:na]
> at
> org.apache.drill.exec.planner.sql.HivePartitionDescriptor.getPartitions(HivePartitionDescriptor.java:117)
> ~[classes/:na]
> at
> org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:185)
> ~[classes/:na]
> at
> org.apache.drill.exec.planner.sql.logical.HivePushPartitionFilterIntoScan$2.onMatch(HivePushPartitionFilterIntoScan.java:92)
> ~[classes/:na]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
> ~[calcite-core-1.4.0-drill-r0.jar:1.4.0-drill-r0]
> {code}
> Looking at {{HivePartitionLocation}}, it looks like we are depending on the
> organization of files on FileSystem to get the partition values. We should
> get the partition values from MetaStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
