Hao Zhu created DRILL-3579:
------------------------------
Summary: Drill on Hive query fails if partition table has
__HIVE_DEFAULT_PARTITION__
Key: DRILL-3579
URL: https://issues.apache.org/jira/browse/DRILL-3579
Project: Apache Drill
Issue Type: Bug
Components: Functions - Hive
Affects Versions: 1.1.0
Environment: Drill 1.1 on Hive 1.0
Reporter: Hao Zhu
Assignee: Mehant Baid
If Hive's partition table has __HIVE_DEFAULT_PARTITION__ in the case of null
values in the partition column, Drill on Hive query will fail.
Minimum reproduce:
1.Hive:
{code}
CREATE TABLE h1_testpart2(id INT) PARTITIONED BY(id2 int);
set hive.exec.dynamic.partition.mode=nonstrict;
INSERT OVERWRITE TABLE h1_testpart2 PARTITION(id2) SELECT 1 as id1 , 20150101
as id2 from h1_passwords limit 1;
INSERT OVERWRITE TABLE h1_testpart2 PARTITION(id2) SELECT 1 as id1 , null as
id2 from h1_passwords limit 1;
{code}
2. Filesystem looks like:
{code}
h1 h1_testpart2]# ls -altr
total 2
drwxrwxrwx 89 mapr mapr 87 Jul 30 00:04 ..
drwxr-xr-x 2 mapr mapr 1 Jul 30 00:05 id2=20150101
drwxr-xr-x 2 mapr mapr 1 Jul 30 00:05 id2=__HIVE_DEFAULT_PARTITION__
drwxr-xr-x 4 mapr mapr 2 Jul 30 00:05 .
{code}
3.Drill will fail:
{code}
select * from h1_testpart2;
Error: SYSTEM ERROR: NumberFormatException: For input string:
"__HIVE_DEFAULT_PARTITION__"
Fragment 0:0
[Error Id: 509eb392-db9a-42f3-96ea-fb597425f49f on h1.poc.com:31010]
(java.lang.reflect.UndeclaredThrowableException) null
org.apache.hadoop.security.UserGroupInformation.doAs():1581
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1142
java.util.concurrent.ThreadPoolExecutor$Worker.run():617
java.lang.Thread.run():745
Caused By (org.apache.drill.common.exceptions.ExecutionSetupException)
Failure while initializing HiveRecordReader: For input string:
"__HIVE_DEFAULT_PARTITION__"
org.apache.drill.exec.store.hive.HiveRecordReader.init():241
org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138
org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58
org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34
org.apache.drill.exec.physical.impl.ImplCreator$2.run():138
org.apache.drill.exec.physical.impl.ImplCreator$2.run():136
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1566
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1142
java.util.concurrent.ThreadPoolExecutor$Worker.run():617
java.lang.Thread.run():745
Caused By (java.lang.NumberFormatException) For input string:
"__HIVE_DEFAULT_PARTITION__"
java.lang.NumberFormatException.forInputString():65
java.lang.Integer.parseInt():580
java.lang.Integer.parseInt():615
org.apache.drill.exec.store.hive.HiveRecordReader.convertPartitionType():605
org.apache.drill.exec.store.hive.HiveRecordReader.init():236
org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138
org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58
org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34
org.apache.drill.exec.physical.impl.ImplCreator$2.run():138
org.apache.drill.exec.physical.impl.ImplCreator$2.run():136
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1566
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173
org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106
org.apache.drill.exec.physical.impl.ImplCreator.getExec():81
org.apache.drill.exec.work.fragment.FragmentExecutor.run():235
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1142
java.util.concurrent.ThreadPoolExecutor$Worker.run():617
java.lang.Thread.run():745 (state=,code=0)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
