Looks like we are failing in split generation of ORC files. Can you log a JIRA and if possible please attach a small orc file for repro?
Thanks Venki On Wed, Jun 17, 2015 at 7:09 AM, Arthur Chan <[email protected]> wrote: > the error log: > > 2015-06-17 22:04:01,114 [2a7e81af-1479-ccf9-e3d5-03cbe60501cf:foreman] > INFO o.a.drill.exec.work.foreman.Foreman - State change requested. > PENDING --> FAILED > > org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception > during fragment initialization: Internal error: Error while applying rule > DrillTableRule, args > [rel#562:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[hive, hdbs, > orders])] > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:252) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_40] > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_40] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40] > > Caused by: java.lang.AssertionError: Internal error: Error while applying > rule DrillTableRule, args > [rel#562:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[hive, hdbs, > orders])] > > at org.apache.calcite.util.Util.newInternal(Util.java:790) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at > > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at > > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:795) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:316) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:507) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:281) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:187) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:177) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:902) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:240) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > ... 3 common frames omitted > > Caused by: java.lang.RuntimeException: serious problem > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$Context.waitForTasks(OrcInputFormat.java:466) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:919) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:944) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > at org.apache.drill.exec.store.hive.HiveScan.splitInput(HiveScan.java:177) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at org.apache.drill.exec.store.hive.HiveScan.getSplits(HiveScan.java:144) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at org.apache.drill.exec.store.hive.HiveScan.<init>(HiveScan.java:116) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at > > org.apache.drill.exec.store.hive.HiveStoragePlugin.getPhysicalScan(HiveStoragePlugin.java:78) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at > > org.apache.drill.exec.store.hive.HiveStoragePlugin.getPhysicalScan(HiveStoragePlugin.java:41) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at > > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan(AbstractStoragePlugin.java:53) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan(DrillTable.java:72) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:81) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:63) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch(DrillScanRule.java:37) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > ... 12 common frames omitted > > Caused by: java.lang.NullPointerException: null > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.run(OrcInputFormat.java:782) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > ... 3 common frames omitted > > 2015-06-17 22:04:01,129 [2a7e81af-1479-ccf9-e3d5-03cbe60501cf:foreman] > INFO o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. > > 2015-06-17 22:04:01,131 [2a7e81af-1479-ccf9-e3d5-03cbe60501cf:foreman] > ERROR o.a.d.c.exceptions.UserException - SYSTEM ERROR: > java.lang.RuntimeException: serious problem > > > > [Error Id: 0e0c1c1d-2ca8-441e-a5d7-628280c70b62 on hd7.localnet.net:31010] > > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > java.lang.RuntimeException: serious problem > > > > [Error Id: 0e0c1c1d-2ca8-441e-a5d7-628280c70b62 on hd7.localnet.net:31010] > > at > > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:522) > ~[drill-common-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:736) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:838) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:780) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) > [drill-common-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:782) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:891) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:251) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_40] > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_40] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40] > > Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected > exception during fragment initialization: Internal error: Error while > applying rule DrillTableRule, args > [rel#562:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[hive, hdbs, > orders])] > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:252) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > ... 3 common frames omitted > > Caused by: java.lang.AssertionError: Internal error: Error while applying > rule DrillTableRule, args > [rel#562:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[hive, hdbs, > orders])] > > at org.apache.calcite.util.Util.newInternal(Util.java:790) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at > > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at > > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:795) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:316) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > at > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:507) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:281) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:187) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:177) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:902) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:240) > [drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > ... 3 common frames omitted > > Caused by: java.lang.RuntimeException: serious problem > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$Context.waitForTasks(OrcInputFormat.java:466) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:919) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:944) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > at org.apache.drill.exec.store.hive.HiveScan.splitInput(HiveScan.java:177) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at org.apache.drill.exec.store.hive.HiveScan.getSplits(HiveScan.java:144) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at org.apache.drill.exec.store.hive.HiveScan.<init>(HiveScan.java:116) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at > > org.apache.drill.exec.store.hive.HiveStoragePlugin.getPhysicalScan(HiveStoragePlugin.java:78) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at > > org.apache.drill.exec.store.hive.HiveStoragePlugin.getPhysicalScan(HiveStoragePlugin.java:41) > ~[drill-storage-hive-core-1.0.0.jar:1.0.0] > > at > > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan(AbstractStoragePlugin.java:53) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan(DrillTable.java:72) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:81) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:63) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch(DrillScanRule.java:37) > ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0] > > at > > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) > ~[calcite-core-1.1.0-drill-r7.jar:1.1.0-drill-r7] > > ... 12 common frames omitted > > Caused by: java.lang.NullPointerException: null > > at > > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.run(OrcInputFormat.java:782) > ~[drill-hive-exec-shaded-1.0.0.jar:1.0.0] > > ... 3 common frames omitted > > > On Wed, Jun 17, 2015 at 9:38 PM, Christopher Matta <[email protected]> > wrote: > > > Yea, leave the default string out when querying the hive default > > configuration. > > > > > > Chris Matta > > [email protected] > > 215-701-3146 > > > > On Wed, Jun 17, 2015 at 9:33 AM, Leon Clayton <[email protected]> > > wrote: > > > > > Just done this on the sandbox and it works fine. > > > > > > 0: jdbc:drill:> select count(1) from hive.orders; > > > +------------+ > > > | EXPR$0 | > > > +------------+ > > > | 122000 | > > > +------------+ > > > > > > On 17 Jun 2015, at 14:22, Arthur Chan <[email protected]> > wrote: > > > > > > > Hi, > > > > > > > > I ran a simple one table hive query "select count(1) from > > > > hive.`default`.txn", > > > > I got > > > > > > > > Query Failed: An Error Occurred > > > > org.apache.drill.common.exceptions.UserRemoteException: > > > > SYSTEM ERROR: java.lang.RuntimeException: serious problem > > > > > > > > Please advise > > > > Regards > > > > > > > > >
