Sorabh Hamirwasia created DRILL-6764:
----------------------------------------
Summary: Query fails with IOB when Unnest has reference to deep
nested field like (t.c_orders.o_lineitems)
Key: DRILL-6764
URL: https://issues.apache.org/jira/browse/DRILL-6764
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.14.0
Reporter: Sorabh Hamirwasia
Assignee: Hanumath Rao Maduri
Fix For: 1.15.0
When we run query with unnest referencing a list within a nested map type then
query fails with IOB. But if list if not nested in map it works fine. For
example:
*Failure Case: with schema*
{code:java}
customer {
element {
c_orders: [O1, O2]
}
}
{code}
{code:java}
SELECT tt.og, tt.ag
FROM dfs.testData.`/*` t,
LATERAL (SELECT l.o as og, l.o as ag FROM UNNEST(t.element.c_orders) l(o))
tt{code}
*Success Case:*
{code:java}
customer {
c_orders: [O1, O2]
}
{code}
{code:java}
SELECT tt.og, tt.ag
FROM dfs.testData.`/*` t,
LATERAL (SELECT l.o as og, l.o as ag FROM UNNEST(t.c_orders)
l(o)) tt{code}
*Exception:*
{code:java}
2018-09-28 11:13:03,633 [245190cf-da1c-02cd-977e-ded27e7abcff:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query with id
245190cf-da1c-02cd-977e-ded27e7abcff issued by anonymous: SELECT tt.og, tt.ag
FROM dfs.testData.`/*` t,
LATERAL (SELECT l.o as og, l.o as ag FROM UNNEST(t.element.c_orders) l(o)) tt
2018-09-28 11:13:03,725 [245190cf-da1c-02cd-977e-ded27e7abcff:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: IndexOutOfBoundsException:
index (3) must be less than size (3)
[Error Id: d5a21cc8-581f-4c35-9950-c7cfbd7de194 on 172.30.8.49:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IndexOutOfBoundsException: index (3) must be less than size (3)
[Error Id: d5a21cc8-581f-4c35-9950-c7cfbd7de194 on 172.30.8.49:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
~[drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:779)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.QueryStateProcessor.checkCommonStates(QueryStateProcessor.java:325)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.QueryStateProcessor.planning(QueryStateProcessor.java:221)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.QueryStateProcessor.moveToState(QueryStateProcessor.java:83)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:299)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_102]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_102]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected
exception during fragment initialization: Error while applying rule
DrillMergeProjectRule:force_mode, args
[rel#4423:LogicalProject.NONE.ANY([]).[](input=rel#4422:Subset#8.NONE.ANY([]).[],og=$2,ag=$3),
rel#4421:LogicalProject.NONE.ANY([]).[](input=rel#4420:Subset#7.NONE.ANY([]).[],**=$0,c_orders=$1,og=$3)]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:300)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
... 3 common frames omitted
Caused by: java.lang.RuntimeException: Error while applying rule
DrillMergeProjectRule:force_mode, args
[rel#4423:LogicalProject.NONE.ANY([]).[](input=rel#4422:Subset#8.NONE.ANY([]).[],og=$2,ag=$3),
rel#4421:LogicalProject.NONE.ANY([]).[](input=rel#4420:Subset#7.NONE.ANY([]).[],**=$0,c_orders=$1,og=$3)]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:236)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:648)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:425)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:365)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:252)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:314)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:179)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:145)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:83)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:584)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:272)
[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
... 3 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: index (3) must be less than
size (3)
at
com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
~[guava-19.0.jar:na]
at
com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
~[guava-19.0.jar:na]
at
com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
~[guava-19.0.jar:na]
at
org.apache.calcite.plan.RelOptUtil$2.visitInputRef(RelOptUtil.java:2638)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at
org.apache.calcite.plan.RelOptUtil$2.visitInputRef(RelOptUtil.java:2636)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at org.apache.calcite.rex.RexShuttle.visitList(RexShuttle.java:166)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at
org.apache.calcite.plan.RelOptUtil.pushPastProject(RelOptUtil.java:2631)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
at
org.apache.drill.exec.planner.logical.DrillMergeProjectRule.onMatch(DrillMergeProjectRule.java:134)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
... 14 common frames omitted
2018-09-28 11:13:03,734 [qtp224809804-247] ERROR
o.a.d.e.server.rest.QueryResources - Query from Web UI Failed
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IndexOutOfBoundsException: index (3) must be less than size (3)
[Error Id: d5a21cc8-581f-4c35-9950-c7cfbd7de194 on 172.30.8.49:31010]
at
org.apache.drill.exec.server.rest.QueryWrapper.run(QueryWrapper.java:115)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.server.rest.QueryResources.submitQueryJSON(QueryResources.java:72)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at
org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:87)
~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
~[na:na]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:195)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na]
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na]
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na]
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na]
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505]
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
