[
https://issues.apache.org/jira/browse/DRILL-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583126#comment-16583126
]
ASF GitHub Bot commented on DRILL-6694:
---------------------------------------
sohami commented on a change in pull request #1434: DRILL-6694: NPE in
UnnestRecordBatch when query uses a column name no…
URL: https://github.com/apache/drill/pull/1434#discussion_r210759060
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unnest/UnnestRecordBatch.java
##########
@@ -62,6 +62,8 @@
private int remainderIndex = 0;
private int recordCount;
private MaterializedField unnestFieldMetadata;
+ // Reference of TypedFieldId for Unnest column. It's always set in
schemaChanged method and later used by others
+ private TypedFieldId unnestTypedFieldId;
Review comment:
kept a reference of it here since it's costly to get it everytime from
incoming batch based on column name
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> NPE in UnnestRecordBatch when query uses a column name not present in data
> --------------------------------------------------------------------------
>
> Key: DRILL-6694
> URL: https://issues.apache.org/jira/browse/DRILL-6694
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Affects Versions: 1.14.0
> Reporter: Sorabh Hamirwasia
> Assignee: Sorabh Hamirwasia
> Priority: Major
> Fix For: 1.15.0
>
>
> When the array column name doesn't exist in the underlying data and is used
> in query with Unnest then there is NPE. The reason is Unnest tries to get the
> ValueVector of unnest column from incoming based on TypedFieldId which will
> be null in this case and hence the exception.
> {code:java}
> [Error Id: 6f8461ee-92c7-4865-b5e6-3e2f756391c4 on pssc-67.qa.lab:31010] at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:361)
> [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:216)
> [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:327)
> [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_151] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
> Caused by: java.lang.NullPointerException: null at
> org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.schemaChanged(UnnestRecordBatch.java:422)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.innerNext(UnnestRecordBatch.java:208)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:64)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:142)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.prefetchFirstBatchFromBothSides(LateralJoinBatch.java:331)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.buildSchema(LateralJoinBatch.java:356)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:152)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.aggregate.StreamingAggBatch.buildSchema(StreamingAggBatch.java:158)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:152)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:294)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:281)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at
> java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_151] at
> javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_151] at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1633)
> ~[hadoop-common-2.7.0-mapr-1707.jar:na] at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:281)
> [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] ... 4 common frames
> omitted 14:18:13.952 [qtp667758892-38873] ERROR
> o.a.d.e.server.rest.QueryResources - Query from Web UI Failed
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> NullPointerException{code}
>
> Query:
> select count(t.l.l_shipdate) from dfs.nestedPar.`cl60_60` c,
> unnest(c.c_ldata) t(l)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
