[
https://issues.apache.org/jira/browse/DRILL-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545566#comment-16545566
]
ASF GitHub Bot commented on DRILL-6475:
---------------------------------------
HanumathRao commented on a change in pull request #1381: DRILL-6475: Unnest:
Null fieldId Pointer.
URL: https://github.com/apache/drill/pull/1381#discussion_r202772270
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/JoinPrelRenameVisitor.java
##########
@@ -19,36 +19,70 @@
import java.util.ArrayList;
import java.util.List;
-
+import java.util.Map;
+import java.util.HashMap;
+
+import com.google.common.base.Preconditions;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeField;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexNode;
import org.apache.drill.exec.planner.physical.JoinPrel;
import org.apache.drill.exec.planner.physical.LateralJoinPrel;
import org.apache.drill.exec.planner.physical.Prel;
import org.apache.calcite.rel.RelNode;
import com.google.common.collect.Lists;
+import org.apache.drill.exec.planner.physical.UnnestPrel;
public class JoinPrelRenameVisitor extends BasePrelVisitor<Prel, Void,
RuntimeException>{
+ private final Map<String, Prel> sourceOperatorRegistry = new HashMap();
+
private static JoinPrelRenameVisitor INSTANCE = new JoinPrelRenameVisitor();
public static Prel insertRenameProject(Prel prel){
return prel.accept(INSTANCE, null);
}
+ private void register(Prel toRegister) {
+ this.sourceOperatorRegistry.put(toRegister.getClass().getSimpleName(),
toRegister);
Review comment:
Yes, I agree that each lateral can be associated to many unnest. I think the
current design itself doesn't take into account the correct association with
unnest and lateral. It traverses the right of the lateral looking for unnest
and associates them. This can cause potential issues when we support binary
operators like union, join etc. I think ideally we should have an id with each
lateral and then associate that id to an unnest. This id can be used in the map
while traversal etc. But for now as we don't support binary operators I have
used the classname instead of id's to associate the lateral and unnest. Let me
know if you have a better way to fix this issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Unnest: Null fieldId Pointer
> -----------------------------
>
> Key: DRILL-6475
> URL: https://issues.apache.org/jira/browse/DRILL-6475
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Reporter: Boaz Ben-Zvi
> Assignee: Hanumath Rao Maduri
> Priority: Major
> Fix For: 1.14.0
>
>
> Executing the following (in TestE2EUnnestAndLateral.java) causes an NPE as
> `fieldId` is null in `schemaChanged()`:
> {code}
> @Test
> public void testMultipleBatchesLateral_twoUnnests() throws Exception {
> String sql = "SELECT t5.l_quantity FROM dfs.`lateraljoin/multipleFiles/` t,
> LATERAL " +
> "(SELECT t2.ordrs FROM UNNEST(t.c_orders) t2(ordrs)) t3(ordrs), LATERAL " +
> "(SELECT t4.l_quantity FROM UNNEST(t3.ordrs) t4(l_quantity)) t5";
> test(sql);
> }
> {code}
>
> And the error is:
> {code}
> Error: SYSTEM ERROR: NullPointerException
> Fragment 0:0
> [Error Id: 25f42765-8f68-418e-840a-ffe65788e1e2 on 10.254.130.25:31020]
> (java.lang.NullPointerException) null
>
> org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.schemaChanged():381
> org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.innerNext():199
> org.apache.drill.exec.record.AbstractRecordBatch.next():172
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
>
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.prefetchFirstBatchFromBothSides():241
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.buildSchema():264
> org.apache.drill.exec.record.AbstractRecordBatch.next():152
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():137
> org.apache.drill.exec.record.AbstractRecordBatch.next():172
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():137
> org.apache.drill.exec.record.AbstractRecordBatch.next():172
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():229
> org.apache.drill.exec.physical.impl.BaseRootExec.next():103
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():83
> org.apache.drill.exec.physical.impl.BaseRootExec.next():93
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():292
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():279
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():279
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
