[ https://issues.apache.org/jira/browse/DRILL-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sorabh Hamirwasia updated DRILL-6766: ------------------------------------- Reviewer: Boaz Ben-Zvi > Lateral Unnest query : IllegalStateException - rowId in right batch of > lateral is smaller than rowId in left batch being processed > ----------------------------------------------------------------------------------------------------------------------------------- > > Key: DRILL-6766 > URL: https://issues.apache.org/jira/browse/DRILL-6766 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Relational Operators > Affects Versions: 1.14.0 > Reporter: Kedar Sankar Behera > Assignee: Sorabh Hamirwasia > Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > > The error is coming when one batch of streaming agg is split across multiple > output batches. In that case the first output batch is sent downstream and in > subsequent next() StreamingAgg discards the unprocessed rows in previous > incoming and call's next() to upstream again. At this point lateral is > waiting to receive the rows for unprocessed ones from right side and instead > it see's a batch with lower rowId from streaming agg resulting in the > exception. > This is also a regression in previous behavior where without EMIT outcome > support StreamingAgg was handling this case properly. > {code:java} > SELECT ttt.number_segments_in_group from BGFsmall t, LATERAL (select > l.o.`element`.geo_segments.list as ot from unnest(t.geo_onds.list) l(o)) tt, > LATERAL (select count(*) as number_segments_in_group from unnest(tt.ot) > ll(o)) ttt limit 3; > {code} > Log:- > {code:java} > [Error Id: cfe63aaa-7115-4fab-b0fb-d5098d3b6350 on drill182:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > IllegalStateException: Unexpected case where rowId 0 in right batch of > lateral is smaller than rowId 32768 in left batch being processed > > Fragment 0:0 > > [Error Id: cfe63aaa-7115-4fab-b0fb-d5098d3b6350 on drill182:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) > ~[drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:360) > [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:215) > [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:326) > [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [na:1.8.0_161] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [na:1.8.0_161] > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161] > Caused by: java.lang.IllegalStateException: Unexpected case where rowId 0 in > right batch of lateral is smaller than rowId 32768 in left batch being > processed > at > org.apache.drill.shaded.guava.com.google.common.base.Preconditions.checkState(Preconditions.java:609) > ~[drill-shaded-guava-23.0.jar:23.0] > at > org.apache.drill.exec.physical.impl.join.LateralJoinBatch.crossJoinAndOutputRecords(LateralJoinBatch.java:1024) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.join.LateralJoinBatch.produceOutputBatch(LateralJoinBatch.java:575) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.join.LateralJoinBatch.innerNext(LateralJoinBatch.java:238) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:175) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:122) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:112) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:101) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:175) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:122) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:112) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:101) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:175) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:122) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:112) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:175) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:122) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:112) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:69) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:143) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:175) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:293) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:280) > ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[na:1.8.0_161] > at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_161] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595) > ~[hadoop-common-2.7.0-mapr-1707.jar:na] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:280) > [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] > ... 4 common frames omitted > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)