[
https://issues.apache.org/jira/browse/DRILL-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14335501#comment-14335501
]
Rahul Challapalli commented on DRILL-2160:
------------------------------------------
Though the exceptions looks different, I thought we are wrongly interpreting
the schema. Drill some how detects a schema change when there is none. This
could be the root cause and depending on the execution path we might be hitting
either "Unsupported Operation Exception", "Class Cast Exception" or "Sort
currently only supports a single schema" errors.
This could also not be an improvement as the same original works fine on a
smaller data set. The bigger data set is simply generated by repeatedly copying
over the smaller data set.
> External Sort and Flatten error : Sort currently only supports single schema
> ----------------------------------------------------------------------------
>
> Key: DRILL-2160
> URL: https://issues.apache.org/jira/browse/DRILL-2160
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Reporter: Rahul Challapalli
> Assignee: Chris Westin
> Fix For: Future
>
> Attachments: data.json
>
>
> git.commit.id.abbrev=3e33880
> Attached the data set which only contains 2 records
> The below query works as expected with the 2 record data set
> {code}
> select d.type type, d.uid uid, flatten(d.map.rm) rms from `data.json` d order
> by d.uid;
> {code}
> However the same query fails with the below error when we copy the smaller
> dataset 50,000 times
> {code}
> ERROR o.a.d.e.p.i.xsort.ExternalSortBatch - Failure during query
> org.apache.drill.exec.exception.SchemaChangeException: Sort currently only
> supports a single schema.
> at
> org.apache.drill.exec.physical.impl.sort.SortRecordBatchBuilder.build(SortRecordBatchBuilder.java:144)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:365)
> ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:97)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:110)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> {code}
> When we remove flatten, the query works as expected
> {code}
> select d.type type, d.uid uid from `data.json` d order by d.uid;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
