[jira] [Commented] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502153#comment-15502153
]
ASF GitHub Bot commented on DRILL-4884:
---
Github user zbdzzg commented on the issue:
https://github.com/apache/drill/pull/584
@dsbos
> Drill produced IOB exception while querying data of 65536 limitation using
> non batched reader
> -
>
> Key: DRILL-4884
> URL: https://issues.apache.org/jira/browse/DRILL-4884
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.8.0
> Environment: CentOS 6.5 / JAVA 8
>Reporter: Hongze Zhang
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Drill produces IOB while using a non batched scanner and limiting SQL by
> 65536.
> SQL:
> {noformat}
> select id from xx limit 1 offset 65535
> {noformat}
> Result:
> {noformat}
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [classes/:na]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [classes/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
> (expected: range(0, 131072))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
> ~[classes/:4.0.27.Final]
> at
> org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[classes/:na]
> at
>
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Component/s: (was: Functions - Drill)
Query Planning & Optimization
> Drill produced IOB exception while querying data of 65536 limitation using
> non batched reader
> -
>
> Key: DRILL-4884
> URL: https://issues.apache.org/jira/browse/DRILL-4884
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.8.0
> Environment: CentOS 6.5 / JAVA 8
>Reporter: Hongze Zhang
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Drill produces IOB while using a non batched scanner and limiting SQL by
> 65536.
> SQL:
> {noformat}
> select id from xx limit 1 offset 65535
> {noformat}
> Result:
> {noformat}
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [classes/:na]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [classes/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
> (expected: range(0, 131072))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
> ~[classes/:4.0.27.Final]
> at
> org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
> ~[classes/:na]
>
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Reviewer: Daniel Barclay
> Drill produced IOB exception while querying data of 65536 limitation using
> non batched reader
> -
>
> Key: DRILL-4884
> URL: https://issues.apache.org/jira/browse/DRILL-4884
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.8.0
> Environment: CentOS 6.5 / JAVA 8
>Reporter: Hongze Zhang
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Drill produces IOB while using a non batched scanner and limiting SQL by
> 65536.
> SQL:
> {noformat}
> select id from xx limit 1 offset 65535
> {noformat}
> Result:
> {noformat}
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [classes/:na]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [classes/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
> (expected: range(0, 131072))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
> ~[classes/:4.0.27.Final]
> at
> org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
> ~[classes/:na]
> at
>
[jira] [Updated] (DRILL-3168) Char overflow in LimitRecordBatch
[
https://issues.apache.org/jira/browse/DRILL-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
F Méthot updated DRILL-3168:
Assignee: (was: F Méthot)
> Char overflow in LimitRecordBatch
> --
>
> Key: DRILL-3168
> URL: https://issues.apache.org/jira/browse/DRILL-3168
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 0.9.0
>Reporter: 徐波
> Fix For: Future
>
>
> The variable named 'i' in 'limitWithNoSV' may overflow when fetch - offset >
> Character.MAX_VALUE - offset. eg. offset=0, fetch=65536.
> Code in limitWithNoSV:
> int svIndex = 0;
> for(char i = (char) offset; i < fetch; i++) {
> outgoingSv.setIndex(svIndex, i);
> svIndex++;
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4889) throw NPE or AssertionError in FindPartitionConditions when running some SQLs on partition table
[
https://issues.apache.org/jira/browse/DRILL-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated DRILL-4889:
--
Description:
When I ran some SQLs on partition table, I got NPE or AssertionError in
FindPartitionConditions as below:
partition_table is a partition table
partition_column is the partition column
none_partition_column is a normal column, not the partition column
*SQL 1:* select * from partition_table where none_partition_column like
'%test%';
*SQL 2:* select partition_column from partition_table where
none_partition_column like '%test%';
*SQL 3:* select * from partition_table where case when partition_column =
'2000' then none_partition_column = 'test' end;
{quote}
Caused by: java.lang.NullPointerException: null
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions$OpState.access$000(FindPartitionConditions.java:63)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.analyzeCall(FindPartitionConditions.java:315)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:260)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:46)
{quote}
*SQL 4:* select * from partition_table where none_partition_column = 'a' or
(none_partition_column = 'b' and partition_column = '2000');
{quote}
Caused by: java.lang.AssertionError: null
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.popOpStackAndBuildFilter(FindPartitionConditions.java:198)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.analyzeCall(FindPartitionConditions.java:335)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:260)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:46)
{quote}
was:
When I ran some SQLs on partition table, I got NPE or AssertionError in
FindPartitionConditions as below:
partition_table is a partition table
partition_column is the partition column
none_partition_column is a normal column, not the partition column
*SQL 1:* select * from partition_table where none_partition_column like
'%test%';
and also
*SQL 2:* select * from partition_table where case when partition_column =
'2000' then none_partition_column = 'test' end;
{quote}
Caused by: java.lang.NullPointerException: null
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions$OpState.access$000(FindPartitionConditions.java:63)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.analyzeCall(FindPartitionConditions.java:315)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:260)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:46)
{quote}
*SQL 3:* select * from partition_table where none_partition_column = 'a' or
(none_partition_column = 'b' and partition_column = '2000');
{quote}
Caused by: java.lang.AssertionError: null
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.popOpStackAndBuildFilter(FindPartitionConditions.java:198)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.analyzeCall(FindPartitionConditions.java:335)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:260)
at
org.apache.drill.exec.planner.logical.partition.FindPartitionConditions.visitCall(FindPartitionConditions.java:46)
{quote}
> throw NPE or AssertionError in FindPartitionConditions when running some SQLs
> on partition table
>
>
> Key: DRILL-4889
> URL: https://issues.apache.org/jira/browse/DRILL-4889
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.8.0
>Reporter: godfrey he
>
> When I ran some SQLs on partition table, I got NPE or AssertionError in
> FindPartitionConditions as below:
> partition_table is a partition table
> partition_column is the partition column
> none_partition_column is a normal column, not the partition column
> *SQL 1:* select * from partition_table where none_partition_column like
> '%test%';
> *SQL 2:* select partition_column from partition_table where
> none_partition_column like '%test%';
> *SQL 3:* select * from partition_table where case when partition_column =
> '2000' then none_partition_column = 'test' end;
> {quote}
> Caused by:
