[
https://issues.apache.org/jira/browse/DRILL-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-5521:
------------------------------------
Description:
In DRILL-5419 introduced some changes in UnionAllRecordBatch.
One change allowed to transfer data directly if column name is the same.
This part of code existed before but was not executed since if condition was
always returning false:
{noformat}
if(outputFields.get(index).getPath().equals(inputPath)) {
{noformat}
But fixing this condition lead to the following error
Change in code:
{noformat}
if (outputFields.get(index).getPath().equals(inputPath.getAsUnescapedPath())) {
{noformat}
Data set:
{noformat}
cat missing-map.json
{"rk": "a", "m": {"a":"1"}}
{noformat}
Query:
{noformat}
select kvgen(m) from (select m from dfs.`missing-map.json` union all select
convert_from('{"a" : null}' ,'json') as m from (values(1)))
{noformat}
Error:
{noformat}
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
Fragment 0:0
[Error Id: 7f95af7b-036b-4f74-820a-c720d527d2e5 on localhost:31010]
(java.lang.IndexOutOfBoundsException) index: 0, length: 1 (expected: range(0,
0))
io.netty.buffer.DrillBuf.checkIndexD():123
io.netty.buffer.DrillBuf.chk():147
io.netty.buffer.DrillBuf.getByte():775
org.apache.drill.exec.vector.UInt1Vector$Accessor.get():354
org.apache.drill.exec.vector.NullableVarCharVector$Accessor.isSet():399
org.apache.drill.exec.vector.NullableVarCharVector$Accessor.isNull():395
org.apache.drill.exec.vector.complex.impl.NullableVarCharReaderImpl.isSet():100
org.apache.drill.exec.expr.fn.impl.MappifyUtility.mappify():59
org.apache.drill.exec.test.generated.ProjectorGen2.doEval():38
org.apache.drill.exec.test.generated.ProjectorGen2.projectRecords():67
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():198
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.innerNext():121
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.physical.impl.BaseRootExec.next():105
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
org.apache.drill.exec.physical.impl.BaseRootExec.next():95
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():421
org.apache.hadoop.security.UserGroupInformation.doAs():1657
org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1145
java.util.concurrent.ThreadPoolExecutor$Worker.run():615
java.lang.Thread.run():745
{noformat}
was:
In DRILL-5419 introduced some changes in UnionAllRecordBatch.
One change allowed to transfer data directly if column name is the same.
This part of code existed before but was not executed since if condition was
always returning false:
{noformat}
if(outputFields.get(index).getPath().equals(inputPath)) {
{noformat}
But fixing this condition lead to the following error
Change in code:
{noformat}
if (outputFields.get(index).getPath().equals(inputPath.getAsUnescapedPath())) {
{noformat}
Data set:
{noformat}
cat missing-map.json
{"rk": "a", "m": {"a":"1"}}
{noformat}
Query:
{noformat}
select kvgen(m) from dfs.`missing-map.json` union all select convert_from('{"a"
: null}' ,'json') as m from (values(1))
{noformat}
Error:
{noformat}
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
Fragment 0:0
[Error Id: 7f95af7b-036b-4f74-820a-c720d527d2e5 on localhost:31010]
(java.lang.IndexOutOfBoundsException) index: 0, length: 1 (expected: range(0,
0))
io.netty.buffer.DrillBuf.checkIndexD():123
io.netty.buffer.DrillBuf.chk():147
io.netty.buffer.DrillBuf.getByte():775
org.apache.drill.exec.vector.UInt1Vector$Accessor.get():354
org.apache.drill.exec.vector.NullableVarCharVector$Accessor.isSet():399
org.apache.drill.exec.vector.NullableVarCharVector$Accessor.isNull():395
org.apache.drill.exec.vector.complex.impl.NullableVarCharReaderImpl.isSet():100
org.apache.drill.exec.expr.fn.impl.MappifyUtility.mappify():59
org.apache.drill.exec.test.generated.ProjectorGen2.doEval():38
org.apache.drill.exec.test.generated.ProjectorGen2.projectRecords():67
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():198
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.innerNext():121
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
org.apache.drill.exec.physical.impl.BaseRootExec.next():105
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
org.apache.drill.exec.physical.impl.BaseRootExec.next():95
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():421
org.apache.hadoop.security.UserGroupInformation.doAs():1657
org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1145
java.util.concurrent.ThreadPoolExecutor$Worker.run():615
java.lang.Thread.run():745
{noformat}
> IndexOutOfBoundsException during union all for map types
> --------------------------------------------------------
>
> Key: DRILL-5521
> URL: https://issues.apache.org/jira/browse/DRILL-5521
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.11.0
> Reporter: Arina Ielchiieva
> Assignee: Arina Ielchiieva
> Fix For: 1.11.0
>
>
> In DRILL-5419 introduced some changes in UnionAllRecordBatch.
> One change allowed to transfer data directly if column name is the same.
> This part of code existed before but was not executed since if condition was
> always returning false:
> {noformat}
> if(outputFields.get(index).getPath().equals(inputPath)) {
> {noformat}
> But fixing this condition lead to the following error
> Change in code:
> {noformat}
> if (outputFields.get(index).getPath().equals(inputPath.getAsUnescapedPath()))
> {
> {noformat}
> Data set:
> {noformat}
> cat missing-map.json
> {"rk": "a", "m": {"a":"1"}}
> {noformat}
> Query:
> {noformat}
> select kvgen(m) from (select m from dfs.`missing-map.json` union all select
> convert_from('{"a" : null}' ,'json') as m from (values(1)))
> {noformat}
> Error:
> {noformat}
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> Fragment 0:0
> [Error Id: 7f95af7b-036b-4f74-820a-c720d527d2e5 on localhost:31010]
> (java.lang.IndexOutOfBoundsException) index: 0, length: 1 (expected:
> range(0, 0))
> io.netty.buffer.DrillBuf.checkIndexD():123
> io.netty.buffer.DrillBuf.chk():147
> io.netty.buffer.DrillBuf.getByte():775
> org.apache.drill.exec.vector.UInt1Vector$Accessor.get():354
> org.apache.drill.exec.vector.NullableVarCharVector$Accessor.isSet():399
> org.apache.drill.exec.vector.NullableVarCharVector$Accessor.isNull():395
>
> org.apache.drill.exec.vector.complex.impl.NullableVarCharReaderImpl.isSet():100
> org.apache.drill.exec.expr.fn.impl.MappifyUtility.mappify():59
> org.apache.drill.exec.test.generated.ProjectorGen2.doEval():38
> org.apache.drill.exec.test.generated.ProjectorGen2.projectRecords():67
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():198
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
>
> org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.innerNext():121
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
> org.apache.drill.exec.record.AbstractRecordBatch.next():119
> org.apache.drill.exec.record.AbstractRecordBatch.next():109
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
> org.apache.drill.exec.record.AbstractRecordBatch.next():162
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():215
> org.apache.drill.exec.physical.impl.BaseRootExec.next():105
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
> org.apache.drill.exec.physical.impl.BaseRootExec.next():95
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():234
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():227
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():421
> org.apache.hadoop.security.UserGroupInformation.doAs():1657
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():227
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
