[
https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-7355:
---------------------------------
Description:
Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset
vector. During the writing of these vectors, the offset value overflow is not
processed. Therefore, upon subsequent reading of such a vector to the
corresponding holder, the value in the "end" field may be less than the value
of the "start" field. In some cases, the "start" and "end" values are used to
get the size of the buffer (for example,ParquetOutputRecordWriter). This causes
the target buffer size to be negative.
Stack trace:
Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799
(expected: range(0, 524288))
at
io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125)
at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120)
at
io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244)
at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734)
at
io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34)
at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364)
at
org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824)
at
org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48)
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
at
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283)
was:
Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset
vector. When writing and reading these vectors, the overflow of the offset
value is not processed. While reading values into the corresponding holder, the
values of their offset vectors are written into the "start" and "end" fields as
ordinary integers. w In some cases, the "start" and "end" values are used to
get the size of the buffer (for example,ParquetOutputRecordWriter). This causes
the target buffer size to be negative.
Stack trace:
Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799
(expected: range(0, 524288))
at
io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125)
at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120)
at
io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244)
at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734)
at
io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34)
at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364)
at
org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824)
at
org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48)
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
at
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283)
> Reading a offset based vectors may produce IndexOutOfBoundsException
> --------------------------------------------------------------------
>
> Key: DRILL-7355
> URL: https://issues.apache.org/jira/browse/DRILL-7355
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: Oleg Zinoviev
> Priority: Major
>
> Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset
> vector. During the writing of these vectors, the offset value overflow is not
> processed. Therefore, upon subsequent reading of such a vector to the
> corresponding holder, the value in the "end" field may be less than the value
> of the "start" field. In some cases, the "start" and "end" values are used to
> get the size of the buffer (for example,ParquetOutputRecordWriter). This
> causes the target buffer size to be negative.
> Stack trace:
> Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799
> (expected: range(0, 524288))
> at
> io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125)
> at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120)
> at
> io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244)
> at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734)
> at
> io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34)
> at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364)
> at
> org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824)
> at
> org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48)
> at
> org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111)
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> at
> org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
