[
https://issues.apache.org/jira/browse/DRILL-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325050#comment-14325050
]
Deneche A. Hakim commented on DRILL-2253:
-----------------------------------------
I generated a small csv file that contains one "date" column. It has 600 rows
with a total of 287 distinct values for the date column.
I used CTAS to create a parquet file based on the csv file, using the following
query:
{noformat}
create table dfs.tmp.`2253` as select cast(columns[0] as date) `date` from
`2253.csv`;
{noformat}
Querying the parquet file will throw an IndexOutOfBounds exception:
{noformat}
0: jdbc:drill:zk=local> select * from dfs.tmp.`2253`;
Query failed: Query stopped., index: 685, length: 1 (expected: range(0, 685)) [
22150668-9b69-4946-b4f0-106defaa455e on 10.250.0.86:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{noformat}
The logs contain the following details about the exception:
{noformat}
java.lang.IndexOutOfBoundsException: index: 685, length: 1 (expected: range(0,
685))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:673)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.store.parquet.columnreaders.NullableFixedByteAlignedReaders$NullableDateReader.readIntLittleEndian(NullableFixedByteAlignedReaders.java:198)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.NullableFixedByteAlignedReaders$NullableDateReader.addNext(NullableFixedByteAlignedReaders.java:191)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.NullableFixedByteAlignedReaders$NullableConvertedReader.readField(NullableFixedByteAlignedReaders.java:172)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.NullableColumnReader.processPages(NullableColumnReader.java:130)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.NullableFixedByteAlignedReaders$NullableDateReader.processPages(NullableFixedByteAlignedReaders.java:179)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:367)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:410)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:97)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:116)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
{noformat}
> Vectorized Parquet reader fails to read correctly against RLE Dictionary
> encoded DATE column
> --------------------------------------------------------------------------------------------
>
> Key: DRILL-2253
> URL: https://issues.apache.org/jira/browse/DRILL-2253
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
> Reporter: Jacques Nadeau
> Assignee: Deneche A. Hakim
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
