[jira] [Commented] (PARQUET-244) DeltaByteArrayReader fails with ArrayIndexOutOfBoundsException when moving across pages

[Alosh Bennett (JIRA)]

    [ 
https://issues.apache.org/jira/browse/PARQUET-244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14483098#comment-14483098
 ] 

Alosh Bennett commented on PARQUET-244:
---------------------------------------

Code to reproduce the bug. This will create a Parquet file that should contain 
a UTF-8 column split across multiple pages. The bug can be reproduced by 
reading this file from a sample program or via spark-shell.
{code:title=Bug.java|borderStyle=solid}
    public static void main(String[] args) throws IOException {
        String parquetFile = "file:///home/abennett/parquet/bug/sample.par";
        String schema = "message Document { required binary message (UTF8); }";
        WriteSupport<Document> writeSupport = new WriteSupport<Document>() {
            RecordConsumer rec;
            @Override
            public WriteContext init(Configuration configuration) {
                return new 
WriteContext(MessageTypeParser.parseMessageType(schema), new HashMap<>());
            }

            @Override
            public void prepareForWrite(RecordConsumer recordConsumer) {
                rec = recordConsumer;
            }

            @Override
            public void write(Document document) {
                rec.startMessage();
                rec.startField("message", 0);
                rec.addBinary(Binary.fromString(document.message));
                rec.endField("message", 0);
                rec.endMessage();

            }
        };
        ParquetWriter<Document> writer = new ParquetWriter<Document>(new 
Path(parquetFile), writeSupport, CompressionCodecName.SNAPPY,
                ParquetWriter.DEFAULT_BLOCK_SIZE, 
ParquetWriter.DEFAULT_PAGE_SIZE,
                ParquetWriter.DEFAULT_PAGE_SIZE, true, false, 
ParquetProperties.WriterVersion.PARQUET_2_0);
        Document doc = new Document();
        for(int i = 0; i < 100000; i++) {
            doc.message = UUID.randomUUID().toString();
            writer.write(doc);
        }
        writer.close();
    }

    private static class Document {
        String message;
    }
{code}

> DeltaByteArrayReader fails with ArrayIndexOutOfBoundsException when moving 
> across pages
> ---------------------------------------------------------------------------------------
>
>                 Key: PARQUET-244
>                 URL: https://issues.apache.org/jira/browse/PARQUET-244
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>    Affects Versions: parquet-mr_1.6.0
>            Reporter: Alosh Bennett
>
> DeltaByteArrayReader.readBytes() fails with  ArrayIndexOutOfBoundsException 
> soon after it has processed a new page via initFromPage(). This is happening 
> because 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)