[
https://issues.apache.org/jira/browse/TAJO-1250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14255527#comment-14255527
]
Hudson commented on TAJO-1250:
------------------------------
SUCCESS: Integrated in Tajo-master-build #516 (See
[https://builds.apache.org/job/Tajo-master-build/516/])
TAJO-1250: RawFileAppender occasionally causes BufferOverflowException. (jinho)
(jhkim: rev cf66a390060c79ba757097886703e30f93d31401)
* CHANGES
*
tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/RawFile.java
*
tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/DelimitedLineReader.java
* tajo-storage/tajo-storage-common/src/test/resources/storage-default.xml
* tajo-storage/tajo-storage-common/src/main/resources/storage-default.xml
* tajo-storage/tajo-storage-hdfs/src/test/resources/storage-default.xml
*
tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/DelimitedTextFile.java
*
tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
*
tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageManager.java
> RawFileAppender occasionally causes BufferOverflowException
> -----------------------------------------------------------
>
> Key: TAJO-1250
> URL: https://issues.apache.org/jira/browse/TAJO-1250
> Project: Tajo
> Issue Type: Bug
> Components: data shuffle, storage
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Jinho Kim
> Assignee: Jinho Kim
> Fix For: 0.10
>
>
> The main problem is the maximum bytes size of variant int64.
> if a value is long.max, varint64 length is 10 bytes .
> * Variant int32 : 1~5 bytes
> * Variant int64 : 1~10 bytes
> {noformat}
> 2014-12-15 13:25:47,908 ERROR
> org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec:
> java.nio.BufferOverflowException
> at java.nio.Buffer.nextPutIndex(Buffer.java:513)
> at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:291)
> at
> org.apache.tajo.storage.RawFile$RawFileAppender.writeRawVarint64(RawFile.java:604)
> at
> org.apache.tajo.storage.RawFile$RawFileAppender.addTuple(RawFile.java:659)
> at
> org.apache.tajo.storage.HashShuffleAppender.addTuples(HashShuffleAppender.java:92)
> at
> org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:137)
> at org.apache.tajo.worker.Task.run(Task.java:407)
> at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:276)
> at java.lang.Thread.run(Thread.java:745)
> 2014-12-15 13:25:47,910 ERROR org.apache.tajo.worker.Task:
> java.nio.BufferOverflowException
> java.io.IOException: java.nio.BufferOverflowException
> at
> org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:152)
> at org.apache.tajo.worker.Task.run(Task.java:407)
> at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:276)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.nio.BufferOverflowException
> at java.nio.Buffer.nextPutIndex(Buffer.java:513)
> at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:291)
> at
> org.apache.tajo.storage.RawFile$RawFileAppender.writeRawVarint64(RawFile.java:604)
> at
> org.apache.tajo.storage.RawFile$RawFileAppender.addTuple(RawFile.java:659)
> at
> org.apache.tajo.storage.HashShuffleAppender.addTuples(HashShuffleAppender.java:92)
> at
> org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:137)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
