[
https://issues.apache.org/jira/browse/TAJO-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068301#comment-14068301
]
Mai Hai Thanh commented on TAJO-666:
------------------------------------
Hi,
I created a patch to solve to this bug. Instead of allocating a fixed
ByteBuffer of 4096 bytes, we estimate the size of row data and then use this
estimated value for allocating the ByteBuffer. This solution not only
eliminates the BufferOverflowException but also reduces the memory usage in
many cases (4096 bytes is often too big for short rows, especially the ones
that do not contain any TEXT value).
> java.nio.BufferOverflowException occurs when the query includes an order by
> clause on a TEXT column
> ---------------------------------------------------------------------------------------------------
>
> Key: TAJO-666
> URL: https://issues.apache.org/jira/browse/TAJO-666
> Project: Tajo
> Issue Type: Bug
> Reporter: Jihoon Son
>
> See the title.
> The query and the error log of the query master are as follows.
> The 'method' column type is TEXT.
> {noformat}
> tajo> select method, sum(totaltime) from logs_20130404 group by method order
> by method;
> 2014-03-06 11:26:03,432 ERROR querymaster.QueryUnitAttempt
> (QueryUnitAttempt.java:transition(409)) - FROM xxx.xxx.xxx.xxx >>
> java.nio.BufferOverflowException
> at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:183)
> at java.nio.ByteBuffer.put(ByteBuffer.java:832)
> at
> org.apache.tajo.storage.RowStoreUtil$RowStoreEncoder.toBytes(RowStoreUtil.java:181)
> at
> org.apache.tajo.storage.index.bst.BSTIndex$BSTIndexWriter.writeHeader(BSTIndex.java:168)
> at
> org.apache.tajo.storage.index.bst.BSTIndex$BSTIndexWriter.close(BSTIndex.java:187)
> at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:237)
> at
> org.apache.tajo.engine.planner.physical.RangeShuffleFileWriteExec.close(RangeShuffleFileWriteExec.java:123)
> at org.apache.tajo.worker.Task.run(Task.java:385)
> at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:395)
> at java.lang.Thread.run(Thread.java:744)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
