[
https://issues.apache.org/jira/browse/HBASE-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659860#action_12659860
]
stack commented on HBASE-1097:
------------------------------
Good. Let me make sure there are no migration implications before committing
above patch.
> SequenceFile.Reader keeps around buffer whose size is that of largest item
> read -> results in lots of dead heap
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-1097
> URL: https://issues.apache.org/jira/browse/HBASE-1097
> Project: Hadoop HBase
> Issue Type: Bug
> Environment: apurtell 25 node TRUNK on hadoop 0.18.1 cluster
> Reporter: stack
> Fix For: 0.19.0
>
> Attachments: mf.patch
>
>
> Andrew is OOMEing again. Looking at some of his heaps, I can count Reader
> with DataOutputBuffers of ~600MB in a 2G heap. Testing I see that the
> DataOutputBuffer allocated at head of Mapfile.Reader is reused when we call
> next, a reset is called. If I trace, the DataOutputBuffer has in it an
> internal Buffer class which is based on ByteArrayOutputStream. Reset of the
> DOB eventually goes through to the BAOS reset. This just sets the position.
> It keeps the buffer sized to whatever it grew to last time this BAOS was used
> (Figuring this was a little complicated by the fact that DOB does some fancy
> footwork in a reset override to avoid copies).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
