[
https://issues.apache.org/jira/browse/HBASE-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149932#comment-15149932
]
Anoop Sam John commented on HBASE-15180:
----------------------------------------
2 reasons why did not go that way
1. Codec being non private if we add new methods, users may have to change in
their code when use new version.. Can we add method to Codec interface in a
minor/patch release? If we do fulfill our need with out changing the
interface, I thought that is better,
2. Ya we have BB now.. Was thinking of doing PoC with reading data into more
than one buffer when we use these buffers from a pool (We have BBBpool now).
We know the size need to read the reqs. As in BBBPool we can make have running
avg of size and create bigger buffers. But on demand creation of buffers off
heap may create some -ve impact on full GC? Any way all these are to be
tested. Said that, continue with IS instead of a data structure like
byte[]/BB is giving us all sort of flexibility.
> Reduce garbage created while reading Cells from Codec Decoder
> -------------------------------------------------------------
>
> Key: HBASE-15180
> URL: https://issues.apache.org/jira/browse/HBASE-15180
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Affects Versions: 0.98.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0, 1.2.1
>
> Attachments: HBASE-15180.patch, HBASE-15180_V2.patch,
> HBASE-15180_V4.patch
>
>
> In KeyValueDecoder#parseCell (Default Codec decoder) we use
> KeyValueUtil#iscreate to read cells from the InputStream. Here we 1st create
> a byte[] of length 4 and read the cell length and then an array of Cell's
> length and read in cell bytes into it and create a KV.
> Actually in server we read the reqs into a byte[] and CellScanner is created
> on top of a ByteArrayInputStream on top of this. By default in write path, we
> have MSLAB usage ON. So while adding Cells to memstore, we will copy the Cell
> bytes to MSLAB memory chunks (default 2 MB size) and recreate Cells over that
> bytes. So there is no issue if we create Cells over the RPC read byte[]
> directly here in Decoder. No need for 2 byte[] creation and copy for every
> Cell in request.
> My plan is to make a Cell aware ByteArrayInputStream which can read Cells
> directly from it.
> Same Codec path is used in client side also. There better we can avoid this
> direct Cell create and continue to do the copy to smaller byte[]s path. Plan
> to introduce some thing like a CodecContext associated with every Codec
> instance which can say the server/client context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
