[
https://issues.apache.org/jira/browse/HBASE-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122878#comment-15122878
]
Anoop Sam John commented on HBASE-15180:
----------------------------------------
You mean to make Codec decoder to work over a byte[] also rather than on
InputStream? That is nice. Why I did not go through that path is my next work
is in E2E off heap in write path also. So when the write req comes, we might
be reading it into an off heap BB (from a pool). So then byte[] based decoding
is not possible. May be BB based then. Again am trying to experiment with
reading the req not just into one single large BB. From the pool we might be
getting fixed sized smaller BBs (Say 64 KB or so) And we can read in to those
many BBs. And the CellScanner need to work on a set of BBs (Like the
MultiByteBuff stuff) Then again even this BB based API is an issue..
Continue with an InputStream based API gives us the freedom of experimenting
with this different data structures.
bq.Should we default to MSLAB for good? I don't think anybody runs with MSLAB
off.
Ya we have MSALB enabled by default. I agree that doing the MSLAB check in RPC
layer looks ugly. Wanted to avoid we refer to the req read byte[] (from
memstore cells) when some one turns MSLAB off. So what do you say? Remove
this check?
bq.Can the byte[4]'s be statically allocated?
No we can not. There will be many parallel req processing and many instances of
this InputStream in action. On one instance only we dont have multi threaded
reads. Making it static make all IS instances refer the same byte [] !
> Reduce garbage created while reading Cells from Codec Decoder
> -------------------------------------------------------------
>
> Key: HBASE-15180
> URL: https://issues.apache.org/jira/browse/HBASE-15180
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-15180.patch
>
>
> In KeyValueDecoder#parseCell (Default Codec decoder) we use
> KeyValueUtil#iscreate to read cells from the InputStream. Here we 1st create
> a byte[] of length 4 and read the cell length and then an array of Cell's
> length and read in cell bytes into it and create a KV.
> Actually in server we read the reqs into a byte[] and CellScanner is created
> on top of a ByteArrayInputStream on top of this. By default in write path, we
> have MSLAB usage ON. So while adding Cells to memstore, we will copy the Cell
> bytes to MSLAB memory chunks (default 2 MB size) and recreate Cells over that
> bytes. So there is no issue if we create Cells over the RPC read byte[]
> directly here in Decoder. No need for 2 byte[] creation and copy for every
> Cell in request.
> My plan is to make a Cell aware ByteArrayInputStream which can read Cells
> directly from it.
> Same Codec path is used in client side also. There better we can avoid this
> direct Cell create and continue to do the copy to smaller byte[]s path. Plan
> to introduce some thing like a CodecContext associated with every Codec
> instance which can say the server/client context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
