[
https://issues.apache.org/jira/browse/HBASE-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135646#comment-15135646
]
ramkrishna.s.vasudevan commented on HBASE-15204:
------------------------------------------------
bq.the return is orthogonal to why we call the CP. We are now dependent on CP
implementation to do work for us.
[~saint....@gmail.com] Not very clear on this part. If CP is returning some
mutations - we explicitly add that to a WALEdit. I am just counting if
something is there. That is only to get an exact count because that will also
be added to the WALEdit that we create in doMiniBatchMutation.
bq.It is kind of odd that doMiniBatch takes a cell count when the other param
is a fat batchOp with description of what the edit its. If cell count should be
anywhere, it should be in batchop?
I can remove that if we don't want that. Ya in batchOp we could add the
Cellcount but many of the mutations may not be applied in that batch. But if we
don't want the cellCountFromCP to be passed then am fine with it too. I thought
since we can use it and get the exact count it is preferable to use it.
bq.Do we have to add a Cell at a time? Can we not pass the whole list in one go?
LEt me check this out. Since WALEdit was backed by an Arraylist I thought if we
can size that backing list properly we are fine with it.
bq.I appreciate the savings in garbage but perhaps we can do it in a cleaner
way.
Okie. I can revert this patch and see what else can be done.
> Try to estimate the cell count for adding into WALEdit
> ------------------------------------------------------
>
> Key: HBASE-15204
> URL: https://issues.apache.org/jira/browse/HBASE-15204
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15204.patch, HBASE-15204_1.patch,
> HBASE-15204_1.patch, WAlEdit_add_allocation.jpg,
> WAlEdit_add_allocation_after_patch.jpg
>
>
> The write path profiling shows that when we try to add Cells to WALEdits we
> try to do a lot of Array copy inorder to grow the Arraylist backing the
> WALEdits. In a simple one min profiling of the write path with 50 YCSB
> threads shows around 261MB of allocation done for the Array copy to happen.
> We can try to avoid that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
