[
https://issues.apache.org/jira/browse/HBASE-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15204:
-------------------------------------------
Attachment: HBASE-15204.patch
WAlEdit_add_allocation_after_patch.jpg
WAlEdit_add_allocation.jpg
Just attaching a first cut version of the patch to get feedback. We can also
estimate the WALEdit size based on the number of Cells added to WALEdits by the
CP. That is also covered here. But in order to pass the cellcount either we
need to change the signature of batchMutate in Region or need to directly call
the new batchMutate API with CellCount in HRegion directly.
Attached jpgs shows the allocation size and count just for this Arraylist to
grow.
> Try to estimate the cell count for adding into WALEdit
> ------------------------------------------------------
>
> Key: HBASE-15204
> URL: https://issues.apache.org/jira/browse/HBASE-15204
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15204.patch, WAlEdit_add_allocation.jpg,
> WAlEdit_add_allocation_after_patch.jpg
>
>
> The write path profiling shows that when we try to add Cells to WALEdits we
> try to do a lot of Array copy inorder to grow the Arraylist backing the
> WALEdits. In a simple one min profiling of the write path with 50 YCSB
> threads shows around 261MB of allocation done for the Array copy to happen.
> We can try to avoid that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
