[
https://issues.apache.org/jira/browse/HBASE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207084#comment-15207084
]
Vladimir Rodionov commented on HBASE-15493:
-------------------------------------------
[~stack], I have spent many hours, during last several days, analyzing memory
allocation and hot spots we have in a write path. Low hanging fruits, such as
HBASE-15479 are now gone. Now, to reduce memory allocation by 10% we need to
fix 3-5, may be more hot spots. If I will be doing this in a "right" way -
adding all complexity to smart HBase, by allowing clueless users to get all
benefits for free - I will be spending several days per tickets. I do not have
that much time. That is why I decided to delegate all the tuning to advanced
users. AutoSmart feature is complex and not worth spending time imo.
Spoiler,
The only other low hanging fruit remained is protobuf SerDe.
> Default ArrayList size may not be optimal for Mutation
> ------------------------------------------------------
>
> Key: HBASE-15493
> URL: https://issues.apache.org/jira/browse/HBASE-15493
> Project: HBase
> Issue Type: Improvement
> Components: Client, regionserver
> Affects Versions: 2.0.0
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-15493-v1.patch, HBASE-15493-v2.patch
>
>
> {code}
> List<Cell> getCellList(byte[] family) {
> List<Cell> list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<Cell>();
> }
> return list;
> }
> {code}
> Creates list of size 10, this is up to 80 bytes per column family in mutation
> object.
> Suggested:
> {code}
> List<Cell> getCellList(byte[] family) {
> List<Cell> list = this.familyMap.get(family);
> if (list == null) {
> list = new ArrayList<Cell>(CELL_LIST_INITIAL_CAPACITY);
> }
> return list;
> }
> {code}
> CELL_LIST_INITIAL_CAPACITY = 2 in the patch, this is debatable. For mutation
> where every CF has 1 cell, this gives decent reduction in memory allocation
> rate in both client and server during write workload. ~2%, not a big number,
> but as I said, already, memory optimization will include many small steps.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
