[
https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133706#comment-15133706
]
ramkrishna.s.vasudevan commented on HBASE-15205:
------------------------------------------------
I have a query here. In order to make things simple - Does it makes sense to
add the scope of all the families in the table descriptor which are NOT having
default replication scope into every WALKey?
Currently we are iterating the cells in a WALEdit and get every family in that
cell - compare it with the HTD families scope and then add those scopes which
are NOT having default scope. (as a map).
Again while doing the filtering in ScopeWalFilter - we get the scope from the
WALKey, iterate over the Cells in the WALEdit, check the family of the cells -
if it is not in the scope map of the WALKey or if it is having default scope
(we wont have this case as we don't write families with scope default) - we
remove those cells.
So instead while actually adding the scope for every WALKey during WAL append,
can we just add all those families with out default replication scope per
WALKey? The disadvantage is that suppose a Cell is having only one family and
it has default scope still we will be writing other families which has non
default scope but the advantage in terms of write path is that just have your
scopes calculated once (may be in HRegion) and just use that every where and
that will make your write path faster because we need not check every family in
the cells and see if it is scope is default or not.
If we don't need to do that way due to extra space - then we could have our
scopes evaluated once per HRegion and in every append we may still need to
check the family of every cell and see the scope of that family in our
evaluated scope map and then create a new set of map which has scopes of the
families that are not having default scope.
> Do not find the replication scope for every WAL#append()
> --------------------------------------------------------
>
> Key: HBASE-15205
> URL: https://issues.apache.org/jira/browse/HBASE-15205
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: ScopeWALEdits.jpg
>
>
> After the byte[] and char[] the other top contributor for lot of GC (though
> it is only 2.86%) is the UTF_8.newDecoder.
> This happens because for every WAL append we try to calculate the replication
> scope associate with the families associated with the TableDescriptor. I
> think per WAL append doing this is very costly and creates lot of garbage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
