[
https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15151851#comment-15151851
]
ramkrishna.s.vasudevan commented on HBASE-15205:
------------------------------------------------
bq.From above review and still prevalent in v4 of the patch: "Sometimes you use
getScopes. Other times you use this.replicationScopes." Use the method always
I'd say.
Sorry for the oversight. I read it as I was using getScopes and also
getReplicationScopes. So in the last patch I unified and made
getreplicationScopes every where. Now I got what you said.
bq.In WALKey, can this be package private 421 public NavigableMap<byte[],
Integer> getReplicationScopes() {
This was not added by me. I only renamed and made it to getReplicationScopes
from getScopes. Anyway I can remove it.
bq. FSWALEntry has repilcation scopes added by this patch. You seem to be
duplicating scopes by adding to WALEdit replication scopes again? i.e. A
FSWALEntry has scopes recorded twice... once in FSWALEntry, and then again in
WALKey?
In the existing code the replicationScope was set in WALKey from the HTD. Since
we had removed that I wanted it to be passed over from FSWalEntry. Because
{code}
for (WALActionsListener i: listeners) {
i.visitLogEntryBeforeWrite(entry.getReplicationScope(),
entry.getKey(),
entry.getEdit());
}
{code}
This method FSHLog.append(FSWalEntry) accepts a WALEntry. So you want me to
change in such a way that during WALKey construction itself pass the
replicationScope? ACtually can do that but the WALKey has an explicit setter
for replicationScope (already available) just because for DLR and in region
replicas case you don want the replication scope to be added to the WALKey. Now
if we use a constructor then may be though we pass the replicationScope during
construction of WALKey again we may need to set it to null if the WALEdit says
it is for region replica or during replay. Also this way is better considering
the case when there is only bulk load replication then at that time there are
chances that when there are no bulk load entries in WAL there is nothing to be
replicated and hence we need not set the replicaitonScope to the WALKey. Hence
passing on the replicationScope to the WALKey thro FSWALEntry seems fine to me.
bq.There is convention for naming of keys used internally only. IIRC, you
prefix with an '_'.
Before updating the next patch you still want me to go with the way of setting
the scope on HTD and use the same HTD to retrive the scope per family? But I
fear it wont serve the purpose of short living objects because the
HTD.setValue() will convert things to Bytes. So every time we add it to the PB
serde
{code}
builder.addScopes(FamilyScope.newBuilder()
.setFamily(family).setScopeType(ScopeType.valueOf(e.getValue())));
{code}
We need to get the value from that HTD map and to convert it into Integer again
we need to do Bytes.toInt(byte[]).
> Do not find the replication scope for every WAL#append()
> --------------------------------------------------------
>
> Key: HBASE-15205
> URL: https://issues.apache.org/jira/browse/HBASE-15205
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15205.patch, HBASE-15205_1.patch,
> HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch,
> ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg
>
>
> After the byte[] and char[] the other top contributor for lot of GC (though
> it is only 2.86%) is the UTF_8.newDecoder.
> This happens because for every WAL append we try to calculate the replication
> scope associate with the families associated with the TableDescriptor. I
> think per WAL append doing this is very costly and creates lot of garbage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
