[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks all for the reviews. Will keep watching the builds to be on the safer side. Committed to master. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_11.patch, > HBASE-15205_12.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_12.patch Patch for final QA run > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_11.patch, > HBASE-15205_12.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_11.patch, > HBASE-15205_12.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_11.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch, > HBASE-15205_6.patch, HBASE-15205_6.patch, HBASE-15205_7.patch, > HBASE-15205_8.patch, HBASE-15205_9.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) Also there is one checkstyle comment which talks about number of lines being > 150. I am not addressing that in this patch as it will be a bigger change. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_11.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch, > HBASE-15205_6.patch, HBASE-15205_6.patch, HBASE-15205_7.patch, > HBASE-15205_8.patch, HBASE-15205_9.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_11.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch, > HBASE-15205_6.patch, HBASE-15205_6.patch, HBASE-15205_7.patch, > HBASE-15205_8.patch, HBASE-15205_9.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_11.patch Updated checkstyle and whitespaces in test case. Will wait for one more QA run if the test could run once again (Metrics related test cases are not stable). > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_11.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch, > HBASE-15205_6.patch, HBASE-15205_6.patch, HBASE-15205_7.patch, > HBASE-15205_8.patch, HBASE-15205_9.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_10.patch PAtch correcting the test case. Checkstyle comments am not sure what is as the link is not working. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_10.patch, HBASE-15205_2.patch, > HBASE-15205_3.patch, HBASE-15205_4.patch, HBASE-15205_6.patch, > HBASE-15205_6.patch, HBASE-15205_7.patch, HBASE-15205_8.patch, > HBASE-15205_9.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_9.patch Rebased patch for QA. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, HBASE-15205_9.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[
https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15205:
---
Attachment: HBASE-15205_8.patch
Latest patch. Removed replicationScope from FSWALEntry. WALKey construction
itself will now have replicationScope. As I said earlier since we are now
passing the scope during WALKey construction itself now it is upto us to
nullify the scopes in logKey when ever it is not needed. I felt that was not
looking clean. Still we are removing HTD from WAL#append() and things work as
in previous patches.
Now we will call this
{code}
+if ((!replicationForBulkLoadEnabled && !foundOtherEdits) ||
logEdit.isReplay()) {
+ logKey.serializeReplicationScope(false);
}
{code}
instead of setReplciationScope(). Inside WALKEy#serializeReplicationScope() we
will just nullify the replicationScope that was passed during construction.
> Do not find the replication scope for every WAL#append()
>
>
> Key: HBASE-15205
> URL: https://issues.apache.org/jira/browse/HBASE-15205
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15204_6.patch, HBASE-15205.patch,
> HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch,
> HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch,
> HBASE-15205_7.patch, HBASE-15205_8.patch, ScopeWALEdits.jpg,
> ScopeWALEdits_afterpatch.jpg
>
>
> After the byte[] and char[] the other top contributor for lot of GC (though
> it is only 2.86%) is the UTF_8.newDecoder.
> This happens because for every WAL append we try to calculate the replication
> scope associate with the families associated with the TableDescriptor. I
> think per WAL append doing this is very costly and creates lot of garbage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, HBASE-15205_8.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_7.patch bq.FSWALEntry still has replicationScope. That intentional? I thought only WALKey was going to have scopes? Yes Stack as said in my earlier comments I was not able to directly pass it to WALKey on construction. Hence using the replicationScope in FSWAlEntry to be passed directly to HLog.append() and use that scope to be set on the WALKey. Corrected the failing testcase. Patch for QA before commit. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > HBASE-15205_7.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_6.patch Updated patch for QA. [~saint@gmail.com] I wanted the getReplicationScope() to be public in HRegion because it is used in a test case TestIOFencing which was created WALMarkers. Hence I had to mark it as VisibileForTesting. The WALKey#getReplicationScope() is getting used in ReplicationProtobufUtil, hence we need that too. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, HBASE-15205_6.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_6.patch I think the patch number was wrong. Reattaching it. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, HBASE-15205_6.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15204_6.patch Correcting the checkstyle comments. The FSWALEntry and WALKey having replicationscopes is still there in this. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15204_6.patch, HBASE-15205.patch, > HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch, > HBASE-15205_4.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Hadoop Flags: Reviewed Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_4.patch I think the change to Region. java itself was not necessary. Where ever we are getting the replicationScope() it is all HRegion only. I think this patch should be fine now. Same as previous patch without the changes to Region.java > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, HBASE-15205_4.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_3.patch Updated patch. Corrects the failing test cases. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, HBASE-15205_3.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Patch Available (was: Open) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Status: Open (was: Patch Available) > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_2.patch Updated patch. Corrects the checkstyle and findbugs comments. It was mostly related with unused variables and unused imports. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > HBASE-15205_2.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: HBASE-15205_1.patch This patch does the following -> collects the scope per family (that has non default scope) during HRegion Construction and that scope is passed over to WAL.append(). -> Now we don't iterate and find the scope per Cell in WAlEdit and create local map which is added to WALKey that in turn gets persisted in PB. Instead we directly set the scope that is passed over from HRegion per WALKey. -> This avoids lot of short lived objects like the String created while fetching the scope from HTD, the CellUtil.cloneFamily() that we do per cell for getting the scope and the local map creation for every WAL#append call. -> The patch is big because of the change in WAL.append() API signature. -> I think the replication scope for a family can only be changed by alter_table command. For which we have to disable and enable the table. So this should workout. Is there any chance that the replication scope of a table can be changed without using the alter_table command? > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, HBASE-15205_1.patch, > ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: ScopeWALEdits_afterpatch.jpg HBASE-15205.patch The patch seems bloated mainly because of lot of test case changes as the WAL.append() signature is changed. The HTD was being passed to the WAL.append() just to get the scope of the column families from it. Ideally it can be obtained once per HRegion and pass that to the FSWALEntry and use it from there. First thought of adding some getScope() directly under HTD and return a Map from that but that is not straight forward because HTD is public and also should we populate the map on getScope() call every time when it is called? Also there was already a TODO asking why HTD is getting passed to WAL.append() so I thought it made sense to remove it. But this patch avoids lot of garbage. One more thing to see is that since replication deals with families we are bound to do lot of copy to store the family byte[] in some maps. If we could avoid that too then Offheaping work would be greatly benefited. Will try addressing that later. Patch for QA run. Will update the patch based on this QA results. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15205.patch, ScopeWALEdits.jpg, > ScopeWALEdits_afterpatch.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
[ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15205: --- Attachment: ScopeWALEdits.jpg This pic shows how the scopeOfWalEdits creates lot of objects in a 1 min profile run. > Do not find the replication scope for every WAL#append() > > > Key: HBASE-15205 > URL: https://issues.apache.org/jira/browse/HBASE-15205 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: ScopeWALEdits.jpg > > > After the byte[] and char[] the other top contributor for lot of GC (though > it is only 2.86%) is the UTF_8.newDecoder. > This happens because for every WAL append we try to calculate the replication > scope associate with the families associated with the TableDescriptor. I > think per WAL append doing this is very costly and creates lot of garbage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
