[jira] [Commented] (HBASE-16524) Clean procedure wal periodically instead of on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468894#comment-15468894 ] Matteo Bertozzi commented on HBASE-16524: - Instead of scanning all the wals every time, we can rely on the insert/update/delete events we have. and since we want to delete the wals in order we can keep track of what is "holding" that wal, and take a hit on scanning all the trackers only when we remove the first log in the queue. e.g. WAL-1 [1, 2] WAL-2 [1] -> "[2] is holding WAL-1" WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" > Clean procedure wal periodically instead of on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-16524.master.001.patch, flame1.svg > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Clean procedure wal periodically instead of on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468754#comment-15468754 ] Appy commented on HBASE-16524: -- Thinking about it, that should be easy to do. In our current logic, we basically do this: 1. make a copy of current global tracker, say T. 2. Iterate over log files: - keep the file if contain an update to proc P which is not marked deleted in T. - Delete P from T so we don't hold another older log file because of it. We can simply add step 0, which marks all recently updated procs as deleted in T. Sounds good? > Clean procedure wal periodically instead of on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-16524.master.001.patch, flame1.svg > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Clean procedure wal periodically instead of on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457397#comment-15457397 ] Matteo Bertozzi commented on HBASE-16524: - before adding conditions to reduce the number of times we call removeInactiveLogs(), let's try to improve the algorithm. executing every roll does not provide the result we want. e.g. WAL-1: [1, 2] Current: [] as soon we update [1, 2] we can remove WAL-1 e.g. WAL-1: [1], WAL-2: [2] as soon we update [1, 2] we can remove WAL-1 and WAL-2 > Clean procedure wal periodically instead of on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-16524.master.001.patch, flame1.svg > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Clean procedure wal periodically instead of on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456914#comment-15456914
]
Appy commented on HBASE-16524:
--
Attaching flame graph form a run of {{./bin/hbase
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALPerformanceEvaluation
-procs 50}} (captured 15sec).
As expected, bottleneck is subtraction of two bitmaps.
The lowest hanging fruit here is to not call removeInactiveLogs() on every sync.
Also, instead of waiting for a predetermined (and randomly) chosen time, it'll
be better to call the cleaner once every wal roll. Only the written wals (not
the active one) are used to decided the obsolete ones, therefore, calling
removeInactiveLogs() repeatedly until the active log is rolled won't mark any
new logs for removal.
> Clean procedure wal periodically instead of on every sync
> -
>
> Key: HBASE-16524
> URL: https://issues.apache.org/jira/browse/HBASE-16524
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Attachments: HBASE-16524.master.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Clean procedure wal periodically instead of on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450511#comment-15450511
]
Hadoop QA commented on HBASE-16524:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 9s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 54s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 2s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-30 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12826269/HBASE-16524.master.001.patch
|
| JIRA Issue | HBASE-16524 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 4be0be406d2e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-16524) Clean procedure wal periodically instead of on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450442#comment-15450442 ] Matteo Bertozzi commented on HBASE-16524: - let's try to keep 001 as last resource. we can try to think about a couple of other stuff first like find where is the bottleneck of the current removeInactiveLogs() and see if we can improve that. and also I'd will be nice if we can predict exactly when is the best time to call removeInactiveLogs() instead of just guessing based on time. we should have all the information we need to spot the pattern to know when the log should be removed. > Clean procedure wal periodically instead of on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-16524.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
