[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15798436#comment-15798436
]
Appy commented on HBASE-16524:
--
Reviewed it just now. Looks good to me.
Should probably change the functions {{insert(BitSetNode node, long procId)}}
and others to {{insert(long procId, BitSetNode nodeHint)}} because the first
argument isn't mandatory, instead it's a hint to avoid lookup into map
repeatedly.
One thing worth mentioning, wal cleanup logic gets triggered on every wal sync,
(WalProcedureStore#syncLoop), so the effort here is to make it quick.
One case here is, if a procedure gets stuck and was last synced a while back,
we keep holding intermediate wals. Once the stuck procedure finishes, it'll
take some time to cleanup wals since each wal will be sequentially processed as
'holding' wall. This will delay the next sync a bit.
Just a heads up in case we see issues related to wal.
> Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
> -
>
> Key: HBASE-16524
> URL: https://issues.apache.org/jira/browse/HBASE-16524
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Appy
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch,
> HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch,
> HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg
>
>
> Fix performance regression introduced by HBASE-16094.
> Instead of scanning all the wals every time, we can rely on the
> insert/update/delete events we have.
> and since we want to delete the wals in order we can keep track of what is
> "holding" that wal, and take a hit on scanning all the trackers only when we
> remove the first log in the queue.
> e.g.
> WAL-1 [1, 2]
> WAL-2 [1] -> "[2] is holding WAL-1"
> WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781939#comment-15781939 ] Hudson commented on HBASE-16524: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2211 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2211/]) HBASE-16524 Procedure v2 - Compute WALs cleanup on wal modification and (stack: rev 319ecd867a2903c4ce03c38f6ffec62ada1a6049) * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java * (edit) hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/TestProcedureStoreTracker.java > Procedure v2 - Compute WALs cleanup on wal modification and not on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Appy >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch, > HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch, > HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg > > > Fix performance regression introduced by HBASE-16094. > Instead of scanning all the wals every time, we can rely on the > insert/update/delete events we have. > and since we want to delete the wals in order we can keep track of what is > "holding" that wal, and take a hit on scanning all the trackers only when we > remove the first log in the queue. > e.g. > WAL-1 [1, 2] > WAL-2 [1] -> "[2] is holding WAL-1" > WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781605#comment-15781605 ] Stephen Yuan Jiang commented on HBASE-16524: Agree > Procedure v2 - Compute WALs cleanup on wal modification and not on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Appy >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch, > HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch, > HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg > > > Fix performance regression introduced by HBASE-16094. > Instead of scanning all the wals every time, we can rely on the > insert/update/delete events we have. > and since we want to delete the wals in order we can keep track of what is > "holding" that wal, and take a hit on scanning all the trackers only when we > remove the first log in the queue. > e.g. > WAL-1 [1, 2] > WAL-2 [1] -> "[2] is holding WAL-1" > WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781608#comment-15781608 ] Stephen Yuan Jiang commented on HBASE-16524: Agree! > Procedure v2 - Compute WALs cleanup on wal modification and not on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Appy >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch, > HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch, > HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg > > > Fix performance regression introduced by HBASE-16094. > Instead of scanning all the wals every time, we can rely on the > insert/update/delete events we have. > and since we want to delete the wals in order we can keep track of what is > "holding" that wal, and take a hit on scanning all the trackers only when we > remove the first log in the queue. > e.g. > WAL-1 [1, 2] > WAL-2 [1] -> "[2] is holding WAL-1" > WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781598#comment-15781598 ] stack commented on HBASE-16524: --- Lets commit. It is smarter tracking of the WAL files and a prerequisite for what follows. I'm sure we'll get to know this code more intimately in the next months. > Procedure v2 - Compute WALs cleanup on wal modification and not on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Appy >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch, > HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch, > HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg > > > Fix performance regression introduced by HBASE-16094. > Instead of scanning all the wals every time, we can rely on the > insert/update/delete events we have. > and since we want to delete the wals in order we can keep track of what is > "holding" that wal, and take a hit on scanning all the trackers only when we > remove the first log in the queue. > e.g. > WAL-1 [1, 2] > WAL-2 [1] -> "[2] is holding WAL-1" > WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781553#comment-15781553 ] Stephen Yuan Jiang commented on HBASE-16524: [~stack], this is the patch that I have least understanding (of the 4 recent infrastructure improvement patches from [~mbertozzi]). I tried to talk to [~mbertozzi] about it, but it was too late and he left for a long vacation. > Procedure v2 - Compute WALs cleanup on wal modification and not on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Appy >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch, > HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch, > HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg > > > Fix performance regression introduced by HBASE-16094. > Instead of scanning all the wals every time, we can rely on the > insert/update/delete events we have. > and since we want to delete the wals in order we can keep track of what is > "holding" that wal, and take a hit on scanning all the trackers only when we > remove the first log in the queue. > e.g. > WAL-1 [1, 2] > WAL-2 [1] -> "[2] is holding WAL-1" > WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781521#comment-15781521 ] stack commented on HBASE-16524: --- [~syuanjiang] You good w/ this going in sir? > Procedure v2 - Compute WALs cleanup on wal modification and not on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Appy >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch, > HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch, > HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg > > > Fix performance regression introduced by HBASE-16094. > Instead of scanning all the wals every time, we can rely on the > insert/update/delete events we have. > and since we want to delete the wals in order we can keep track of what is > "holding" that wal, and take a hit on scanning all the trackers only when we > remove the first log in the queue. > e.g. > WAL-1 [1, 2] > WAL-2 [1] -> "[2] is holding WAL-1" > WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781430#comment-15781430
]
Hadoop QA commented on HBASE-16524:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 22s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
6s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 4s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12844826/HBASE-16524.master.002.patch
|
| JIRA Issue | HBASE-16524 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e0b543f80d10 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5ffbd4a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5064/testReport/ |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5064/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
> -
>
> Key: HBASE-16524
> URL: https://issues.apache.org/jira/browse/HBASE-16524
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Appy
>Assignee: Matteo Bertozzi
>Priority:
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[ https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781343#comment-15781343 ] stack commented on HBASE-16524: --- I pushed a rebased patch. > Procedure v2 - Compute WALs cleanup on wal modification and not on every sync > - > > Key: HBASE-16524 > URL: https://issues.apache.org/jira/browse/HBASE-16524 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Appy >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16524-v2.patch, HBASE-16524-v3.patch, > HBASE-16524-v4.patch, HBASE-16524-v5.patch, HBASE-16524-v6.patch, > HBASE-16524.master.001.patch, HBASE-16524.master.002.patch, flame1.svg > > > Fix performance regression introduced by HBASE-16094. > Instead of scanning all the wals every time, we can rely on the > insert/update/delete events we have. > and since we want to delete the wals in order we can keep track of what is > "holding" that wal, and take a hit on scanning all the trackers only when we > remove the first log in the queue. > e.g. > WAL-1 [1, 2] > WAL-2 [1] -> "[2] is holding WAL-1" > WAL-3 [2] -> "WAL 1 can be removed, recompute what is holding WAL-2" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685660#comment-15685660
]
Hadoop QA commented on HBASE-16524:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 5s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 28s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 24s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839940/HBASE-16524-v6.patch |
| JIRA Issue | HBASE-16524 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 17135d2635b2 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b297f2d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4565/testReport/ |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4565/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
> -
>
> Key: HBASE-16524
> URL: https://issues.apache.org/jira/browse/HBASE-16524
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Appy
>Assignee: Matteo Bertozzi
>Priority: Minor
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685178#comment-15685178
]
Hadoop QA commented on HBASE-16524:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 7s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 6s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 10s {color}
| {color:red} hbase-procedure in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 14s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.procedure2.store.wal.TestWALProcedureStore |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839906/HBASE-16524-v5.patch |
| JIRA Issue | HBASE-16524 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 2ceb8944dbf4 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 3e9d1a1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4561/artifact/patchprocess/patch-unit-hbase-procedure.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4561/artifact/patchprocess/patch-unit-hbase-procedure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4561/testReport/ |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4561/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
>
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685000#comment-15685000
]
Hadoop QA commented on HBASE-16524:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 29s
{color} | {color:red} hbase-procedure generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 15s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 44s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-procedure |
| |
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(ProcedureStore$ProcedureLoader)
does not release lock on all exception paths At WALProcedureStore.java:on all
exception paths At WALProcedureStore.java:[line 351] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839897/HBASE-16524-v4.patch |
| JIRA Issue | HBASE-16524 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5ac3846f08c4 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3e9d1a1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4558/artifact/patchprocess/new-findbugs-hbase-procedure.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4558/testReport/ |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4558/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684798#comment-15684798
]
Hadoop QA commented on HBASE-16524:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 41s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 32s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 29s
{color} | {color:red} hbase-procedure generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 17s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 15s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-procedure |
| |
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(ProcedureStore$ProcedureLoader)
does not release lock on all exception paths At WALProcedureStore.java:on all
exception paths At WALProcedureStore.java:[line 351] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12839884/HBASE-16524-v3.patch |
| JIRA Issue | HBASE-16524 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b9b12f8eb047 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3e9d1a1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4557/artifact/patchprocess/new-findbugs-hbase-procedure.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4557/testReport/ |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4557/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HBASE-16524) Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
[
https://issues.apache.org/jira/browse/HBASE-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651131#comment-15651131
]
Hadoop QA commented on HBASE-16524:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 16s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 52s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12838178/HBASE-16524-v2.patch |
| JIRA Issue | HBASE-16524 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ed9ead4debef 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3b63f70 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4403/testReport/ |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4403/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Procedure v2 - Compute WALs cleanup on wal modification and not on every sync
> -
>
> Key: HBASE-16524
> URL: https://issues.apache.org/jira/browse/HBASE-16524
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Appy
>Assignee: Matteo Bertozzi
>Priority: Minor
>
