[ 
https://issues.apache.org/jira/browse/HBASE-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570404#comment-15570404
 ] 

Hadoop QA commented on HBASE-16721:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m 39s 
{color} | {color:red} root in branch-1.1 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 6s 
{color} | {color:red} hbase-server in branch-1.1 failed with JDK v1.8.0_101. 
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 3m 45s 
{color} | {color:red} hbase-server in branch-1.1 failed with JDK v1.7.0_80. 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
9s {color} | {color:green} branch-1.1 passed {color} |
| {color:red}-1{color} | {color:red} mvneclipse {color} | {color:red} 0m 14s 
{color} | {color:red} hbase-server in branch-1.1 failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 6s 
{color} | {color:red} hbase-server in branch-1.1 failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 12s 
{color} | {color:red} hbase-server in branch-1.1 failed with JDK v1.8.0_101. 
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 6s 
{color} | {color:red} hbase-server in branch-1.1 failed with JDK v1.7.0_80. 
{color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 6s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 5s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_101. 
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 5s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0_101. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 6s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_80. 
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 6s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.7.0_80. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
7s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvneclipse {color} | {color:red} 0m 6s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
11m 55s {color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 7s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 8s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 6s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_101. 
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 7s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_80. 
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 8s {color} | 
{color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
40s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 28s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:35e2245 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12833008/hbase-16721_addendum2.branch-1.1.patch
 |
| JIRA Issue | HBASE-16721 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 8f73a74263de 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1.1 / edac952 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_101 
/usr/lib/jvm/java-7-oracle:1.7.0_80 |
| mvninstall | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/branch-mvninstall-root.txt
 |
| compile | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/branch-compile-hbase-server-jdk1.8.0_101.txt
 |
| compile | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/branch-compile-hbase-server-jdk1.7.0_80.txt
 |
| mvneclipse | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/branch-mvneclipse-hbase-server.txt
 |
| findbugs | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/branch-findbugs-hbase-server.txt
 |
| javadoc | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/branch-javadoc-hbase-server-jdk1.8.0_101.txt
 |
| javadoc | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/branch-javadoc-hbase-server-jdk1.7.0_80.txt
 |
| mvninstall | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-mvninstall-hbase-server.txt
 |
| compile | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_101.txt
 |
| javac | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_101.txt
 |
| compile | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_80.txt
 |
| javac | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_80.txt
 |
| mvneclipse | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-mvneclipse-hbase-server.txt
 |
| hbaseprotoc | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-hbaseprotoc-hbase-server.txt
 |
| findbugs | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-findbugs-hbase-server.txt
 |
| javadoc | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-javadoc-hbase-server-jdk1.8.0_101.txt
 |
| javadoc | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-javadoc-hbase-server-jdk1.7.0_80.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/3971/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Concurrency issue in WAL unflushed seqId tracking
> -------------------------------------------------
>
>                 Key: HBASE-16721
>                 URL: https://issues.apache.org/jira/browse/HBASE-16721
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>    Affects Versions: 1.0.0, 1.1.0, 1.2.0
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>            Priority: Critical
>             Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.4, 1.1.8
>
>         Attachments: hbase-16721_addendum.patch, 
> hbase-16721_addendum2.branch-1.1.patch, hbase-16721_v1.branch-1.patch, 
> hbase-16721_v2.branch-1.patch, hbase-16721_v2.master.patch
>
>
> I'm inspecting an interesting case where in a production cluster, some 
> regionservers ends up accumulating hundreds of WAL files, even with force 
> flushes going on due to max logs. This happened multiple times on the 
> cluster, but not on other clusters. The cluster has periodic memstore flusher 
> disabled, however, this still does not explain why the force flush of regions 
> due to max limit is not working. I think the periodic memstore flusher just 
> masks the underlying problem, which is why we do not see this in other 
> clusters. 
> The problem starts like this: 
> {code}
> 2016-09-21 17:49:18,272 INFO  [regionserver//10.2.0.55:16020.logRoller] 
> wal.FSHLog: Too many wals: logs=33, maxlogs=32; forcing flush of 1 
> regions(s): d4cf39dc40ea79f5da4d0cf66d03cb1f
> 2016-09-21 17:49:18,273 WARN  [regionserver//10.2.0.55:16020.logRoller] 
> regionserver.LogRoller: Failed to schedule flush of 
> d4cf39dc40ea79f5da4d0cf66d03cb1f, region=null, requester=null
> {code}
> then, it continues until the RS is restarted: 
> {code}
> 2016-09-23 17:43:49,356 INFO  [regionserver//10.2.0.55:16020.logRoller] 
> wal.FSHLog: Too many wals: logs=721, maxlogs=32; forcing flush of 1 
> regions(s): d4cf39dc40ea79f5da4d0cf66d03cb1f
> 2016-09-23 17:43:49,357 WARN  [regionserver//10.2.0.55:16020.logRoller] 
> regionserver.LogRoller: Failed to schedule flush of 
> d4cf39dc40ea79f5da4d0cf66d03cb1f, region=null, requester=null
> {code}
> The problem is that region {{d4cf39dc40ea79f5da4d0cf66d03cb1f}} is already 
> split some time ago, and was able to flush its data and split without any 
> problems. However, the FSHLog still thinks that there is some unflushed data 
> for this region. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to