[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088480#comment-16088480 ] Phil Yang commented on HBASE-18366: --- Thanks [~uagashe]. Reconsidered HBASE-17931, checkIfShouldMoveSystemRegionAsync may only needed when there is a region server startup. When RS stop or HMaster startup we can use a suitable Procedure to make sure we keep system tables in highest version RSs , right? > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[
https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088470#comment-16088470
]
Hadoop QA commented on HBASE-16488:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
56s{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 25s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 20s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}124m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestClientScannerRPCTimeout |
| | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery |
| | hadoop.hbase.regionserver.TestRSKilledWhenInitializing |
| | hadoop.hbase.master.TestMasterFailover |
| Timed out junit tests |
org.apache.hadoop.hbase.client.TestTableSnapshotScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:395d9a0 |
| JIRA Issue | HBASE-16488 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877434/HBASE-16488.v10-branch-1.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088439#comment-16088439 ] Hudson commented on HBASE-18366: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3377 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3377/]) HBASE-18366 Fix flaky test (stack: rev a9352fe9564f57658f918ef63ac4a5ebdc17fb45) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestServerCrashProcedure.java > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18229) create new Async Split API to embrace AM v2
[ https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088438#comment-16088438 ] Hudson commented on HBASE-18229: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3377 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3377/]) HBASE-18229: create new Async Split API to embrace AM v2 (stack: rev 353627b39de73020dd2448b54c0f13f6902b19bf) * (edit) hbase-shell/src/test/ruby/hbase/admin_test.rb * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MasterProtos.java * (edit) hbase-protocol-shaded/src/main/protobuf/Master.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/Util.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ClientProtos.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestIncrementalBackup.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin1.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java > create new Async Split API to embrace AM v2 > --- > > Key: HBASE-18229 > URL: https://issues.apache.org/jira/browse/HBASE-18229 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0-alpha-2 > > Attachments: HBase-18229-master-v1.patch, > hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch, > hbase-18229-master-v4.patch, hbase-18229-master-v5.patch, > hbase-18229-master-v6.patch, hbase-18229-master-v7.patch, > hbase-18229-master-v7.patch, hbase-18229-master-v8.patch > > > See HBASE-18105 for related information, this jira will change the logic of > Path 1 in splitProcedure, the execution path will be: > *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> > MasterRpcServices.splitRegion -> HMaster.splitRegion-> > AssignmentManager.submitProcedure* > Master Will no longer ask Rs and then Rs turn around to ask master to do the > split operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088405#comment-16088405 ] Hudson commented on HBASE-18366: FAILURE: Integrated in Jenkins build HBase-2.0 #175 (See [https://builds.apache.org/job/HBase-2.0/175/]) HBASE-18366 Fix flaky test (stack: rev 4663d7b9a741f3e374510880d2c47a549591da46) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestServerCrashProcedure.java > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-16488: --- Attachment: HBASE-16488.v10-branch-1.patch > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.0.0, 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Attachments: HBASE-16488.v10-branch-1.patch, > HBASE-16488.v1-branch-1.patch, HBASE-16488.v1-master.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v4-branch-1.patch, HBASE-16488.v5-branch-1.patch, > HBASE-16488.v6-branch-1.patch, HBASE-16488.v7-branch-1.patch, > HBASE-16488.v8-branch-1.patch, HBASE-16488.v9-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12349) Add Maven build support module for a custom version of error-prone
[
https://issues.apache.org/jira/browse/HBASE-12349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088388#comment-16088388
]
Hadoop QA commented on HBASE-12349:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 8m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-resource-bundle hbase-testing-util hbase-spark-it hbase-assembly
hbase-shaded hbase-archetypes . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
29s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
24s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
38s{color} | {color:red} hbase-rest in master has 3 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 7m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
39s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-build-support hbase-build-configuration hbase-resource-bundle
hbase-testing-util hbase-spark-it hbase-assembly hbase-shaded hbase-archetypes
. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
8s{color} | {color:red} hbase-error-prone in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
8s{color} | {color:green} hbase-build-support in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
7s{color} | {color:green} hbase-error-prone in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
6s{color} | {color:green} hbase-build-configuration in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
7s{color} |
[jira] [Commented] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[
https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088387#comment-16088387
]
Hadoop QA commented on HBASE-18353:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
50s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}111m
35s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}157m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18353 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877417/HBASE-18353-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c65d3cfdb085 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a9352fe |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7665/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7665/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7665/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in
> HBASE-14614
> ---
>
> Key: HBASE-18353
> URL: https://issues.apache.org/jira/browse/HBASE-18353
> Project: HBase
[jira] [Commented] (HBASE-14070) Hybrid Logical Clocks for HBase
[ https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088380#comment-16088380 ] Hudson commented on HBASE-14070: FAILURE: Integrated in Jenkins build HBASE-14070.HLC #56 (See [https://builds.apache.org/job/HBASE-14070.HLC/56/]) HBASE-14070 Undoing the use of master's timestamp for meta updates (Sai (appy: rev c90948c8e28224c303337db44944d303f853ad4d) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckReplicas.java > Hybrid Logical Clocks for HBase > --- > > Key: HBASE-14070 > URL: https://issues.apache.org/jira/browse/HBASE-14070 > Project: HBase > Issue Type: New Feature >Reporter: Enis Soztutar >Assignee: Amit Patel > Attachments: HBASE-14070.master.001.patch, > HybridLogicalClocksforHBaseandPhoenix.docx, > HybridLogicalClocksforHBaseandPhoenix.pdf > > > HBase and Phoenix uses systems physical clock (PT) to give timestamps to > events (read and writes). This works mostly when the system clock is strictly > monotonically increasing and there is no cross-dependency between servers > clocks. However we know that leap seconds, general clock skew and clock drift > are in fact real. > This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of > hybrid physical clock + a logical clock. HLC is best of both worlds where it > keeps causality relationship similar to logical clocks, but still is > compatible with NTP based physical system clock. HLC can be represented in > 64bits. > A design document is attached and also can be found here: > https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit# -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088335#comment-16088335
]
Ted Yu commented on HBASE-18381:
Looking at the call stack, I don't see mob related class.
With the fix in place, I don't expect non-mob table to crash.
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
>
[jira] [Commented] (HBASE-18384) Add link to refguide schema section on apache blog on hbase application archetypes
[ https://issues.apache.org/jira/browse/HBASE-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088332#comment-16088332 ] Hudson commented on HBASE-18384: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3376 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3376/]) HBASE-18384 Add link to refguide schema section on apache blog on hbase (stack: rev c08db6733def157fb8cc7e379307b2d577ae1550) * (edit) src/main/asciidoc/_chapters/schema_design.adoc > Add link to refguide schema section on apache blog on hbase application > archetypes > -- > > Key: HBASE-18384 > URL: https://issues.apache.org/jira/browse/HBASE-18384 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack >Assignee: stack >Priority: Minor > Fix For: 3.0.0 > > > Add a link to this post > https://blogs.apache.org/hbase/entry/hbase-application-archetypes-redux into > the refguide in the schema section. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088331#comment-16088331 ] Hudson commented on HBASE-18175: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3376 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3376/]) HBASE-18175: (addendum) Add hbase-spark integration test into (busbey: rev f3a398953f9f293d09874622551cce8032dbe4a0) * (edit) hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[ https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088330#comment-16088330 ] Hudson commented on HBASE-18376: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3376 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3376/]) HBASE-18376 Add debug logs to hbase-personality.sh to make it possible (appy: rev ea0062fa54bba3c31a597e94b62f4130624b6f82) * (edit) dev-support/hbase-personality.sh > Flaky exclusion doesn't appear to work in precommit > --- > > Key: HBASE-18376 > URL: https://issues.apache.org/jira/browse/HBASE-18376 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Sean Busbey >Priority: Critical > Attachments: HBASE-18376.master.001.patch > > > Yesterday we started defaulting the precommit parameter for the flaky test > list to point to the job on builds.a.o. Looks like the personality is > ignoring it. > example build that's marked to keep: > https://builds.apache.org/job/PreCommit-HBASE-Build/7646/ > (search for 'Running unit tests' to skip to the right part of the console') > should add some more debug output in there too. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18229) create new Async Split API to embrace AM v2
[ https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088318#comment-16088318 ] Hudson commented on HBASE-18229: FAILURE: Integrated in Jenkins build HBase-2.0 #174 (See [https://builds.apache.org/job/HBase-2.0/174/]) HBASE-18229: create new Async Split API to embrace AM v2 (stack: rev a4575704bcaba8acd26d7634d0fa96bd1df12fd5) * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ClientProtos.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MasterProtos.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestIncrementalBackup.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java * (edit) hbase-protocol-shaded/src/main/protobuf/Master.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/Util.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin1.java * (edit) hbase-shell/src/test/ruby/hbase/admin_test.rb * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java > create new Async Split API to embrace AM v2 > --- > > Key: HBASE-18229 > URL: https://issues.apache.org/jira/browse/HBASE-18229 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0-alpha-2 > > Attachments: HBase-18229-master-v1.patch, > hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch, > hbase-18229-master-v4.patch, hbase-18229-master-v5.patch, > hbase-18229-master-v6.patch, hbase-18229-master-v7.patch, > hbase-18229-master-v7.patch, hbase-18229-master-v8.patch > > > See HBASE-18105 for related information, this jira will change the logic of > Path 1 in splitProcedure, the execution path will be: > *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> > MasterRpcServices.splitRegion -> HMaster.splitRegion-> > AssignmentManager.submitProcedure* > Master Will no longer ask Rs and then Rs turn around to ask master to do the > split operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088317#comment-16088317 ] Hudson commented on HBASE-18175: FAILURE: Integrated in Jenkins build HBase-2.0 #174 (See [https://builds.apache.org/job/HBase-2.0/174/]) HBASE-18175: (addendum) Add hbase-spark integration test into (busbey: rev 7be5e48c240aa8b93e938272a42862c25ad94f67) * (edit) hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-18353: -- Attachment: HBASE-18353-v2.patch Patch v2 with some cleanups > Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in > HBASE-14614 > --- > > Key: HBASE-18353 > URL: https://issues.apache.org/jira/browse/HBASE-18353 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > Attachments: HBASE-18353-v1.patch, HBASE-18353-v2.patch > > > HBASE-14614 disabled TestCorruptedRegionStoreFile, as it depends on a > half-implemented reopen of a region when a store file goes missing. > This JIRA tracks the work to fix/enable the test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088303#comment-16088303
]
huaxiang sun commented on HBASE-18381:
--
Hi [~te...@apache.org], wondering if this is related with MOB. The 64MB is for
column qualifier. I think it will crash for non-MOB tables.
If this is the case, can we have a separate unittest for HFile? Thanks.
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
>
[jira] [Updated] (HBASE-18354) Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-18354: -- Status: Open (was: Patch Available) > Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614 > - > > Key: HBASE-18354 > URL: https://issues.apache.org/jira/browse/HBASE-18354 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > Attachments: HBASE-18354-v1.patch > > > With Core Proc-V2 AM change in HBASE-14614, stuff is different now around > startup which messes up the TestMasterMetrics test. HBASE-14614 disabled two > of three tests. > This JIRA tracks work to fix the disabled tests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18362) Investigate why replica region shows up as key in meta table
[ https://issues.apache.org/jira/browse/HBASE-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088286#comment-16088286 ] huaxiang sun commented on HBASE-18362: -- I went through the code again this week, still do not have clear idea how this could happen yet. Will spend some more time on this. > Investigate why replica region shows up as key in meta table > > > Key: HBASE-18362 > URL: https://issues.apache.org/jira/browse/HBASE-18362 > Project: HBase > Issue Type: Bug > Components: hbck >Affects Versions: 1.4.0, 2.0.0-alpha-1 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > > As described in HBASE-18247, replica region shows up as key in meta table. We > need to find out the code/logic introducing the issue. I suspected that it > was introduced by hbck, but last round of code review did not find anything. > Create jira to track this effort. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18354) Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088287#comment-16088287 ] Vladimir Rodionov commented on HBASE-18354: --- Let me take a look at the patch [~devaraj]. Probably, it is the ServerLoad metrics issue. > Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614 > - > > Key: HBASE-18354 > URL: https://issues.apache.org/jira/browse/HBASE-18354 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > Attachments: HBASE-18354-v1.patch > > > With Core Proc-V2 AM change in HBASE-14614, stuff is different now around > startup which messes up the TestMasterMetrics test. HBASE-14614 disabled two > of three tests. > This JIRA tracks work to fix the disabled tests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18328) Undoing using the master's timestamp for meta updates
[ https://issues.apache.org/jira/browse/HBASE-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088236#comment-16088236 ] Appy commented on HBASE-18328: -- Committed to HBASE-14070.HLC branch. Thanks for the patch [~amit.patel]. > Undoing using the master's timestamp for meta updates > - > > Key: HBASE-18328 > URL: https://issues.apache.org/jira/browse/HBASE-18328 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: > 0001-HBASE-14070-Undoing-the-use-of-master-s-timestamp-fo.patch, > HBASE-18328.HBASE-14070.HLC.001.patch, HBASE-18328.HBASE-14070.HLC.002.patch > > > After introducing [the Core HLC > framework|https://issues.apache.org/jira/browse/HBASE-18305], the next step > is to remove all instances where the timestamp is explicitly set for meta > updates. Most of these changes take place in MetaTableAccessor. The patch for > this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch. > Credit for this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18328) Undoing using the master's timestamp for meta updates
[ https://issues.apache.org/jira/browse/HBASE-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18328. -- Resolution: Fixed > Undoing using the master's timestamp for meta updates > - > > Key: HBASE-18328 > URL: https://issues.apache.org/jira/browse/HBASE-18328 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: > 0001-HBASE-14070-Undoing-the-use-of-master-s-timestamp-fo.patch, > HBASE-18328.HBASE-14070.HLC.001.patch, HBASE-18328.HBASE-14070.HLC.002.patch > > > After introducing [the Core HLC > framework|https://issues.apache.org/jira/browse/HBASE-18305], the next step > is to remove all instances where the timestamp is explicitly set for meta > updates. Most of these changes take place in MetaTableAccessor. The patch for > this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch. > Credit for this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18328) Undoing using the master's timestamp for meta updates
[ https://issues.apache.org/jira/browse/HBASE-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18328: - Fix Version/s: HBASE-14070 > Undoing using the master's timestamp for meta updates > - > > Key: HBASE-18328 > URL: https://issues.apache.org/jira/browse/HBASE-18328 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: > 0001-HBASE-14070-Undoing-the-use-of-master-s-timestamp-fo.patch, > HBASE-18328.HBASE-14070.HLC.001.patch, HBASE-18328.HBASE-14070.HLC.002.patch > > > After introducing [the Core HLC > framework|https://issues.apache.org/jira/browse/HBASE-18305], the next step > is to remove all instances where the timestamp is explicitly set for meta > updates. Most of these changes take place in MetaTableAccessor. The patch for > this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch. > Credit for this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18354) Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088227#comment-16088227 ] Devaraj Das commented on HBASE-18354: - The test works fine with the patch. But could you please clarify as to why that totalNumberBefore could be non-zero? > Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614 > - > > Key: HBASE-18354 > URL: https://issues.apache.org/jira/browse/HBASE-18354 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > Attachments: HBASE-18354-v1.patch > > > With Core Proc-V2 AM change in HBASE-14614, stuff is different now around > startup which messes up the TestMasterMetrics test. HBASE-14614 disabled two > of three tests. > This JIRA tracks work to fix the disabled tests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18385) Enable HLC for just the meta table
[ https://issues.apache.org/jira/browse/HBASE-18385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18385: --- Attachment: HBASE-18385.HBASE-14070.HLC.001.patch > Enable HLC for just the meta table > -- > > Key: HBASE-18385 > URL: https://issues.apache.org/jira/browse/HBASE-18385 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18385.HBASE-14070.HLC.001.patch > > > This task covers the patch for enabling HLC for the meta table while keeping > the remaining tables by default use the system clock. The patch for this task > will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. > Credit for this work goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18385) Enable HLC for just the meta table
Amit Patel created HBASE-18385: -- Summary: Enable HLC for just the meta table Key: HBASE-18385 URL: https://issues.apache.org/jira/browse/HBASE-18385 Project: HBase Issue Type: Sub-task Reporter: Amit Patel Assignee: Amit Patel Priority: Trivial This task covers the patch for enabling HLC for the meta table while keeping the remaining tables by default use the system clock. The patch for this task will be going against the [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] branch with the intent that it would be added as a commit. Credit for this work goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088212#comment-16088212 ] Alex Leblang commented on HBASE-18020: -- I think I've addressed all of your concerns [~dspivak] > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020.branch-1.2.004.patch > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18377) Error handling for FileNotFoundException should consider RemoteException in ReplicationSource#openReader()
[
https://issues.apache.org/jira/browse/HBASE-18377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088209#comment-16088209
]
Hadoop QA commented on HBASE-18377:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
48s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}119m
21s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18377 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877116/18377.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 17f55de2d442 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ea0062f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7662/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7662/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7662/console |
| Powered by | Apache Yetus
[jira] [Commented] (HBASE-18367) Reduce ProcedureInfo usage
[
https://issues.apache.org/jira/browse/HBASE-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088196#comment-16088196
]
stack commented on HBASE-18367:
---
Making getState and isRunnable public seems small price to pay if can remove
use of PI.
Maybe s/CompletedProcedure/CompletedProcedureRetainer/ since it not a
Procedure... it just has one?
Should we add a method to the CompletedProcedure or whatever its called that is
like this:
boolean evict(long now);
... and inside it, it would do all the checks you have here:
189 if ((completedProcedure.hasClientAckTime() && (now -
completedProcedure.getClientAckTime()) >= evictAckTtl) ||
190 (now - proc.getLastUpdate()) >= evictTtl) {
Why a FailedProcedure ?
Nice cleanup in ProcedureStore and in ProcedureWALFormatReader
This looks like a good direction to me [~balazs.meszaros]
> Reduce ProcedureInfo usage
> --
>
> Key: HBASE-18367
> URL: https://issues.apache.org/jira/browse/HBASE-18367
> Project: HBase
> Issue Type: Sub-task
> Components: master, proc-v2
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
> Fix For: 2.0.0
>
> Attachments: HBASE-18367.001.patch, HBASE-18367.002.patch
>
>
> If we want to replace ProcedureInfo objects with jsons (HBASE-18106) we have
> to reduce ProcedureInfo usage. Currently it is used several places in the
> code where it could be replaced with Procedure (e.g. ProcedureExecutor). We
> should use ProcedureInfo only for the communication before removing it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18332) Upgrade asciidoctor-maven-plugin
[
https://issues.apache.org/jira/browse/HBASE-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088152#comment-16088152
]
stack commented on HBASE-18332:
---
See above [~psomogyi] (when you get back from vacation on monday sir -- thank
you) -- and ok on the pdfmarks... but how is the pdf on hbase.apache.org made
(I'd have thought it an asciidoc doohickey?) Thanks.
> Upgrade asciidoctor-maven-plugin
>
>
> Key: HBASE-18332
> URL: https://issues.apache.org/jira/browse/HBASE-18332
> Project: HBase
> Issue Type: Improvement
> Components: site
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18332.master.001.patch
>
>
> HBASE-18264 upgraded asciidoctor-maven-plugin and asciidoctorj-pdf but it
> caused build failure for {{site}} goal due to a change in pdfmark generation.
> These plugins were rolled back in HBASE-18320.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088142#comment-16088142 ] Umesh Agashe commented on HBASE-18366: -- Thanks for disabling the test, [~stack]! > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18366: -- Fix Version/s: 2.0.0 > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088141#comment-16088141 ] stack commented on HBASE-18366: --- Oh, disabled this test in master and branch-2. > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088139#comment-16088139 ] stack commented on HBASE-18366: --- Pushed disabling the test for now until the more fundamental fix is in place. See nice finding above [~yangzhe1991] sir. Thanks [~uagashe] > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18328) Undoing using the master's timestamp for meta updates
[ https://issues.apache.org/jira/browse/HBASE-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18328: --- Attachment: HBASE-18328.HBASE-14070.HLC.002.patch > Undoing using the master's timestamp for meta updates > - > > Key: HBASE-18328 > URL: https://issues.apache.org/jira/browse/HBASE-18328 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: > 0001-HBASE-14070-Undoing-the-use-of-master-s-timestamp-fo.patch, > HBASE-18328.HBASE-14070.HLC.001.patch, HBASE-18328.HBASE-14070.HLC.002.patch > > > After introducing [the Core HLC > framework|https://issues.apache.org/jira/browse/HBASE-18305], the next step > is to remove all instances where the timestamp is explicitly set for meta > updates. Most of these changes take place in MetaTableAccessor. The patch for > this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch. > Credit for this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18366: -- Priority: Blocker (was: Major) > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088121#comment-16088121 ] stack commented on HBASE-18366: --- bq. Initially all RS are at the same version i.e. 3.0.0-SNAPSHOT. HMaster.getRegionServerVersion() returns version 0.0.0 for dead RS (carrying meta)MoveRegionProcedure to move meta region from RS with version 0.0.0 to one of other RS with latest version. This is good. We have double the procedures working on reassign now. bq. I found that server can be online and dead at the same time! Good one [~uagashe] This is a 'hole'. On the change, it looks good to me. I wonder though how something went into the serverdead list w/o being pulled from the online list. That seems like a backdoor we want to close. I can disable for now but will not resolve this issue. I like pulling the checkIfShouldMoveSystemRegionAsync logic handling into your new procedure, HBASE-18261. Would be good to figure why addition to dead list does not get a server purged from the online list? Because it has not been processed yet by crash procedure? How did it get into dead list then? Thanks [~uagashe] > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12349) Add Maven build support module for a custom version of error-prone
[ https://issues.apache.org/jira/browse/HBASE-12349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-12349: -- Attachment: HBASE-12349.v2.patch Got this into a workable state, finally. * Create new hbase-build-support module to hold hbase-error-prone. ** We need this so that we can separate the compilation configurations to compile our rules using plain javac and compile the rest using javac+rules ** Eventually, we would move hbase-checkstyle and hbase-resource-bundle into there also, I think. * Created hbase-build-configuration module to hold the error-prone profile. Moved all other modules to inherit from it instead of hbase-root. ** This can have other configurations in it that we don't want affecting the build support modules, but I don't know what they'd be at this point. Even without them, this is kind of clunky but still necessary, I think. * While testing this, I ended up fixing a few error-prone bugs to make sure that the results are actually correct. I can back them out and include them separately, but it seems fine to include them. There's still plenty more follow on work to be done. * Added a placeholder custom error-prone rule that never fails. Can remove this when we have other rules in place, but I wanted to have some java source available. Future work we will want to add @Generated to all of the protoc code, this will speed up error-prone-javac a bit, and also start chopping away the reported errors and then the warnings. There's plenty of work to do, unsure how much is false positives and how much is true, but it's a non zero amount of actual bugs. test-patch is very very likely to fail, we'll see if there's something I can iterate on in this patch or wait until the next one. > Add Maven build support module for a custom version of error-prone > -- > > Key: HBASE-12349 > URL: https://issues.apache.org/jira/browse/HBASE-12349 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Mike Drob > Fix For: 2.0.0 > > Attachments: HBASE-12349.patch, HBASE-12349.v2.patch > > > Add a new Maven build support module that builds and publishes a custom > error-prone artifact for use by the rest of the build. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12349) Add Maven build support module for a custom version of error-prone
[ https://issues.apache.org/jira/browse/HBASE-12349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-12349: -- Status: Patch Available (was: Open) > Add Maven build support module for a custom version of error-prone > -- > > Key: HBASE-12349 > URL: https://issues.apache.org/jira/browse/HBASE-12349 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Mike Drob > Fix For: 2.0.0 > > Attachments: HBASE-12349.patch, HBASE-12349.v2.patch > > > Add a new Maven build support module that builds and publishes a custom > error-prone artifact for use by the rest of the build. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18229) create new Async Split API to embrace AM v2
[ https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18229: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master and branch-2. Very nice work [~easyliangjob] (sorry for delay pushing... yeah, all tests ran and passed). > create new Async Split API to embrace AM v2 > --- > > Key: HBASE-18229 > URL: https://issues.apache.org/jira/browse/HBASE-18229 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0-alpha-2 > > Attachments: HBase-18229-master-v1.patch, > hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch, > hbase-18229-master-v4.patch, hbase-18229-master-v5.patch, > hbase-18229-master-v6.patch, hbase-18229-master-v7.patch, > hbase-18229-master-v7.patch, hbase-18229-master-v8.patch > > > See HBASE-18105 for related information, this jira will change the logic of > Path 1 in splitProcedure, the execution path will be: > *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> > MasterRpcServices.splitRegion -> HMaster.splitRegion-> > AssignmentManager.submitProcedure* > Master Will no longer ask Rs and then Rs turn around to ask master to do the > split operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18107) [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
[
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088099#comment-16088099
]
Hadoop QA commented on HBASE-18107:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m
31s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
1s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
57s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
44s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 78m 13s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface |
| | org.apache.hadoop.hbase.regionserver.TestRowTooBig |
| | org.apache.hadoop.hbase.regionserver.wal.TestAsyncWALReplay |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| |
[jira] [Resolved] (HBASE-18384) Add link to refguide schema section on apache blog on hbase application archetypes
[ https://issues.apache.org/jira/browse/HBASE-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-18384. --- Resolution: Fixed Fix Version/s: 3.0.0 Pushed to master branch. > Add link to refguide schema section on apache blog on hbase application > archetypes > -- > > Key: HBASE-18384 > URL: https://issues.apache.org/jira/browse/HBASE-18384 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack >Assignee: stack >Priority: Minor > Fix For: 3.0.0 > > > Add a link to this post > https://blogs.apache.org/hbase/entry/hbase-application-archetypes-redux into > the refguide in the schema section. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18384) Add link to refguide schema section on apache blog on hbase application archetypes
stack created HBASE-18384: - Summary: Add link to refguide schema section on apache blog on hbase application archetypes Key: HBASE-18384 URL: https://issues.apache.org/jira/browse/HBASE-18384 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor Add a link to this post https://blogs.apache.org/hbase/entry/hbase-application-archetypes-redux into the refguide in the schema section. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18305) Initial patch (Core HLC) for public branch
[ https://issues.apache.org/jira/browse/HBASE-18305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18305: --- Attachment: HBASE-18305.master.005.patch > Initial patch (Core HLC) for public branch > -- > > Key: HBASE-18305 > URL: https://issues.apache.org/jira/browse/HBASE-18305 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > Fix For: HBASE-14070 > > Attachments: 0001-HBASE-14070-Core-HLC-revision-1.patch, > HBASE-18305.HBASE-14070.HLC.001.patch, HBASE-18305.master.001.patch, > HBASE-18305.master.002.patch, HBASE-18305.master.003.patch, > HBASE-18305.master.004.patch, HBASE-18305.master.005.patch > > > Used for tracking the status of the initial patch for HLC. Core HLC refers to > the addition of the HLC framework (i.e., Clock, Timestamp API), integration > of said framework with the codebase, and changes to tests. Practically all of > this work has been done by our [~saitejar]. > This patch in particular intentionally does not include: > * undoing using the master's timestamp for meta updates, > * enabling HLC for meta > * updating the clock for events like recovery, replication, region > open/close. > These changes will be introduced soon in smaller followup commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18332) Upgrade asciidoctor-maven-plugin
[
https://issues.apache.org/jira/browse/HBASE-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088021#comment-16088021
]
Misty Stanley-Jones commented on HBASE-18332:
-
The HTML looks great. I believe the PDF has a regression that the logo image no
longer shows up on the first page. It's not an urgent issue but it's not a
regression I'm happy with. Can you look into it?
> Upgrade asciidoctor-maven-plugin
>
>
> Key: HBASE-18332
> URL: https://issues.apache.org/jira/browse/HBASE-18332
> Project: HBase
> Issue Type: Improvement
> Components: site
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18332.master.001.patch
>
>
> HBASE-18264 upgraded asciidoctor-maven-plugin and asciidoctorj-pdf but it
> caused build failure for {{site}} goal due to a change in pdfmark generation.
> These plugins were rolled back in HBASE-18320.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations
[
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088007#comment-16088007
]
Hadoop QA commented on HBASE-15816:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
39s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 55s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
19s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
43s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}119m
47s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
4s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}185m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-15816 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877345/HBASE-15816.v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 499ef68be3b2 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 79a702d |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |
[jira] [Updated] (HBASE-18305) Initial patch (Core HLC) for public branch
[ https://issues.apache.org/jira/browse/HBASE-18305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18305: --- Attachment: (was: HBASE-18305.HBASE-14070.HLC.002.patch) > Initial patch (Core HLC) for public branch > -- > > Key: HBASE-18305 > URL: https://issues.apache.org/jira/browse/HBASE-18305 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > Fix For: HBASE-14070 > > Attachments: 0001-HBASE-14070-Core-HLC-revision-1.patch, > HBASE-18305.HBASE-14070.HLC.001.patch, HBASE-18305.master.001.patch, > HBASE-18305.master.002.patch, HBASE-18305.master.003.patch, > HBASE-18305.master.004.patch > > > Used for tracking the status of the initial patch for HLC. Core HLC refers to > the addition of the HLC framework (i.e., Clock, Timestamp API), integration > of said framework with the codebase, and changes to tests. Practically all of > this work has been done by our [~saitejar]. > This patch in particular intentionally does not include: > * undoing using the master's timestamp for meta updates, > * enabling HLC for meta > * updating the clock for events like recovery, replication, region > open/close. > These changes will be introduced soon in smaller followup commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18305) Initial patch (Core HLC) for public branch
[ https://issues.apache.org/jira/browse/HBASE-18305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18305: --- Attachment: HBASE-18305.HBASE-14070.HLC.002.patch > Initial patch (Core HLC) for public branch > -- > > Key: HBASE-18305 > URL: https://issues.apache.org/jira/browse/HBASE-18305 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > Fix For: HBASE-14070 > > Attachments: 0001-HBASE-14070-Core-HLC-revision-1.patch, > HBASE-18305.HBASE-14070.HLC.001.patch, HBASE-18305.HBASE-14070.HLC.002.patch, > HBASE-18305.master.001.patch, HBASE-18305.master.002.patch, > HBASE-18305.master.003.patch, HBASE-18305.master.004.patch > > > Used for tracking the status of the initial patch for HLC. Core HLC refers to > the addition of the HLC framework (i.e., Clock, Timestamp API), integration > of said framework with the codebase, and changes to tests. Practically all of > this work has been done by our [~saitejar]. > This patch in particular intentionally does not include: > * undoing using the master's timestamp for meta updates, > * enabling HLC for meta > * updating the clock for events like recovery, replication, region > open/close. > These changes will be introduced soon in smaller followup commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087947#comment-16087947 ] Hudson commented on HBASE-18175: FAILURE: Integrated in Jenkins build HBase-2.0 #173 (See [https://builds.apache.org/job/HBase-2.0/173/]) HBASE-18175 Add hbase-spark integration test into hbase-spark-it (busbey: rev 53f354f95489a346bf24e5613883dcf9e5a044bb) * (add) hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java * (edit) pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-assembly/src/main/assembly/src.xml * (add) hbase-spark-it/pom.xml > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18187) Release hbase-2.0.0-alpha1
[ https://issues.apache.org/jira/browse/HBASE-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087946#comment-16087946 ] Hudson commented on HBASE-18187: FAILURE: Integrated in Jenkins build HBase-2.0 #173 (See [https://builds.apache.org/job/HBase-2.0/173/]) HBASE-18187 update version to 2.0.0-alpha-2-SNAPSHOT (busbey: rev 9cc94042618b9e518f41fc88c284e5273b7bf1fb) * (edit) hbase-it/pom.xml * (edit) hbase-archetypes/hbase-archetype-builder/pom.xml * (edit) pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-metrics-api/pom.xml * (edit) hbase-shell/pom.xml * (edit) hbase-protocol-shaded/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-annotations/pom.xml * (edit) hbase-archetypes/hbase-shaded-client-project/pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-archetypes/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-endpoint/pom.xml * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-testing-util/pom.xml * (edit) hbase-archetypes/hbase-client-project/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-spark/pom.xml * (edit) hbase-rsgroup/pom.xml * (edit) hbase-server/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-metrics/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-protocol/pom.xml > Release hbase-2.0.0-alpha1 > -- > > Key: HBASE-18187 > URL: https://issues.apache.org/jira/browse/HBASE-18187 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Push an hbase-2.0.0-alpha1 > Will file subtasks here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[
https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087937#comment-16087937
]
Sean Busbey commented on HBASE-18376:
-
sure. {{yetus_error}} is the function for that.
> Flaky exclusion doesn't appear to work in precommit
> ---
>
> Key: HBASE-18376
> URL: https://issues.apache.org/jira/browse/HBASE-18376
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Priority: Critical
> Attachments: HBASE-18376.master.001.patch
>
>
> Yesterday we started defaulting the precommit parameter for the flaky test
> list to point to the job on builds.a.o. Looks like the personality is
> ignoring it.
> example build that's marked to keep:
> https://builds.apache.org/job/PreCommit-HBASE-Build/7646/
> (search for 'Running unit tests' to skip to the right part of the console')
> should add some more debug output in there too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18175: Resolution: Fixed Status: Resolved (was: Patch Available) addendum pushed. thanks again [~easyliangjob]! > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[ https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087924#comment-16087924 ] Appy commented on HBASE-18376: -- since it's error, echoing irrespective of whether yetus_debug is on/off made sense to me. What do you think? > Flaky exclusion doesn't appear to work in precommit > --- > > Key: HBASE-18376 > URL: https://issues.apache.org/jira/browse/HBASE-18376 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Sean Busbey >Priority: Critical > Attachments: HBASE-18376.master.001.patch > > > Yesterday we started defaulting the precommit parameter for the flaky test > list to point to the job on builds.a.o. Looks like the personality is > ignoring it. > example build that's marked to keep: > https://builds.apache.org/job/PreCommit-HBASE-Build/7646/ > (search for 'Running unit tests' to skip to the right part of the console') > should add some more debug output in there too. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[
https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087920#comment-16087920
]
Hadoop QA commented on HBASE-18376:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-18376 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18376 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877377/HBASE-18376.master.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7663/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Flaky exclusion doesn't appear to work in precommit
> ---
>
> Key: HBASE-18376
> URL: https://issues.apache.org/jira/browse/HBASE-18376
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Priority: Critical
> Attachments: HBASE-18376.master.001.patch
>
>
> Yesterday we started defaulting the precommit parameter for the flaky test
> list to point to the job on builds.a.o. Looks like the personality is
> ignoring it.
> example build that's marked to keep:
> https://builds.apache.org/job/PreCommit-HBASE-Build/7646/
> (search for 'Running unit tests' to skip to the right part of the console')
> should add some more debug output in there too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[
https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087909#comment-16087909
]
Sean Busbey commented on HBASE-18376:
-
oh! but it'd be good to add an else clause to the downloads that did
{{yetus_debug "downloading failed."}} or similar
> Flaky exclusion doesn't appear to work in precommit
> ---
>
> Key: HBASE-18376
> URL: https://issues.apache.org/jira/browse/HBASE-18376
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Priority: Critical
> Attachments: HBASE-18376.master.001.patch
>
>
> Yesterday we started defaulting the precommit parameter for the flaky test
> list to point to the job on builds.a.o. Looks like the personality is
> ignoring it.
> example build that's marked to keep:
> https://builds.apache.org/job/PreCommit-HBASE-Build/7646/
> (search for 'Running unit tests' to skip to the right part of the console')
> should add some more debug output in there too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[ https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087906#comment-16087906 ] Sean Busbey commented on HBASE-18376: - +1 on https://issues.apache.org/jira/secure/attachment/12877377/HBASE-18376.master.001.patch > Flaky exclusion doesn't appear to work in precommit > --- > > Key: HBASE-18376 > URL: https://issues.apache.org/jira/browse/HBASE-18376 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Sean Busbey >Priority: Critical > Attachments: HBASE-18376.master.001.patch > > > Yesterday we started defaulting the precommit parameter for the flaky test > list to point to the job on builds.a.o. Looks like the personality is > ignoring it. > example build that's marked to keep: > https://builds.apache.org/job/PreCommit-HBASE-Build/7646/ > (search for 'Running unit tests' to skip to the right part of the console') > should add some more debug output in there too. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[ https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18376: Status: Patch Available (was: Open) > Flaky exclusion doesn't appear to work in precommit > --- > > Key: HBASE-18376 > URL: https://issues.apache.org/jira/browse/HBASE-18376 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Sean Busbey >Priority: Critical > Attachments: HBASE-18376.master.001.patch > > > Yesterday we started defaulting the precommit parameter for the flaky test > list to point to the job on builds.a.o. Looks like the personality is > ignoring it. > example build that's marked to keep: > https://builds.apache.org/job/PreCommit-HBASE-Build/7646/ > (search for 'Running unit tests' to skip to the right part of the console') > should add some more debug output in there too. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18061) [C++] Fix retry logic in multi-get calls
[
https://issues.apache.org/jira/browse/HBASE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087897#comment-16087897
]
Enis Soztutar commented on HBASE-18061:
---
Thanks Sudeep for the patch.
I've run the unit tests which pass. I was also be to run simple-client doing
multi-gets against 500K rows with batch sizes 1,10,100,1000,1 and 10.
A batch size of 500K results in
{code}
what(): LifoSemMPMCQueue full, can't add item
{code}
but it is fine for now.
Why are we doing try lock here? What happens when we cannot acquire the lock?
{code}
-std::lock_guard lock(multi_mutex_);
-for (uint64_t num = 0; num < completed_responses.size(); ++num) {
+std::unique_lock lock(multi_mutex_, std::try_to_lock);
{code}
We are doing this in multiple places actually, and we never check the results
of whether the lock was acquired or not. Seems something to fix before commit.
remove these from the patch:
{code}
+//#include
+//using ::testing::Return;
+//using ::testing::_;
{code}
I was double checking the logic again against the java implementation. It seems
that we are missing the function:
{code}
private List removeErrors(Action action) {
synchronized (action2Errors) {
return action2Errors.remove(action);
}
}
{code}
which is called from failOne() etc, and removes the error from the
actions2Errors map. However, logically, I think it is fine to not have this
removal logic. Just making a note here in case we run into it later.
- Mainly a code cleanness issue, but you should move this logic:
{code}
auto itr = search->second->actions_by_region().find(region_loc->region_name());
if (itr != search->second->actions_by_region().end()) {
search->second->AddActionsByRegion(region_loc, action);
} else {
search->second = std::make_shared(region_loc);
search->second->AddActionsByRegion(region_loc, action);
}
{code}
to be inside ServerRequest::AddActionsByRegion(). The logic for "add the action
to the list of actions by region by finding the region" belongs in this method,
and should not be exposed outside normally.
I think there is no multi-region test with failure conditions. We can look at
adding that in a follow up issue if needed.
In {{::Send()}}, this logic:
{code}
if (!ExceptionUtil::ShouldRetry(ew)) {
std::vector failed_actions;
for (auto : action_by_server.second->actions_by_region()) {
for (const auto _action : value.second->actions()) {
failed_actions.push_back(failed_action);
}
}
FailAll(failed_actions, tries);
} else {
OnError(action_by_server.second->actions_by_region(), tries, ew,
action_by_server.first);
}
{code}
can be replaced by a simple OnError() call, no? OnError() already checks
whether exception should retry and calls FailAll() with the same vector of
actions. Can you please double check.
Other than these patch looks pretty good.
> [C++] Fix retry logic in multi-get calls
>
>
> Key: HBASE-18061
> URL: https://issues.apache.org/jira/browse/HBASE-18061
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Sudeep Sunthankar
> Fix For: HBASE-14850
>
> Attachments: HBASE-18061.HBASE-14850.v1.patch,
> HBASE-18061.HBASE-14850.v3.patch, HBASE-18061.HBASE-14850.v5.patch,
> HBASE-18061.HBASE-14850.v6.patch, HBASE-18061.HBASE-14850.v7.patch,
> hbase-18061-v8.patch
>
>
> HBASE-17576 adds multi-gets. There are a couple of todos to fix in the retry
> logic, and some unit testing to be done for the multi-gets. We'll do these in
> this issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18376) Flaky exclusion doesn't appear to work in precommit
[ https://issues.apache.org/jira/browse/HBASE-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18376: - Attachment: HBASE-18376.master.001.patch > Flaky exclusion doesn't appear to work in precommit > --- > > Key: HBASE-18376 > URL: https://issues.apache.org/jira/browse/HBASE-18376 > Project: HBase > Issue Type: Bug > Components: community, test >Reporter: Sean Busbey >Priority: Critical > Attachments: HBASE-18376.master.001.patch > > > Yesterday we started defaulting the precommit parameter for the flaky test > list to point to the job on builds.a.o. Looks like the personality is > ignoring it. > example build that's marked to keep: > https://builds.apache.org/job/PreCommit-HBASE-Build/7646/ > (search for 'Running unit tests' to skip to the right part of the console') > should add some more debug output in there too. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087869#comment-16087869
]
Hadoop QA commented on HBASE-18381:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
53s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 54s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}148m
8s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}195m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18381 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877333/18381.v2.txt |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c73c1044b6c7 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 79a702d |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7657/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7657/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7657/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> HBase regionserver crashes when
[jira] [Commented] (HBASE-18332) Upgrade asciidoctor-maven-plugin
[
https://issues.apache.org/jira/browse/HBASE-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087868#comment-16087868
]
Misty Stanley-Jones commented on HBASE-18332:
-
By the way, we don't use the Change ID, so you might want to modify your Git
set-up so that it doesn't get added to commit messages in your HBase repo.
> Upgrade asciidoctor-maven-plugin
>
>
> Key: HBASE-18332
> URL: https://issues.apache.org/jira/browse/HBASE-18332
> Project: HBase
> Issue Type: Improvement
> Components: site
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18332.master.001.patch
>
>
> HBASE-18264 upgraded asciidoctor-maven-plugin and asciidoctorj-pdf but it
> caused build failure for {{site}} goal due to a change in pdfmark generation.
> These plugins were rolled back in HBASE-18320.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18332) Upgrade asciidoctor-maven-plugin
[
https://issues.apache.org/jira/browse/HBASE-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087864#comment-16087864
]
Misty Stanley-Jones commented on HBASE-18332:
-
Let me try building the docs with this patch. Thanks [~psomogyi]
> Upgrade asciidoctor-maven-plugin
>
>
> Key: HBASE-18332
> URL: https://issues.apache.org/jira/browse/HBASE-18332
> Project: HBase
> Issue Type: Improvement
> Components: site
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18332.master.001.patch
>
>
> HBASE-18264 upgraded asciidoctor-maven-plugin and asciidoctorj-pdf but it
> caused build failure for {{site}} goal due to a change in pdfmark generation.
> These plugins were rolled back in HBASE-18320.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18086) Create native client which creates load on selected cluster
[ https://issues.apache.org/jira/browse/HBASE-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18086: --- Attachment: 18086.v12.txt Patch v12 changes the default value for multi_get_size. Also paired logs to mark the start / end of write phase so that it is easier to read. > Create native client which creates load on selected cluster > --- > > Key: HBASE-18086 > URL: https://issues.apache.org/jira/browse/HBASE-18086 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18086.v11.txt, 18086.v12.txt, 18086.v1.txt, > 18086.v3.txt, 18086.v4.txt, 18086.v5.txt, 18086.v6.txt, 18086.v7.txt, > 18086.v8.txt > > > This task is to create a client which uses multiple threads to conduct Puts > followed by Gets against selected cluster. > Default is to run the tool against local cluster. > This would give us some idea on the characteristics of native client in terms > of handling high load. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[
https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087815#comment-16087815
]
Hadoop QA commented on HBASE-18175:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
13s{color} | {color:green} hbase-spark-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.13.1 Server=1.13.1 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18175 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877354/addendum-master.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c00a5ba9b3cb 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 79a702d |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7661/testReport/ |
| modules | C: hbase-spark-it U: hbase-spark-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7661/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Add hbase-spark integration test into hbase-spark-it
>
>
> Key: HBASE-18175
>
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087791#comment-16087791 ] Sean Busbey commented on HBASE-18147: - I can remove those tabs before commit, fyi. > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch, > HBASE-18147.v1.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087790#comment-16087790 ] Sean Busbey commented on HBASE-18147: - bq. -1 whitespace 0m 0s The patch 7 line(s) with tabs. lol. great catch. amazed I missed this. > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch, > HBASE-18147.v1.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[
https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087772#comment-16087772
]
Hadoop QA commented on HBASE-18147:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 7 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18147 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12877172/HBASE-18147.v1.patch |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux fbfd387b0f65 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 79a702d |
| shellcheck | v0.4.6 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/7659/artifact/patchprocess/whitespace-tabs.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7659/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> nightly job to check health of active branches
> --
>
> Key: HBASE-18147
> URL: https://issues.apache.org/jira/browse/HBASE-18147
> Project: HBase
> Issue Type: Test
> Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch,
> HBASE-18147.v1.patch
>
>
> We should set up a job that runs Apache Yetus Test Patch's nightly mode.
> Essentially, it produces a report that considers how the branch measures up
> against the things we check in our precommit checks.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087756#comment-16087756 ] Alex Leblang commented on HBASE-18147: -- I think that given the jdk change this is a good move > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch, > HBASE-18147.v1.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087735#comment-16087735 ] Hudson commented on HBASE-18175: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3375 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3375/]) HBASE-18175 Add hbase-spark integration test into hbase-spark-it (busbey: rev 79a702d1126288a0dd45ba9458265cff8e3619c6) * (edit) pom.xml * (edit) hbase-assembly/src/main/assembly/src.xml * (add) hbase-spark-it/pom.xml * (edit) hbase-assembly/pom.xml * (add) hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18175: Status: Patch Available (was: Reopened) > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18107) [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
[ https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18107: - Fix Version/s: (was: 2.0.0) 3.0.0 Affects Version/s: (was: 2.0.0) 3.0.0 Status: Patch Available (was: Reopened) > [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions > > > Key: HBASE-18107 > URL: https://issues.apache.org/jira/browse/HBASE-18107 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Affects Versions: 3.0.0 >Reporter: stack >Assignee: Yi Liang > Fix For: 3.0.0 > > Attachments: draft.patch > > > They don't align with how we have named the Split equivalents; i.e. > SplitRegion (so should be MergeRegion...). They probably have these awkward > names because the obvious slots are occupied... so this may not be fixable > but filing issue anyways. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-18175: - > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18107) [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
[ https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18107: - Attachment: draft.patch Attach a draft patch, to try unit test, will rebase after HBASE-18229 commit > [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions > > > Key: HBASE-18107 > URL: https://issues.apache.org/jira/browse/HBASE-18107 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: draft.patch > > > They don't align with how we have named the Split equivalents; i.e. > SplitRegion (so should be MergeRegion...). They probably have these awkward > names because the obvious slots are occupied... so this may not be fixable > but filing issue anyways. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18175: - Attachment: addendum-master.patch addendum attached > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: addendum-master.patch, hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087716#comment-16087716 ] Hadoop QA commented on HBASE-18147: --- (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HBASE-Build/7659/console in case of problems. > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch, > HBASE-18147.v1.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18147: Status: Patch Available (was: In Progress) > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch, > HBASE-18147.v1.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087693#comment-16087693 ] Yi Liang commented on HBASE-18175: -- And I think I also need to change the usage in source code comments, I still use hbase-it, I will provide a addendum soon, > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[
https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087687#comment-16087687
]
Yi Liang commented on HBASE-18175:
--
Hi Sean,
I run it on cluster before attache V9, since I add HBASE_HOME/lib/* and also
hbase-spark.jar into spark classpath, so everytime i need only to run
{{spark-submit --class
org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad
HBASE_HOME/lib/hbase-spark-it-XXX-tests.jar
-Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic }}
> Add hbase-spark integration test into hbase-spark-it
>
>
> Key: HBASE-18175
> URL: https://issues.apache.org/jira/browse/HBASE-18175
> Project: HBase
> Issue Type: Test
> Components: spark
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: hbase-18175-master-v2.patch,
> hbase-18175-master-v3.patch, hbase-18175-master-v4.patch,
> hbase-18175-master-v5.patch, hbase-18175-master-v6.patch,
> hbase-18175-master-v7.patch, hbase-18175-master-v8.patch,
> hbase-18175-master-v9.patch, hbase-18175-v1.patch
>
>
> After HBASE-17574, all test under hbase-spark are regarded as unit test, and
> this jira will add integration test of hbase-spark into hbase-it. This patch
> run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to
> spark.
> test in Maven:
> mvn verify -Dit.test=IntegrationTestSparkBulkLoad
> test on cluster:
> spark-submit --class
> org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad
> HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar
> -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18086) Create native client which creates load on selected cluster
[ https://issues.apache.org/jira/browse/HBASE-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18086: --- Attachment: 18086.v11.txt Patch v11 logs progress every 5000 rows. > Create native client which creates load on selected cluster > --- > > Key: HBASE-18086 > URL: https://issues.apache.org/jira/browse/HBASE-18086 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18086.v11.txt, 18086.v1.txt, 18086.v3.txt, 18086.v4.txt, > 18086.v5.txt, 18086.v6.txt, 18086.v7.txt, 18086.v8.txt > > > This task is to create a client which uses multiple threads to conduct Puts > followed by Gets against selected cluster. > Default is to run the tool against local cluster. > This would give us some idea on the characteristics of native client in terms > of handling high load. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087653#comment-16087653 ] Sean Busbey commented on HBASE-18175: - I suspect we probably need to add some additional jars too, since the test class relies on classes in hbase-it. :/ [~easyliangjob] did you happen to run this on a cluster after the addition of the new module? > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18175: Release Note: HBase now ships with an integration test for our integration with Apache Spark. You can run this test on a cluster by using an equivalent to the below, e.g. if the version of HBase is 2.0.0-alpha-2 ``` spark-submit --class org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad HBASE_HOME/lib/hbase-spark-it-2.0.0-alpha-2-tests.jar -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic ``` was: HBase now ships with an integration test for our integration with Apache Spark. You can run this test on a cluster by using an equivalent to the below, e.g. if the version of HBase is 2.0.0-alpha-2 ``` spark-submit --class org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad HBASE_HOME/lib/hbase-it-2.0.0-alpha-2-tests.jar -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic ``` > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087643#comment-16087643 ] Sean Busbey commented on HBASE-18175: - yes, good catch. > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[
https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087638#comment-16087638
]
Mike Drob commented on HBASE-18175:
---
On the release note:
{{spark-submit --class
org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad
HBASE_HOME/lib/hbase-it-2.0.0-alpha-2-tests.jar
-Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic }}
should that be hbase-spark-it-2.0.0...?
> Add hbase-spark integration test into hbase-spark-it
>
>
> Key: HBASE-18175
> URL: https://issues.apache.org/jira/browse/HBASE-18175
> Project: HBase
> Issue Type: Test
> Components: spark
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: hbase-18175-master-v2.patch,
> hbase-18175-master-v3.patch, hbase-18175-master-v4.patch,
> hbase-18175-master-v5.patch, hbase-18175-master-v6.patch,
> hbase-18175-master-v7.patch, hbase-18175-master-v8.patch,
> hbase-18175-master-v9.patch, hbase-18175-v1.patch
>
>
> After HBASE-17574, all test under hbase-spark are regarded as unit test, and
> this jira will add integration test of hbase-spark into hbase-it. This patch
> run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to
> spark.
> test in Maven:
> mvn verify -Dit.test=IntegrationTestSparkBulkLoad
> test on cluster:
> spark-submit --class
> org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad
> HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar
> -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18175: Resolution: Fixed Fix Version/s: (was: 2.0.0) 2.0.0-alpha-2 3.0.0 Release Note: HBase now ships with an integration test for our integration with Apache Spark. You can run this test on a cluster by using an equivalent to the below, e.g. if the version of HBase is 2.0.0-alpha-2 ``` spark-submit --class org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad HBASE_HOME/lib/hbase-it-2.0.0-alpha-2-tests.jar -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic ``` Status: Resolved (was: Patch Available) pushed to master and branch-2. Thanks for working through this [~easyliangjob]! thanks for the extended review [~mdrob]! > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations
[ https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15816: --- Status: Patch Available (was: Open) > Provide client with ability to set priority on Operations > -- > > Key: HBASE-15816 > URL: https://issues.apache.org/jira/browse/HBASE-15816 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15816.patch, HBASE-15816-v1.patch, > HBASE-15816.v2.patch > > > First round will just be to expose the ability to set priorities for client > operations. For more background: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E > Next step would be to remove AnnotationReadingPriorityFunction and have the > client send priorities explicitly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations
[ https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15816: --- Attachment: HBASE-15816.v2.patch had to rewrite the patch for master and 2.x branch. If this looks okay ill do the backports after a review. > Provide client with ability to set priority on Operations > -- > > Key: HBASE-15816 > URL: https://issues.apache.org/jira/browse/HBASE-15816 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15816.patch, HBASE-15816-v1.patch, > HBASE-15816.v2.patch > > > First round will just be to expose the ability to set priorities for client > operations. For more background: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E > Next step would be to remove AnnotationReadingPriorityFunction and have the > client send priorities explicitly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations
[ https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15816: --- Status: Open (was: Patch Available) > Provide client with ability to set priority on Operations > -- > > Key: HBASE-15816 > URL: https://issues.apache.org/jira/browse/HBASE-15816 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15816.patch, HBASE-15816-v1.patch > > > First round will just be to expose the ability to set priorities for client > operations. For more background: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E > Next step would be to remove AnnotationReadingPriorityFunction and have the > client send priorities explicitly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087585#comment-16087585
]
Hadoop QA commented on HBASE-18381:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
45s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 44s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 59s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}131m 13s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.client.TestReplicasClient |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface |
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.mapreduce.TestHRegionPartitioner |
| | org.apache.hadoop.hbase.client.TestAsyncTableScanAll |
| | org.apache.hadoop.hbase.client.TestAsyncReplicationAdminApi |
| | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.client.TestMultipleTimestamps |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.client.TestAsyncTableBatch |
| | org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi |
| | org.apache.hadoop.hbase.client.TestAsyncProcedureAdminApi |
| | org.apache.hadoop.hbase.quotas.TestSnapshotQuotaObserverChore |
| | org.apache.hadoop.hbase.TestFullLogReconstruction |
| | org.apache.hadoop.hbase.mapred.TestTableInputFormat |
| | org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics |
| | org.apache.hadoop.hbase.TestMultiVersions |
| |
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18381:
---
Attachment: 18381.v2.txt
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18381:
---
Attachment: (was: 18381.v2.txt)
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18381:
---
Status: Patch Available (was: Open)
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18381:
---
Attachment: 18381.v2.txt
Moved the test change to TestDefaultMobStoreFlusher
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18381:
---
Status: Open (was: Patch Available)
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18381:
Affects Version/s: (was: 1.2.0)
2.0.0-alpha-1
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
>
[jira] [Created] (HBASE-18383) Upgrade cyrus SASL library to 2.1.27
Ted Yu created HBASE-18383: -- Summary: Upgrade cyrus SASL library to 2.1.27 Key: HBASE-18383 URL: https://issues.apache.org/jira/browse/HBASE-18383 Project: HBase Issue Type: Sub-task Reporter: Ted Yu On cyrus mailing list, Ken Murchison indicated 2.1.27 would be released in August. We should upgrade to the new release - 2.1.26 was several years old. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18381:
Fix Version/s: 2.0.0-beta-1
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.0
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading MOB file with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18381:
---
Summary: HBase regionserver crashes when reading MOB file with column
qualifier >64MB (was: HBase regionserver crash when flushing MOB with column
qualifier >64MB)
> HBase regionserver crashes when reading MOB file with column qualifier >64MB
>
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.0
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Attachments: 18381.v1.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
>
[jira] [Assigned] (HBASE-18381) HBase regionserver crash when flushing MOB with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-18381:
--
Assignee: Ted Yu
> HBase regionserver crash when flushing MOB with column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.0
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Attachments: 18381.v1.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6412)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crash when flushing MOB with column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18381:
---
Status: Patch Available (was: Open)
> HBase regionserver crash when flushing MOB with column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.0
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Priority: Critical
> Attachments: 18381.v1.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6412)
> at
>
