[jira] [Commented] (HBASE-20588) Space quota change after quota violation doesn't seem to take in effect
[
https://issues.apache.org/jira/browse/HBASE-20588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490263#comment-16490263
]
Hudson commented on HBASE-20588:
Results for branch branch-2.0
[build #344 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Space quota change after quota violation doesn't seem to take in effect
> ---
>
> Key: HBASE-20588
> URL: https://issues.apache.org/jira/browse/HBASE-20588
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20588.master.001.patch,
> HBASE-20588.master.002.patch, HBASE-20588.master.003.patch,
> HBASE-20588.master.004.patch, HBASE-20588.master.005.patch
>
>
> Steps followed
> - Through {{hbase shell}}
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => '2M', POLICY =>
> NO_INSERTS{noformat}
> - Run {{PE}} until the quota is reached
> {noformat}
> hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred
> --rows=2000 sequentialWrite 1{noformat}
> - Through {{HBase}} shell
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => NONE{noformat}
> - Through {{HBase}} shell verify the effective Quotas
> {noformat}
> > list_quotas
> OWNER QUOTAS
>
>
> 0 row(s)
> Took 0.0365 seconds{noformat}
> - Wait for some time (at least 5 mins) and try to add data to the table
> {noformat}
> > put 'TestTable','r1','info0:0','v1'
> ERROR: org.apache.hadoop.hbase.quotas.SpaceLimitingException: NO_INSERTS Puts
> are disallowed due to a space quota.
> at
> org.apache.hadoop.hbase.quotas.policies.NoInsertsViolationPolicyEnforcement.check(NoInsertsViolationPolicyEnforcement.java:47){noformat}
> To resolve the issue, {{RSes}} need to be restarted which points to in memory
> data not getting reset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20548) Master fails to startup on large clusters, refreshing block distribution
[
https://issues.apache.org/jira/browse/HBASE-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490264#comment-16490264
]
Hudson commented on HBASE-20548:
Results for branch branch-2.0
[build #344 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/344//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Master fails to startup on large clusters, refreshing block distribution
>
>
> Key: HBASE-20548
> URL: https://issues.apache.org/jira/browse/HBASE-20548
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.4
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20548.branch-1.4.001.patch,
> HBASE-20548.branch-2.0.001.patch, HBASE-20548.master.001.patch
>
>
> On our large clusters with, master has failed to startup within specified
> time and aborted itself since it was initializing HDFS block distribution.
> Enable table also takes time for larger tables for the same reason. My
> proposal is to refresh HDFS block distribution at the end of master
> initialization and not at retainAssignment()'s createCluster(). This would
> address HBASE-16570's intention, but avoid the problems we ran into.
> cc [~aoxiang] [~tedyu]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20648) HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
[ https://issues.apache.org/jira/browse/HBASE-20648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-20648: - Status: Patch Available (was: Open) > HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for > master branch > -- > > Key: HBASE-20648 > URL: https://issues.apache.org/jira/browse/HBASE-20648 > Project: HBase > Issue Type: Bug >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20648.master.001.patch > > > It seems like the issue mentioned in HBASE-19364 exists in master branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20648) HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
[ https://issues.apache.org/jira/browse/HBASE-20648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490227#comment-16490227 ] Toshihiro Suzuki commented on HBASE-20648: -- I attached a patch for this issue. The added unit tests in the patch are from HBASE-19364, but the fix is different. I think it's fine for master branch. > HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for > master branch > -- > > Key: HBASE-20648 > URL: https://issues.apache.org/jira/browse/HBASE-20648 > Project: HBase > Issue Type: Bug >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20648.master.001.patch > > > It seems like the issue mentioned in HBASE-19364 exists in master branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20648) HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
[ https://issues.apache.org/jira/browse/HBASE-20648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-20648: - Attachment: HBASE-20648.master.001.patch > HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for > master branch > -- > > Key: HBASE-20648 > URL: https://issues.apache.org/jira/browse/HBASE-20648 > Project: HBase > Issue Type: Bug >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20648.master.001.patch > > > It seems like the issue mentioned in HBASE-19364 exists in master branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20616) TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
[
https://issues.apache.org/jira/browse/HBASE-20616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490217#comment-16490217
]
Hudson commented on HBASE-20616:
Results for branch branch-2
[build #779 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> TruncateTableProcedure is stuck in retry loop in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state
> --
>
> Key: HBASE-20616
> URL: https://issues.apache.org/jira/browse/HBASE-20616
> Project: HBase
> Issue Type: Bug
> Components: amv2
> Environment: HDP-2.5.3
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 20616.master.004.patch, HBASE-20616.master.001.patch,
> HBASE-20616.master.002.patch, HBASE-20616.master.003.patch,
> HBASE-20616.master.004.patch
>
>
> At first, TruncateTableProcedure failed to write some files to HDFS in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state for some reason.
> {code:java}
> 2018-05-15 08:00:25,346 WARN [ProcedureExecutorThread-8]
> procedure.TruncateTableProcedure: Retriable error trying to truncate
> table=: state=TRUNCATE_TABLE_CREATE_FS_LAYOUT
> java.io.IOException: java.util.concurrent.ExecutionException:
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /apps/hbase/data/.tmp/data.regioninfo could
> only be replicated to 0 nodes instead of minReplication (=1). There are number of DNs> datanode(s) running and no node(s) are excluded in this
> operation.
> ...
> {code}
> But at this time, seemed like writing some files to HDFS was successful.
> And then, TruncateTableProcedure was stuck in retry loop in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state. At this point, the following log
> messages were shown repeatedly in the master log:
> {code:java}
> 2018-05-15 08:00:25,463 WARN [ProcedureExecutorThread-8]
> procedure.TruncateTableProcedure: Retriable error trying to truncate
> table=: state=TRUNCATE_TABLE_CREATE_FS_LAYOUT
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs:///apps/hbase/data/.tmp/data///
> ...
> {code}
> It seems like this is because TruncateTableProcedure tried to write the files
> that were written successfully in the first try.
> I think we need to delete all the files and directories that are written
> successfully in the previous try before retrying the
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state.
> Actually, this issue was observed in HDP-2.5.3, but I think the upstream has
> the same issue. Also, it looks to me that CreateTableProcedure has a similar
> issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20548) Master fails to startup on large clusters, refreshing block distribution
[
https://issues.apache.org/jira/browse/HBASE-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490218#comment-16490218
]
Hudson commented on HBASE-20548:
Results for branch branch-2
[build #779 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/779//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Master fails to startup on large clusters, refreshing block distribution
>
>
> Key: HBASE-20548
> URL: https://issues.apache.org/jira/browse/HBASE-20548
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.4
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20548.branch-1.4.001.patch,
> HBASE-20548.branch-2.0.001.patch, HBASE-20548.master.001.patch
>
>
> On our large clusters with, master has failed to startup within specified
> time and aborted itself since it was initializing HDFS block distribution.
> Enable table also takes time for larger tables for the same reason. My
> proposal is to refresh HDFS block distribution at the end of master
> initialization and not at retainAssignment()'s createCluster(). This would
> address HBASE-16570's intention, but avoid the problems we ran into.
> cc [~aoxiang] [~tedyu]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20647) Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
[
https://issues.apache.org/jira/browse/HBASE-20647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490216#comment-16490216
]
Hadoop QA commented on HBASE-20647:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} hbase-server: The patch generated 0 new + 0
unchanged - 9 fixed = 0 total (was 9) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
56s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
51s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 29s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}119m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:36a7029 |
| JIRA Issue | HBASE-20647 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925083/HBASE-20647.branch-1.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck
[jira] [Created] (HBASE-20648) HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
Toshihiro Suzuki created HBASE-20648: Summary: HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch Key: HBASE-20648 URL: https://issues.apache.org/jira/browse/HBASE-20648 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki It seems like the issue mentioned in HBASE-19364 exists in master branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20597) Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
[ https://issues.apache.org/jira/browse/HBASE-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20597: Component/s: Replication > Use a lock to serialize access to a shared reference to ZooKeeperWatcher in > HBaseReplicationEndpoint > > > Key: HBASE-20597 > URL: https://issues.apache.org/jira/browse/HBASE-20597 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.2, 1.4.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20597-branch-1.patch, HBASE-20597.patch > > > The code that closes down a ZKW that fails to initialize when attempting to > connect to the remote cluster is not MT safe and can in theory leak > ZooKeeperWatcher instances. The allocation of a new ZKW and store to the > reference is not atomic. Might have concurrent allocations with only one > winning store, leading to leaked ZKW instances. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20518) Need to serialize the enabled field for UpdatePeerConfigProcedure
[
https://issues.apache.org/jira/browse/HBASE-20518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490209#comment-16490209
]
Hadoop QA commented on HBASE-20518:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
1s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
5s{color} | {color:blue} hbase-server in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 41s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 37s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}161m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20518 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925075/HBASE-20518.v01.patch
|
| Optional Tests | asflicense cc unit hbaseprotoc javac javadoc findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
|
[jira] [Commented] (HBASE-18118) Default storage policy if not configured cannot be "NONE"
[ https://issues.apache.org/jira/browse/HBASE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490190#comment-16490190 ] Andrew Purtell commented on HBASE-18118: I came here and did this because of log spew. Exceptions from HDFS. Perhaps something has changed. As long as that does not happen again I'm not concerned. > Default storage policy if not configured cannot be "NONE" > - > > Key: HBASE-18118 > URL: https://issues.apache.org/jira/browse/HBASE-18118 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-18118.patch > > > HBase can't use 'NONE' as default storage policy if not configured because > HDFS supports no such policy. This policy name was probably available in a > precommit or early version of the HDFS side support for heterogeneous > storage. Now the best default is 'HOT'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18118) Default storage policy if not configured cannot be "NONE"
[ https://issues.apache.org/jira/browse/HBASE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490186#comment-16490186 ] Sean Busbey commented on HBASE-18118: - the HDFS API shouldn't be called with NONE, based on reading the code. > Default storage policy if not configured cannot be "NONE" > - > > Key: HBASE-18118 > URL: https://issues.apache.org/jira/browse/HBASE-18118 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-18118.patch > > > HBase can't use 'NONE' as default storage policy if not configured because > HDFS supports no such policy. This policy name was probably available in a > precommit or early version of the HDFS side support for heterogeneous > storage. Now the best default is 'HOT'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20636) Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
[ https://issues.apache.org/jira/browse/HBASE-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490184#comment-16490184 ] Andrew Purtell commented on HBASE-20636: If the file with the new filter cannot be read by an older version of HBase then we are lying about compatibility if the hfile version is not incremented. If you bump only the minor then a fallback will be necessary > Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED > --- > > Key: HBASE-20636 > URL: https://issues.apache.org/jira/browse/HBASE-20636 > Project: HBase > Issue Type: New Feature > Components: HFile, regionserver, scan >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-20636.master.001.patch, > HBASE-20636.master.002.patch > > > As we all know, HBase uses BloomFilter(ROW and ROWCOL) to filter unnecessary > files to improve read performance. But they only support Get and do not > support Scan. > In our company(Tencent), many users need to scan all rows with the same > prefix, such as Tencent Game. Game user's some operational record will be > written into HBase, each game user will have a lot of records, the rowkey is > constructed as userid+'#'+timestamps. So we can scan all records for a given > user for a specified period. > For this scenario, we designed the prefix Bloom filter. If the startRow and > stopRow of the Scan has a valid common prefix, the scan will be allowed to > use BloomFilter to filter files which will enhance the performance of the > scan. > Now, this feature has been running on our cluster over a year, and scan > performance for this scenario has been improved by more than one times than > before. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18118) Default storage policy if not configured cannot be "NONE"
[ https://issues.apache.org/jira/browse/HBASE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490182#comment-16490182 ] Andrew Purtell commented on HBASE-18118: At the very least don't bring back the log spew, thanks > Default storage policy if not configured cannot be "NONE" > - > > Key: HBASE-18118 > URL: https://issues.apache.org/jira/browse/HBASE-18118 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-18118.patch > > > HBase can't use 'NONE' as default storage policy if not configured because > HDFS supports no such policy. This policy name was probably available in a > precommit or early version of the HDFS side support for heterogeneous > storage. Now the best default is 'HOT'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18118) Default storage policy if not configured cannot be "NONE"
[ https://issues.apache.org/jira/browse/HBASE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490180#comment-16490180 ] Andrew Purtell commented on HBASE-18118: NONE is not a valid constant so every attempt to invoke the API will fail. That does not seem like the right thing to do. That was the reason for this change. Suggestion to revert makes no sense without some other kind of fix. > Default storage policy if not configured cannot be "NONE" > - > > Key: HBASE-18118 > URL: https://issues.apache.org/jira/browse/HBASE-18118 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-18118.patch > > > HBase can't use 'NONE' as default storage policy if not configured because > HDFS supports no such policy. This policy name was probably available in a > precommit or early version of the HDFS side support for heterogeneous > storage. Now the best default is 'HOT'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20582) Bump up JRuby version because of some reported vulnerabilities
[
https://issues.apache.org/jira/browse/HBASE-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490173#comment-16490173
]
Josh Elser commented on HBASE-20582:
Ok, that's fine by me. I'll get a patch up tmrw.
> Bump up JRuby version because of some reported vulnerabilities
> --
>
> Key: HBASE-20582
> URL: https://issues.apache.org/jira/browse/HBASE-20582
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Ankit Singhal
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20582.002.patch, HBASE-20582.patch
>
>
> There are some vulnerabilities reported with two of the libraries used in
> HBase.
> {code:java}
> Jruby(version:9.1.10.0):
> CVE-2009-5147
> CVE-2013-4363
> CVE-2014-4975
> CVE-2014-8080
> CVE-2014-8090
> CVE-2015-3900
> CVE-2015-7551
> CVE-2015-9096
> CVE-2017-0899
> CVE-2017-0900
> CVE-2017-0901
> CVE-2017-0902
> CVE-2017-0903
> CVE-2017-10784
> CVE-2017-14064
> CVE-2017-9224
> CVE-2017-9225
> CVE-2017-9226
> CVE-2017-9227
> CVE-2017-9228
> {code}
> Tool somehow able to relate the vulnerability of Ruby with JRuby(Java
> implementation). (Jackson will be handled in a different issue.)
> Not all of them directly affects HBase but [~elserj] suggested that it is
> better to be on the updated version to avoid issues during an audit in
> security sensitive organization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19722) Implement a meta query statistics metrics source
[
https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490170#comment-16490170
]
Hadoop QA commented on HBASE-19722:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
54s{color} | {color:blue} hbase-server in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
2s{color} | {color:red} hbase-server: The patch generated 3 new + 0 unchanged -
0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
15s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 51s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
56s{color} | {color:red} hbase-server generated 1 new + 2 unchanged - 0 fixed =
3 total (was 2) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | org.apache.hadoop.hbase.util.LossyCounting.sweep() makes inefficient use
of keySet iterator instead of entrySet iterator At LossyCounting.java:keySet
iterator instead of entrySet iterator At LossyCounting.java:[line 76] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-19722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925065/HBASE-19722.master.013.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 179cdb62a215 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 1fbce10ff4 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
[jira] [Commented] (HBASE-20647) Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490161#comment-16490161 ] Ted Yu commented on HBASE-20647: I am running test suite locally with patch applied. If you have a spare machine, you can do the same - just in case QA bot reports some flaky test(s). > Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in > TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1 > - > > Key: HBASE-20647 > URL: https://issues.apache.org/jira/browse/HBASE-20647 > Project: HBase > Issue Type: Sub-task > Components: backport >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20647.branch-1.001.patch > > > Backport parent issue to branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18118) Default storage policy if not configured cannot be "NONE"
[ https://issues.apache.org/jira/browse/HBASE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490156#comment-16490156 ] Yu Li commented on HBASE-18118: --- A follow up on the issue [~busbey] points out in the [email|https://s.apache.org/mcGC] sent to our @dev list, it seems a fair suggestion to revert the change here. [~apurtell] mind take a look at the discussion and share your thoughts with us? Thanks. > Default storage policy if not configured cannot be "NONE" > - > > Key: HBASE-18118 > URL: https://issues.apache.org/jira/browse/HBASE-18118 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-18118.patch > > > HBase can't use 'NONE' as default storage policy if not configured because > HDFS supports no such policy. This policy name was probably available in a > precommit or early version of the HDFS side support for heterogeneous > storage. Now the best default is 'HOT'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20647) Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490141#comment-16490141 ] Toshihiro Suzuki commented on HBASE-20647: -- I just attached the patch. Could you please review it? [~yuzhih...@gmail.com] > Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in > TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1 > - > > Key: HBASE-20647 > URL: https://issues.apache.org/jira/browse/HBASE-20647 > Project: HBase > Issue Type: Sub-task > Components: backport >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20647.branch-1.001.patch > > > Backport parent issue to branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20647) Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-20647: - Attachment: HBASE-20647.branch-1.001.patch > Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in > TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1 > - > > Key: HBASE-20647 > URL: https://issues.apache.org/jira/browse/HBASE-20647 > Project: HBase > Issue Type: Sub-task > Components: backport >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20647.branch-1.001.patch > > > Backport parent issue to branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20647) Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-20647: - Status: Patch Available (was: Open) > Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in > TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1 > - > > Key: HBASE-20647 > URL: https://issues.apache.org/jira/browse/HBASE-20647 > Project: HBase > Issue Type: Sub-task > Components: backport >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20647.branch-1.001.patch > > > Backport parent issue to branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20582) Bump up JRuby version because of some reported vulnerabilities
[
https://issues.apache.org/jira/browse/HBASE-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490139#comment-16490139
]
Sean Busbey edited comment on HBASE-20582 at 5/25/18 2:34 AM:
--
I haven't convinced myself that the enforcer plugin is wrong. I'd like to try
to find their discussion of it somewhere but haven't had a chance to dig for it
yet.
I left a comment on JRuby#4899, just to make sure they're aware that this is
still a thing.
I think maybe for now we go to the latest JRuby version that doesn't have this
issue and then wait for an update? From poking around it looks like JRuby
9.1.13.0 is the last enforcer-blessed release. everything after that has a
module-info.class file.
was (Author: busbey):
I haven't convinced myself that the enforcer plugin is wrong. I'd like to try
to find their discussion of it somewhere but haven't had a chance to dig for it
yet.
I left a commend on JRuby#4899, just to make sure they're aware that this is
still a thing.
I think maybe for now we go to the latest JRuby version that doesn't have this
issue and then wait for an update? From poking around it looks like JRuby
9.1.13.0 is the last enforcer-blessed release. everything after that has a
module-info.class file.
> Bump up JRuby version because of some reported vulnerabilities
> --
>
> Key: HBASE-20582
> URL: https://issues.apache.org/jira/browse/HBASE-20582
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Ankit Singhal
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20582.002.patch, HBASE-20582.patch
>
>
> There are some vulnerabilities reported with two of the libraries used in
> HBase.
> {code:java}
> Jruby(version:9.1.10.0):
> CVE-2009-5147
> CVE-2013-4363
> CVE-2014-4975
> CVE-2014-8080
> CVE-2014-8090
> CVE-2015-3900
> CVE-2015-7551
> CVE-2015-9096
> CVE-2017-0899
> CVE-2017-0900
> CVE-2017-0901
> CVE-2017-0902
> CVE-2017-0903
> CVE-2017-10784
> CVE-2017-14064
> CVE-2017-9224
> CVE-2017-9225
> CVE-2017-9226
> CVE-2017-9227
> CVE-2017-9228
> {code}
> Tool somehow able to relate the vulnerability of Ruby with JRuby(Java
> implementation). (Jackson will be handled in a different issue.)
> Not all of them directly affects HBase but [~elserj] suggested that it is
> better to be on the updated version to avoid issues during an audit in
> security sensitive organization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20582) Bump up JRuby version because of some reported vulnerabilities
[
https://issues.apache.org/jira/browse/HBASE-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490139#comment-16490139
]
Sean Busbey commented on HBASE-20582:
-
I haven't convinced myself that the enforcer plugin is wrong. I'd like to try
to find their discussion of it somewhere but haven't had a chance to dig for it
yet.
I left a commend on JRuby#4899, just to make sure they're aware that this is
still a thing.
I think maybe for now we go to the latest JRuby version that doesn't have this
issue and then wait for an update? From poking around it looks like JRuby
9.1.13.0 is the last enforcer-blessed release. everything after that has a
module-info.class file.
> Bump up JRuby version because of some reported vulnerabilities
> --
>
> Key: HBASE-20582
> URL: https://issues.apache.org/jira/browse/HBASE-20582
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Ankit Singhal
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20582.002.patch, HBASE-20582.patch
>
>
> There are some vulnerabilities reported with two of the libraries used in
> HBase.
> {code:java}
> Jruby(version:9.1.10.0):
> CVE-2009-5147
> CVE-2013-4363
> CVE-2014-4975
> CVE-2014-8080
> CVE-2014-8090
> CVE-2015-3900
> CVE-2015-7551
> CVE-2015-9096
> CVE-2017-0899
> CVE-2017-0900
> CVE-2017-0901
> CVE-2017-0902
> CVE-2017-0903
> CVE-2017-10784
> CVE-2017-14064
> CVE-2017-9224
> CVE-2017-9225
> CVE-2017-9226
> CVE-2017-9227
> CVE-2017-9228
> {code}
> Tool somehow able to relate the vulnerability of Ruby with JRuby(Java
> implementation). (Jackson will be handled in a different issue.)
> Not all of them directly affects HBase but [~elserj] suggested that it is
> better to be on the updated version to avoid issues during an audit in
> security sensitive organization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20645) Fix security_available method in security.rb
[
https://issues.apache.org/jira/browse/HBASE-20645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490138#comment-16490138
]
Hadoop QA commented on HBASE-20645:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
5s{color} | {color:red} The patch generated 2 new + 51 unchanged - 2 fixed = 53
total (was 53) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
0s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
12s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20645 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925068/HBASE-20645.patch |
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux 1f5a103e8237 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31
19:38:41 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1fbce10ff4 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| rubocop | v0.56.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/12958/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12958/testReport/ |
| Max. process+thread count | 2443 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12958/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Fix security_available method in security.rb
> -
>
> Key: HBASE-20645
> URL: https://issues.apache.org/jira/browse/HBASE-20645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20645.patch
>
>
> "exists?" method expects parameter tableName to be String but ACL_TABLE_NAME
> is of org.apache.hadoop.hbase.TableName form.
> {code}
> raise(ArgumentError, 'DISABLED: Security features are not available') unless \
>
> exists?(org.apache.hadoop.hbase.security.access.AccessControlLists::ACL_TABLE_NAME.getNameAsString)
> {code}
> Impact of the bug:-
> So , if a user is running any security related
> command(revoke,user_permission) and there is an exception(MasterNotRunning)
> while checking security capabilities, then instead of seeing the underlying
> exception, user is seeing
> {code}
> ERROR: no method 'valueOf' for arguments (org.apache.hadoop.hbase.TableName)
>
[jira] [Commented] (HBASE-20533) Fix the flaky TestAssignmentManagerMetrics
[ https://issues.apache.org/jira/browse/HBASE-20533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490135#comment-16490135 ] Zheng Hu commented on HBASE-20533: -- OK, will dig this today, Thanks [~busbey] for reminding :-) > Fix the flaky TestAssignmentManagerMetrics > -- > > Key: HBASE-20533 > URL: https://issues.apache.org/jira/browse/HBASE-20533 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > > See: > 1. > https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html > 2. https://builds.apache.org/job/HBASE-Flaky-Tests/30837/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20582) Bump up JRuby version because of some reported vulnerabilities
[
https://issues.apache.org/jira/browse/HBASE-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490131#comment-16490131
]
Josh Elser commented on HBASE-20582:
You are a wizard, Sean. Thanks for finding this.
What's your opinion on what we do? Try to fix maven-enforcer-plugin and get a
release? Rollback jruby and wait for a newer version? Remove the check
temporarily?
> Bump up JRuby version because of some reported vulnerabilities
> --
>
> Key: HBASE-20582
> URL: https://issues.apache.org/jira/browse/HBASE-20582
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Ankit Singhal
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20582.002.patch, HBASE-20582.patch
>
>
> There are some vulnerabilities reported with two of the libraries used in
> HBase.
> {code:java}
> Jruby(version:9.1.10.0):
> CVE-2009-5147
> CVE-2013-4363
> CVE-2014-4975
> CVE-2014-8080
> CVE-2014-8090
> CVE-2015-3900
> CVE-2015-7551
> CVE-2015-9096
> CVE-2017-0899
> CVE-2017-0900
> CVE-2017-0901
> CVE-2017-0902
> CVE-2017-0903
> CVE-2017-10784
> CVE-2017-14064
> CVE-2017-9224
> CVE-2017-9225
> CVE-2017-9226
> CVE-2017-9227
> CVE-2017-9228
> {code}
> Tool somehow able to relate the vulnerability of Ruby with JRuby(Java
> implementation). (Jackson will be handled in a different issue.)
> Not all of them directly affects HBase but [~elserj] suggested that it is
> better to be on the updated version to avoid issues during an audit in
> security sensitive organization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20647) Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-20647: - Component/s: backport > Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in > TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1 > - > > Key: HBASE-20647 > URL: https://issues.apache.org/jira/browse/HBASE-20647 > Project: HBase > Issue Type: Sub-task > Components: backport >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > > Backport parent issue to branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20582) Bump up JRuby version because of some reported vulnerabilities
[
https://issues.apache.org/jira/browse/HBASE-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490126#comment-16490126
]
Sean Busbey commented on HBASE-20582:
-
JRuby has an issue [JRuby#4835https://github.com/jruby/jruby/issues/4835]
tracking Java 9 module info needs, and in 9.1.16+ is supposed to be actively
filtering them via [JRuby#4899|https://github.com/jruby/jruby/issues/4899]
> Bump up JRuby version because of some reported vulnerabilities
> --
>
> Key: HBASE-20582
> URL: https://issues.apache.org/jira/browse/HBASE-20582
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Ankit Singhal
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20582.002.patch, HBASE-20582.patch
>
>
> There are some vulnerabilities reported with two of the libraries used in
> HBase.
> {code:java}
> Jruby(version:9.1.10.0):
> CVE-2009-5147
> CVE-2013-4363
> CVE-2014-4975
> CVE-2014-8080
> CVE-2014-8090
> CVE-2015-3900
> CVE-2015-7551
> CVE-2015-9096
> CVE-2017-0899
> CVE-2017-0900
> CVE-2017-0901
> CVE-2017-0902
> CVE-2017-0903
> CVE-2017-10784
> CVE-2017-14064
> CVE-2017-9224
> CVE-2017-9225
> CVE-2017-9226
> CVE-2017-9227
> CVE-2017-9228
> {code}
> Tool somehow able to relate the vulnerability of Ruby with JRuby(Java
> implementation). (Jackson will be handled in a different issue.)
> Not all of them directly affects HBase but [~elserj] suggested that it is
> better to be on the updated version to avoid issues during an audit in
> security sensitive organization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20582) Bump up JRuby version because of some reported vulnerabilities
[
https://issues.apache.org/jira/browse/HBASE-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490126#comment-16490126
]
Sean Busbey edited comment on HBASE-20582 at 5/25/18 2:13 AM:
--
JRuby has an issue [JRuby#4835|https://github.com/jruby/jruby/issues/4835]
tracking Java 9 module info needs, and in 9.1.16+ is supposed to be actively
filtering them via [JRuby#4899|https://github.com/jruby/jruby/issues/4899]
was (Author: busbey):
JRuby has an issue [JRuby#4835https://github.com/jruby/jruby/issues/4835]
tracking Java 9 module info needs, and in 9.1.16+ is supposed to be actively
filtering them via [JRuby#4899|https://github.com/jruby/jruby/issues/4899]
> Bump up JRuby version because of some reported vulnerabilities
> --
>
> Key: HBASE-20582
> URL: https://issues.apache.org/jira/browse/HBASE-20582
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Ankit Singhal
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20582.002.patch, HBASE-20582.patch
>
>
> There are some vulnerabilities reported with two of the libraries used in
> HBase.
> {code:java}
> Jruby(version:9.1.10.0):
> CVE-2009-5147
> CVE-2013-4363
> CVE-2014-4975
> CVE-2014-8080
> CVE-2014-8090
> CVE-2015-3900
> CVE-2015-7551
> CVE-2015-9096
> CVE-2017-0899
> CVE-2017-0900
> CVE-2017-0901
> CVE-2017-0902
> CVE-2017-0903
> CVE-2017-10784
> CVE-2017-14064
> CVE-2017-9224
> CVE-2017-9225
> CVE-2017-9226
> CVE-2017-9227
> CVE-2017-9228
> {code}
> Tool somehow able to relate the vulnerability of Ruby with JRuby(Java
> implementation). (Jackson will be handled in a different issue.)
> Not all of them directly affects HBase but [~elserj] suggested that it is
> better to be on the updated version to avoid issues during an audit in
> security sensitive organization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20647) Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
Toshihiro Suzuki created HBASE-20647: Summary: Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1 Key: HBASE-20647 URL: https://issues.apache.org/jira/browse/HBASE-20647 Project: HBase Issue Type: Sub-task Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Backport parent issue to branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20582) Bump up JRuby version because of some reported vulnerabilities
[
https://issues.apache.org/jira/browse/HBASE-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490123#comment-16490123
]
Sean Busbey commented on HBASE-20582:
-
looks like this is MENFORCER-300
> Bump up JRuby version because of some reported vulnerabilities
> --
>
> Key: HBASE-20582
> URL: https://issues.apache.org/jira/browse/HBASE-20582
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Ankit Singhal
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20582.002.patch, HBASE-20582.patch
>
>
> There are some vulnerabilities reported with two of the libraries used in
> HBase.
> {code:java}
> Jruby(version:9.1.10.0):
> CVE-2009-5147
> CVE-2013-4363
> CVE-2014-4975
> CVE-2014-8080
> CVE-2014-8090
> CVE-2015-3900
> CVE-2015-7551
> CVE-2015-9096
> CVE-2017-0899
> CVE-2017-0900
> CVE-2017-0901
> CVE-2017-0902
> CVE-2017-0903
> CVE-2017-10784
> CVE-2017-14064
> CVE-2017-9224
> CVE-2017-9225
> CVE-2017-9226
> CVE-2017-9227
> CVE-2017-9228
> {code}
> Tool somehow able to relate the vulnerability of Ruby with JRuby(Java
> implementation). (Jackson will be handled in a different issue.)
> Not all of them directly affects HBase but [~elserj] suggested that it is
> better to be on the updated version to avoid issues during an audit in
> security sensitive organization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20645) Fix security_available method in security.rb
[
https://issues.apache.org/jira/browse/HBASE-20645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490122#comment-16490122
]
Josh Elser commented on HBASE-20645:
Nice! Thanks for the patch, Ankit. I missed the masked
MasterNotRunningException.
Looks good to me, but let's see what QA has to say.
> Fix security_available method in security.rb
> -
>
> Key: HBASE-20645
> URL: https://issues.apache.org/jira/browse/HBASE-20645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20645.patch
>
>
> "exists?" method expects parameter tableName to be String but ACL_TABLE_NAME
> is of org.apache.hadoop.hbase.TableName form.
> {code}
> raise(ArgumentError, 'DISABLED: Security features are not available') unless \
>
> exists?(org.apache.hadoop.hbase.security.access.AccessControlLists::ACL_TABLE_NAME.getNameAsString)
> {code}
> Impact of the bug:-
> So , if a user is running any security related
> command(revoke,user_permission) and there is an exception(MasterNotRunning)
> while checking security capabilities, then instead of seeing the underlying
> exception, user is seeing
> {code}
> ERROR: no method 'valueOf' for arguments (org.apache.hadoop.hbase.TableName)
> on Java::OrgApacheHadoopHbase::TableName
> available overloads:
> (java.lang.String)
> (byte[])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20638) nightly source artifact testing should fail the stage if it's going to report an error on jira
[ https://issues.apache.org/jira/browse/HBASE-20638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490120#comment-16490120 ] Sean Busbey commented on HBASE-20638: - yay! branch-2 said failure here and the job also shows failure for the stage. the failure is still the regression caused by HBASE-20582. > nightly source artifact testing should fail the stage if it's going to report > an error on jira > -- > > Key: HBASE-20638 > URL: https://issues.apache.org/jira/browse/HBASE-20638 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20638.0.patch > > > Looks like the source artifact testing is properly reporting failures on > jira, but is marking the stage on jenkins as successful. that makes it much > hard to track over time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20645) Fix security_available method in security.rb
[
https://issues.apache.org/jira/browse/HBASE-20645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20645:
---
Status: Patch Available (was: Open)
> Fix security_available method in security.rb
> -
>
> Key: HBASE-20645
> URL: https://issues.apache.org/jira/browse/HBASE-20645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20645.patch
>
>
> "exists?" method expects parameter tableName to be String but ACL_TABLE_NAME
> is of org.apache.hadoop.hbase.TableName form.
> {code}
> raise(ArgumentError, 'DISABLED: Security features are not available') unless \
>
> exists?(org.apache.hadoop.hbase.security.access.AccessControlLists::ACL_TABLE_NAME.getNameAsString)
> {code}
> Impact of the bug:-
> So , if a user is running any security related
> command(revoke,user_permission) and there is an exception(MasterNotRunning)
> while checking security capabilities, then instead of seeing the underlying
> exception, user is seeing
> {code}
> ERROR: no method 'valueOf' for arguments (org.apache.hadoop.hbase.TableName)
> on Java::OrgApacheHadoopHbase::TableName
> available overloads:
> (java.lang.String)
> (byte[])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20636) Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
[ https://issues.apache.org/jira/browse/HBASE-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490097#comment-16490097 ] Allan Yang commented on HBASE-20636: I think bump up the HFile version for a new type of bloom filter is too much. Maybe we can go around, like: Set the BLOOM_FILTER_TYPE_KEY to NONE if it is a prefix filter. Add another file info e.g. PREFIX_BLOOM_FILTER_FLAG. The type of the new BF can be set here. The old version of RegionServer will regards the BF as NONE while the new server has enough information. I don't know if it is a good idea, [~apurtell], [~andrewcheng] > Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED > --- > > Key: HBASE-20636 > URL: https://issues.apache.org/jira/browse/HBASE-20636 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-20636.master.001.patch, > HBASE-20636.master.002.patch > > > As we all know, HBase uses BloomFilter(ROW and ROWCOL) to filter unnecessary > files to improve read performance. But they only support Get and do not > support Scan. > In our company(Tencent), many users need to scan all rows with the same > prefix, such as Tencent Game. Game user's some operational record will be > written into HBase, each game user will have a lot of records, the rowkey is > constructed as userid+'#'+timestamps. So we can scan all records for a given > user for a specified period. > For this scenario, we designed the prefix Bloom filter. If the startRow and > stopRow of the Scan has a valid common prefix, the scan will be allowed to > use BloomFilter to filter files which will enhance the performance of the > scan. > Now, this feature has been running on our cluster over a year, and scan > performance for this scenario has been improved by more than one times than > before. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20636) Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
[ https://issues.apache.org/jira/browse/HBASE-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20636: Component/s: scan HFile > Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED > --- > > Key: HBASE-20636 > URL: https://issues.apache.org/jira/browse/HBASE-20636 > Project: HBase > Issue Type: New Feature > Components: HFile, regionserver, scan >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-20636.master.001.patch, > HBASE-20636.master.002.patch > > > As we all know, HBase uses BloomFilter(ROW and ROWCOL) to filter unnecessary > files to improve read performance. But they only support Get and do not > support Scan. > In our company(Tencent), many users need to scan all rows with the same > prefix, such as Tencent Game. Game user's some operational record will be > written into HBase, each game user will have a lot of records, the rowkey is > constructed as userid+'#'+timestamps. So we can scan all records for a given > user for a specified period. > For this scenario, we designed the prefix Bloom filter. If the startRow and > stopRow of the Scan has a valid common prefix, the scan will be allowed to > use BloomFilter to filter files which will enhance the performance of the > scan. > Now, this feature has been running on our cluster over a year, and scan > performance for this scenario has been improved by more than one times than > before. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20518) Need to serialize the enabled field for UpdatePeerConfigProcedure
[ https://issues.apache.org/jira/browse/HBASE-20518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-20518: --- Attachment: HBASE-20518.v01.patch Status: Patch Available (was: Open) > Need to serialize the enabled field for UpdatePeerConfigProcedure > - > > Key: HBASE-20518 > URL: https://issues.apache.org/jira/browse/HBASE-20518 > Project: HBase > Issue Type: Sub-task > Components: Replication >Affects Versions: 3.0.0, 2.1.0 >Reporter: Duo Zhang >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20518.v01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[
https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490095#comment-16490095
]
Hadoop QA commented on HBASE-20172:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
22s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
25s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
51s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}103m 8s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:36a7029 |
| JIRA Issue | HBASE-20172 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925051/HBASE-20172-branch-1.patch
|
| Optional Tests | asflicense javac
[jira] [Commented] (HBASE-20636) Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
[ https://issues.apache.org/jira/browse/HBASE-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490084#comment-16490084 ] Andrew Purtell commented on HBASE-20636: bq. t reviewed the patch, this does not need a HFile version increment. If we end up writing a HFile that cannot be read by an earlier version of HBase, HFile needs a version increment. bq. It the old version of RegionServer can't recognize the bloom filter type, it will throw a IllegalArgumentException here. And this is why. > Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED > --- > > Key: HBASE-20636 > URL: https://issues.apache.org/jira/browse/HBASE-20636 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-20636.master.001.patch, > HBASE-20636.master.002.patch > > > As we all know, HBase uses BloomFilter(ROW and ROWCOL) to filter unnecessary > files to improve read performance. But they only support Get and do not > support Scan. > In our company(Tencent), many users need to scan all rows with the same > prefix, such as Tencent Game. Game user's some operational record will be > written into HBase, each game user will have a lot of records, the rowkey is > constructed as userid+'#'+timestamps. So we can scan all records for a given > user for a specified period. > For this scenario, we designed the prefix Bloom filter. If the startRow and > stopRow of the Scan has a valid common prefix, the scan will be allowed to > use BloomFilter to filter files which will enhance the performance of the > scan. > Now, this feature has been running on our cluster over a year, and scan > performance for this scenario has been improved by more than one times than > before. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490081#comment-16490081 ] Andrew Purtell commented on HBASE-20172: It's repeatable for me. I'm using java version "1.8.0_162" on MacOS. Can't commit until this is addressed somehow sorry. > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20636) Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
[
https://issues.apache.org/jira/browse/HBASE-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490078#comment-16490078
]
Allan Yang edited comment on HBASE-20636 at 5/25/18 1:16 AM:
-
{quote}
Does this require a HFile version increment?
{quote}
[~apurtell], just reviewed the patch, this does not need a HFile version
increment.
{quote}
There should also be a fallback. We can at least process the HFile without
blooms if we don't recognize the bloom filter type.
{quote}
Well, I think it is hard to fallback, since we read bloom filter type like this
in StoreFileReader.loadFileInfo():
{code:java}
byte[] b = fi.get(BLOOM_FILTER_TYPE_KEY);
if (b != null) {
bloomFilterType = BloomType.valueOf(Bytes.toString(b));
}
{code}
It the old version of RegionServer can't recognize the bloom filter type, it
will throw a IllegalArgumentException here.
was (Author: allan163):
{quote}
Does this require a HFile version increment?
{quote}
[~apurte4ll], just reviewed the patch, this does not need a HFile version
increment.
{quote}
There should also be a fallback. We can at least process the HFile without
blooms if we don't recognize the bloom filter type.
{quote}
Well, I think it is hard to fallback, since we read bloom filter type like this
in StoreFileReader.loadFileInfo():
{code:java}
byte[] b = fi.get(BLOOM_FILTER_TYPE_KEY);
if (b != null) {
bloomFilterType = BloomType.valueOf(Bytes.toString(b));
}
{code}
It the old version of RegionServer can't recognize the bloom filter type, it
will throw a IllegalArgumentException here.
> Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
> ---
>
> Key: HBASE-20636
> URL: https://issues.apache.org/jira/browse/HBASE-20636
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-20636.master.001.patch,
> HBASE-20636.master.002.patch
>
>
> As we all know, HBase uses BloomFilter(ROW and ROWCOL) to filter unnecessary
> files to improve read performance. But they only support Get and do not
> support Scan.
> In our company(Tencent), many users need to scan all rows with the same
> prefix, such as Tencent Game. Game user's some operational record will be
> written into HBase, each game user will have a lot of records, the rowkey is
> constructed as userid+'#'+timestamps. So we can scan all records for a given
> user for a specified period.
> For this scenario, we designed the prefix Bloom filter. If the startRow and
> stopRow of the Scan has a valid common prefix, the scan will be allowed to
> use BloomFilter to filter files which will enhance the performance of the
> scan.
> Now, this feature has been running on our cluster over a year, and scan
> performance for this scenario has been improved by more than one times than
> before.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20636) Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
[
https://issues.apache.org/jira/browse/HBASE-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490078#comment-16490078
]
Allan Yang commented on HBASE-20636:
{quote}
Does this require a HFile version increment?
{quote}
[~apurte4ll], just reviewed the patch, this does not need a HFile version
increment.
{quote}
There should also be a fallback. We can at least process the HFile without
blooms if we don't recognize the bloom filter type.
{quote}
Well, I think it is hard to fallback, since we read bloom filter type like this
in StoreFileReader.loadFileInfo():
{code:java}
byte[] b = fi.get(BLOOM_FILTER_TYPE_KEY);
if (b != null) {
bloomFilterType = BloomType.valueOf(Bytes.toString(b));
}
{code}
It the old version of RegionServer can't recognize the bloom filter type, it
will throw a IllegalArgumentException here.
> Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
> ---
>
> Key: HBASE-20636
> URL: https://issues.apache.org/jira/browse/HBASE-20636
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-20636.master.001.patch,
> HBASE-20636.master.002.patch
>
>
> As we all know, HBase uses BloomFilter(ROW and ROWCOL) to filter unnecessary
> files to improve read performance. But they only support Get and do not
> support Scan.
> In our company(Tencent), many users need to scan all rows with the same
> prefix, such as Tencent Game. Game user's some operational record will be
> written into HBase, each game user will have a lot of records, the rowkey is
> constructed as userid+'#'+timestamps. So we can scan all records for a given
> user for a specified period.
> For this scenario, we designed the prefix Bloom filter. If the startRow and
> stopRow of the Scan has a valid common prefix, the scan will be allowed to
> use BloomFilter to filter files which will enhance the performance of the
> scan.
> Now, this feature has been running on our cluster over a year, and scan
> performance for this scenario has been improved by more than one times than
> before.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[
https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490074#comment-16490074
]
Ankit Singhal edited comment on HBASE-20172 at 5/25/18 1:14 AM:
[~apurtell], I'm not able to reproduce the test failure locally on branch-1.
{code}
[ERROR] Tests run: 84, Failures: 1, Errors: 0, Skipped: 4, Time elapsed:
227.777 s <<< FAILURE! - in
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
[ERROR]
testCheckAndDeleteWithCompareOp(org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor)
Time elapsed: 1.337 s <<< FAILURE!
java.lang.AssertionError: expected: but was:
{code}
was (Author: an...@apache.org):
[~apurtell], I'm not able to reproduce the test failure locally.
{code}
[ERROR] Tests run: 84, Failures: 1, Errors: 0, Skipped: 4, Time elapsed:
227.777 s <<< FAILURE! - in
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
[ERROR]
testCheckAndDeleteWithCompareOp(org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor)
Time elapsed: 1.337 s <<< FAILURE!
java.lang.AssertionError: expected: but was:
{code}
> During coprocessor load, switch classloader only if it's a custom CP.
> -
>
> Key: HBASE-20172
> URL: https://issues.apache.org/jira/browse/HBASE-20172
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch
>
>
> Current Impact:-
> Metric registries will not be able to load its implementation through service
> loader and etc.
> We are not observing this with Java8 because ServiceLoader uses system class
> loader if provided class loader is null but it gets exposed with Java7
> easily(TEPHRA-285)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[
https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490074#comment-16490074
]
Ankit Singhal commented on HBASE-20172:
---
[~apurtell], I'm not able to reproduce the test failure locally.
{code}
[ERROR] Tests run: 84, Failures: 1, Errors: 0, Skipped: 4, Time elapsed:
227.777 s <<< FAILURE! - in
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
[ERROR]
testCheckAndDeleteWithCompareOp(org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor)
Time elapsed: 1.337 s <<< FAILURE!
java.lang.AssertionError: expected: but was:
{code}
> During coprocessor load, switch classloader only if it's a custom CP.
> -
>
> Key: HBASE-20172
> URL: https://issues.apache.org/jira/browse/HBASE-20172
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch
>
>
> Current Impact:-
> Metric registries will not be able to load its implementation through service
> loader and etc.
> We are not observing this with Java8 because ServiceLoader uses system class
> loader if provided class loader is null but it gets exposed with Java7
> easily(TEPHRA-285)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20646) TestWALProcedureStoreOnHDFS failing on branch-1
[ https://issues.apache.org/jira/browse/HBASE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-20646. Resolution: Fixed > TestWALProcedureStoreOnHDFS failing on branch-1 > --- > > Key: HBASE-20646 > URL: https://issues.apache.org/jira/browse/HBASE-20646 > Project: HBase > Issue Type: Test >Affects Versions: 1.4.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 1.5.0, 1.4.5 > > Attachments: HBASE-20646-branch-1.patch > > > TestWALProcedureStoreOnHDFS fails sometimes on branch-1 depending on junit > particulars. An @After decoration was improperly added. Remove to fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[
https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490062#comment-16490062
]
Hadoop QA commented on HBASE-20172:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
4s{color} | {color:blue} hbase-server in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}137m
1s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}186m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20172 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913929/HBASE-20172.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 24eaf9798724 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1792f541c6 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12954/testReport/ |
| Max. process+thread count | 4854 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Updated] (HBASE-20646) TestWALProcedureStoreOnHDFS failing on branch-1
[ https://issues.apache.org/jira/browse/HBASE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20646: --- Attachment: HBASE-20646-branch-1.patch > TestWALProcedureStoreOnHDFS failing on branch-1 > --- > > Key: HBASE-20646 > URL: https://issues.apache.org/jira/browse/HBASE-20646 > Project: HBase > Issue Type: Test >Affects Versions: 1.4.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 1.5.0, 1.4.5 > > Attachments: HBASE-20646-branch-1.patch > > > TestWALProcedureStoreOnHDFS fails sometimes on branch-1 depending on junit > particulars. An @After decoration was improperly added. Remove to fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20646) TestWALProcedureStoreOnHDFS failing on branch-1
[ https://issues.apache.org/jira/browse/HBASE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490070#comment-16490070 ] Andrew Purtell commented on HBASE-20646: Going to commit trivial test fix shortly unless objection > TestWALProcedureStoreOnHDFS failing on branch-1 > --- > > Key: HBASE-20646 > URL: https://issues.apache.org/jira/browse/HBASE-20646 > Project: HBase > Issue Type: Test >Affects Versions: 1.4.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 1.5.0, 1.4.5 > > Attachments: HBASE-20646-branch-1.patch > > > TestWALProcedureStoreOnHDFS fails sometimes on branch-1 depending on junit > particulars. An @After decoration was improperly added. Remove to fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20646) TestWALProcedureStoreOnHDFS failing on branch-1
Andrew Purtell created HBASE-20646: -- Summary: TestWALProcedureStoreOnHDFS failing on branch-1 Key: HBASE-20646 URL: https://issues.apache.org/jira/browse/HBASE-20646 Project: HBase Issue Type: Test Affects Versions: 1.4.4 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.5.0, 1.4.5 TestWALProcedureStoreOnHDFS fails sometimes on branch-1 depending on junit particulars. An @After decoration was improperly added. Remove to fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20478) move import checks from hbaseanti to checkstyle
[
https://issues.apache.org/jira/browse/HBASE-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490057#comment-16490057
]
Hudson commented on HBASE-20478:
Results for branch HBASE-20478
[build #5 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20478/5/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20478/5//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20478/5//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20478/5//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> move import checks from hbaseanti to checkstyle
> ---
>
> Key: HBASE-20478
> URL: https://issues.apache.org/jira/browse/HBASE-20478
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Mike Drob
>Priority: Minor
> Attachments: HBASE-20478.0.patch, HBASE-20478.1.patch,
> HBASE-20478.2.patch, HBASE-20478.3.patch, HBASE-20478.4.patch,
> HBASE-20478.WIP.2.patch, HBASE-20478.WIP.2.patch, HBASE-20478.WIP.patch,
> HBASE-anti-check.patch
>
>
> came up in discussion on HBASE-20332. our check of "don't do this" things in
> the codebase doesn't log the specifics of complaints anywhere, which forces
> those who want to follow up to reverse engineer the check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490055#comment-16490055 ] Ankit Singhal commented on HBASE-20172: --- bq. Not all coprocessor unit tests pass once this change is applied. For example Thanks for verifying, let me take a look at the failure. > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19722) Implement a meta query statistics metrics source
[ https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490033#comment-16490033 ] Xu Cang commented on HBASE-19722: - patch version 013 contains logic using lossy count to maintain high-frequency client request metrics. > Implement a meta query statistics metrics source > > > Key: HBASE-19722 > URL: https://issues.apache.org/jira/browse/HBASE-19722 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-19722.branch-1.v001.patch, > HBASE-19722.master.010.patch, HBASE-19722.master.011.patch, > HBASE-19722.master.012.patch, HBASE-19722.master.013.patch > > > Implement a meta query statistics metrics source, created whenever a > regionserver starts hosting meta, removed when meta hosting moves. Provide > views on top tables by request counts, top meta rowkeys by request count, top > clients making requests by their hostname. > Can be implemented as a coprocessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19722) Implement a meta query statistics metrics source
[ https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-19722: Attachment: HBASE-19722.master.013.patch > Implement a meta query statistics metrics source > > > Key: HBASE-19722 > URL: https://issues.apache.org/jira/browse/HBASE-19722 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Xu Cang >Priority: Major > Attachments: HBASE-19722.branch-1.v001.patch, > HBASE-19722.master.010.patch, HBASE-19722.master.011.patch, > HBASE-19722.master.012.patch, HBASE-19722.master.013.patch > > > Implement a meta query statistics metrics source, created whenever a > regionserver starts hosting meta, removed when meta hosting moves. Provide > views on top tables by request counts, top meta rowkeys by request count, top > clients making requests by their hostname. > Can be implemented as a coprocessor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20601) Add multiPut support and other miscellaneous to PE
[
https://issues.apache.org/jira/browse/HBASE-20601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490049#comment-16490049
]
Allan Yang commented on HBASE-20601:
{quote}
something that may be useful is setting branch.master.rebase to true in your
git config - it's a little bit of black magic but has worked well for me so
far. https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-config.html
(search for branch..rebase)
{quote}
Thanks! [~mdrob]
> Add multiPut support and other miscellaneous to PE
> --
>
> Key: HBASE-20601
> URL: https://issues.apache.org/jira/browse/HBASE-20601
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Minor
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20601.002.patch, HBASE-20601.003.patch,
> HBASE-20601.branch-2.002.patch, HBASE-20601.branch-2.003.patch,
> HBASE-20601.branch-2.004.patch, HBASE-20601.branch-2.005.patch,
> HBASE-20601.branch-2.006.patch, HBASE-20601.branch-2.patch, HBASE-20601.patch
>
>
> Add some useful stuff and some refinement to PE tool
> 1. Add multiPut support
> Though we have BufferedMutator, sometimes we need to benchmark batch put in a
> certain number.
> Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set
> to false)
> 2. Add Connection Number support
> Before, there is only on parameter to control the connection used by threads.
> oneCon=true means all threads use one connection, false means each thread has
> it own connection.
> When thread number is high and oneCon=false, we noticed high context switch
> frequency in the machine which PE run on, disturbing the benchmark
> results(each connection has its own netty worker threads, 2*CPU IIRC).
> So, added a new parameter connCount to PE. set --connCount=2 means all
> threads will share 2 connections.
> 3. Add avg RT and avg TPS/QPS statstic for all threads
> Useful when we want to meansure the total throughtput of the cluster
> 4. Delete some redundant code
> Now RandomWriteTest is inherited from SequentialWrite.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20645) Fix security_available method in security.rb
[
https://issues.apache.org/jira/browse/HBASE-20645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated HBASE-20645:
--
Attachment: HBASE-20645.patch
> Fix security_available method in security.rb
> -
>
> Key: HBASE-20645
> URL: https://issues.apache.org/jira/browse/HBASE-20645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20645.patch
>
>
> "exists?" method expects parameter tableName to be String but ACL_TABLE_NAME
> is of org.apache.hadoop.hbase.TableName form.
> {code}
> raise(ArgumentError, 'DISABLED: Security features are not available') unless \
>
> exists?(org.apache.hadoop.hbase.security.access.AccessControlLists::ACL_TABLE_NAME.getNameAsString)
> {code}
> Impact of the bug:-
> So , if a user is running any security related
> command(revoke,user_permission) and there is an exception(MasterNotRunning)
> while checking security capabilities, then instead of seeing the underlying
> exception, user is seeing
> {code}
> ERROR: no method 'valueOf' for arguments (org.apache.hadoop.hbase.TableName)
> on Java::OrgApacheHadoopHbase::TableName
> available overloads:
> (java.lang.String)
> (byte[])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20645) Fix security_available method in security.rb
Ankit Singhal created HBASE-20645:
-
Summary: Fix security_available method in security.rb
Key: HBASE-20645
URL: https://issues.apache.org/jira/browse/HBASE-20645
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Ankit Singhal
Assignee: Ankit Singhal
Attachments: HBASE-20645.patch
"exists?" method expects parameter tableName to be String but ACL_TABLE_NAME is
of org.apache.hadoop.hbase.TableName form.
{code}
raise(ArgumentError, 'DISABLED: Security features are not available') unless \
exists?(org.apache.hadoop.hbase.security.access.AccessControlLists::ACL_TABLE_NAME.getNameAsString)
{code}
Impact of the bug:-
So , if a user is running any security related command(revoke,user_permission)
and there is an exception(MasterNotRunning) while checking security
capabilities, then instead of seeing the underlying exception, user is seeing
{code}
ERROR: no method 'valueOf' for arguments (org.apache.hadoop.hbase.TableName) on
Java::OrgApacheHadoopHbase::TableName
available overloads:
(java.lang.String)
(byte[])
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20645) Fix security_available method in security.rb
[
https://issues.apache.org/jira/browse/HBASE-20645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490048#comment-16490048
]
Ankit Singhal commented on HBASE-20645:
---
FYI [~elserj]
> Fix security_available method in security.rb
> -
>
> Key: HBASE-20645
> URL: https://issues.apache.org/jira/browse/HBASE-20645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20645.patch
>
>
> "exists?" method expects parameter tableName to be String but ACL_TABLE_NAME
> is of org.apache.hadoop.hbase.TableName form.
> {code}
> raise(ArgumentError, 'DISABLED: Security features are not available') unless \
>
> exists?(org.apache.hadoop.hbase.security.access.AccessControlLists::ACL_TABLE_NAME.getNameAsString)
> {code}
> Impact of the bug:-
> So , if a user is running any security related
> command(revoke,user_permission) and there is an exception(MasterNotRunning)
> while checking security capabilities, then instead of seeing the underlying
> exception, user is seeing
> {code}
> ERROR: no method 'valueOf' for arguments (org.apache.hadoop.hbase.TableName)
> on Java::OrgApacheHadoopHbase::TableName
> available overloads:
> (java.lang.String)
> (byte[])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[
https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490042#comment-16490042
]
Andrew Purtell commented on HBASE-20172:
Not all coprocessor unit tests pass once this change is applied. For example
{noformat}
[ERROR] Tests run: 84, Failures: 1, Errors: 0, Skipped: 4, Time elapsed:
227.777 s <<< FAILURE! - in
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
[ERROR]
testCheckAndDeleteWithCompareOp(org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor)
Time elapsed: 1.337 s <<< FAILURE!
java.lang.AssertionError: expected: but was:
{noformat}
[~an...@apache.org]
> During coprocessor load, switch classloader only if it's a custom CP.
> -
>
> Key: HBASE-20172
> URL: https://issues.apache.org/jira/browse/HBASE-20172
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch
>
>
> Current Impact:-
> Metric registries will not be able to load its implementation through service
> loader and etc.
> We are not observing this with Java8 because ServiceLoader uses system class
> loader if provided class loader is null but it gets exposed with Java7
> easily(TEPHRA-285)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20616) TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
[
https://issues.apache.org/jira/browse/HBASE-20616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490041#comment-16490041
]
Toshihiro Suzuki commented on HBASE-20616:
--
Thank you for reviewing and committing the patch [~yuzhih...@gmail.com].
> TruncateTableProcedure is stuck in retry loop in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state
> --
>
> Key: HBASE-20616
> URL: https://issues.apache.org/jira/browse/HBASE-20616
> Project: HBase
> Issue Type: Bug
> Components: amv2
> Environment: HDP-2.5.3
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 20616.master.004.patch, HBASE-20616.master.001.patch,
> HBASE-20616.master.002.patch, HBASE-20616.master.003.patch,
> HBASE-20616.master.004.patch
>
>
> At first, TruncateTableProcedure failed to write some files to HDFS in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state for some reason.
> {code:java}
> 2018-05-15 08:00:25,346 WARN [ProcedureExecutorThread-8]
> procedure.TruncateTableProcedure: Retriable error trying to truncate
> table=: state=TRUNCATE_TABLE_CREATE_FS_LAYOUT
> java.io.IOException: java.util.concurrent.ExecutionException:
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /apps/hbase/data/.tmp/data.regioninfo could
> only be replicated to 0 nodes instead of minReplication (=1). There are number of DNs> datanode(s) running and no node(s) are excluded in this
> operation.
> ...
> {code}
> But at this time, seemed like writing some files to HDFS was successful.
> And then, TruncateTableProcedure was stuck in retry loop in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state. At this point, the following log
> messages were shown repeatedly in the master log:
> {code:java}
> 2018-05-15 08:00:25,463 WARN [ProcedureExecutorThread-8]
> procedure.TruncateTableProcedure: Retriable error trying to truncate
> table=: state=TRUNCATE_TABLE_CREATE_FS_LAYOUT
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs:///apps/hbase/data/.tmp/data///
> ...
> {code}
> It seems like this is because TruncateTableProcedure tried to write the files
> that were written successfully in the first try.
> I think we need to delete all the files and directories that are written
> successfully in the previous try before retrying the
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state.
> Actually, this issue was observed in HDP-2.5.3, but I think the upstream has
> the same issue. Also, it looks to me that CreateTableProcedure has a similar
> issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20548) Master fails to startup on large clusters, refreshing block distribution
[ https://issues.apache.org/jira/browse/HBASE-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1649#comment-1649 ] Hudson commented on HBASE-20548: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #413 (See [https://builds.apache.org/job/HBase-1.3-IT/413/]) HBASE-20548 Master fails to startup on large clusters, refreshing block (apurtell: rev b62d12ffcb5a70ed60c577abe8722153a642d01f) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java > Master fails to startup on large clusters, refreshing block distribution > > > Key: HBASE-20548 > URL: https://issues.apache.org/jira/browse/HBASE-20548 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.4 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20548.branch-1.4.001.patch, > HBASE-20548.branch-2.0.001.patch, HBASE-20548.master.001.patch > > > On our large clusters with, master has failed to startup within specified > time and aborted itself since it was initializing HDFS block distribution. > Enable table also takes time for larger tables for the same reason. My > proposal is to refresh HDFS block distribution at the end of master > initialization and not at retainAssignment()'s createCluster(). This would > address HBASE-16570's intention, but avoid the problems we ran into. > cc [~aoxiang] [~tedyu] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20548) Master fails to startup on large clusters, refreshing block distribution
[ https://issues.apache.org/jira/browse/HBASE-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20548: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.3.3 2.1.0 3.0.0 Status: Resolved (was: Patch Available) Applied to branch-1.3 and up. Ran all o.a.h.h.master.\*\* tests on all branches (so, all balancer, assignment, and region deployment coverage), all passed. > Master fails to startup on large clusters, refreshing block distribution > > > Key: HBASE-20548 > URL: https://issues.apache.org/jira/browse/HBASE-20548 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.4 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20548.branch-1.4.001.patch, > HBASE-20548.branch-2.0.001.patch, HBASE-20548.master.001.patch > > > On our large clusters with, master has failed to startup within specified > time and aborted itself since it was initializing HDFS block distribution. > Enable table also takes time for larger tables for the same reason. My > proposal is to refresh HDFS block distribution at the end of master > initialization and not at retainAssignment()'s createCluster(). This would > address HBASE-16570's intention, but avoid the problems we ran into. > cc [~aoxiang] [~tedyu] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489978#comment-16489978 ] Andrew Purtell commented on HBASE-20172: Sure, committing after some local checks, thanks [~an...@apache.org] > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489971#comment-16489971 ] Ankit Singhal commented on HBASE-20172: --- Thanks [~apurtell], I have uploaded a patch for branch-1(applicable for other branch-1.* as well). can you please commit it. > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-20172: -- Attachment: HBASE-20172-branch-1.patch > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20638) nightly source artifact testing should fail the stage if it's going to report an error on jira
[
https://issues.apache.org/jira/browse/HBASE-20638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489953#comment-16489953
]
Hudson commented on HBASE-20638:
Results for branch branch-1.3
[build #340 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> nightly source artifact testing should fail the stage if it's going to report
> an error on jira
> --
>
> Key: HBASE-20638
> URL: https://issues.apache.org/jira/browse/HBASE-20638
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20638.0.patch
>
>
> Looks like the source artifact testing is properly reporting failures on
> jira, but is marking the stage on jenkins as successful. that makes it much
> hard to track over time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20597) Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
[
https://issues.apache.org/jira/browse/HBASE-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489952#comment-16489952
]
Hudson commented on HBASE-20597:
Results for branch branch-1.3
[build #340 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/340//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Use a lock to serialize access to a shared reference to ZooKeeperWatcher in
> HBaseReplicationEndpoint
>
>
> Key: HBASE-20597
> URL: https://issues.apache.org/jira/browse/HBASE-20597
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2, 1.4.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20597-branch-1.patch, HBASE-20597.patch
>
>
> The code that closes down a ZKW that fails to initialize when attempting to
> connect to the remote cluster is not MT safe and can in theory leak
> ZooKeeperWatcher instances. The allocation of a new ZKW and store to the
> reference is not atomic. Might have concurrent allocations with only one
> winning store, leading to leaked ZKW instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
[
https://issues.apache.org/jira/browse/HBASE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489951#comment-16489951
]
Ted Yu commented on HBASE-20644:
Master was waiting for server to check in:
{code}
2018-05-23 21:54:27,893 WARN
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.AssignmentManager: No servers available; cannot place 1 unassigned
regions.
2018-05-23 21:54:28,877 INFO
[master/ctr-e138-1518143905142-329221-01-03:2] master.ServerManager:
Waiting on regionserver count=0; waited=42119ms, expecting min=1 server(s),
max=NO_LIMIT server(s), timeout=3ms, lastChange=-42119ms
{code}
Finally when 002 checked in, master started to assign regions to it:
{code}
2018-05-23 21:54:28,984 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
master.ServerManager: Registering
regionserver=ctr-e138-1518143905142-329221-01-
02.hwx.site,16020,1527112463065
2018-05-23 21:54:29,033 INFO
[master/ctr-e138-1518143905142-329221-01-03:2] master.ServerManager:
Waiting on regionserver count=1; waited=42274ms, expecting min=1 server(s),
max=NO_LIMIT server(s), timeout=3ms, lastChange=0ms
{code}
Here is related code:
{code}
for (ServerName serverName: offlineServersWithOnlineRegions) {
if (!master.getServerManager().isServerOnline(serverName)) {
LOG.info("KILL RegionServer=" + serverName + " hosting regions but not
online.");
killRegionServer(serverName);
{code}
This is how AM handles offline servers.
> Master shutdown due to service ClusterSchemaServiceImpl failing to start
>
>
> Key: HBASE-20644
> URL: https://issues.apache.org/jira/browse/HBASE-20644
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Priority: Major
> Attachments:
> 101383-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-02.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
>
>
> From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
> {code}
> 2018-05-23 22:14:29,750 ERROR
> [master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
> to become active master
> java.lang.IllegalStateException: Expected the service
> ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
> {code}
> Earlier in the log , the namespace region, 01a7f9ba9fffd691f261d3fbc620da06 ,
> was deemed OPEN on 01-07.hwx.site,16020,1527112194788 which was declared
> not online:
> {code}
> 2018-05-23 21:54:34,786 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.RegionStateStore: Load hbase:meta entry
> region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
> lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
>
> regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
> seqnum=43
> 2018-05-23 21:54:34,787 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: Number of RegionServers=1
> 2018-05-23 21:54:34,788 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: KILL
> RegionServer=ctr-e138-1518143905142-329221-01-07.
> hwx.site,16020,1527112194788 hosting regions but not online.
> {code}
> Later, even though a different instance on 007 registered with master:
> {code}
> 2018-05-23 21:55:13,541 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.ServerManager: Registering
> regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
> ...
> 2018-05-23 21:55:43,881 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
> started=69001 ms ago,cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not
> online on ctr-e138-1518143905142-329221-
> 01-07.hwx.site,16020,1527112506002
> at
>
[jira] [Commented] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
[
https://issues.apache.org/jira/browse/HBASE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489943#comment-16489943
]
Josh Elser commented on HBASE-20644:
{noformat}
2018-05-23 21:54:34,788 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.AssignmentManager: KILL
RegionServer=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788
hosting regions but not online.{noformat}
This seems bad. How did we get into this place in the first place? Maybe a
region which was once on this RegionServer is in transition as a result of this?
Are there two issues to untangle here?
> Master shutdown due to service ClusterSchemaServiceImpl failing to start
>
>
> Key: HBASE-20644
> URL: https://issues.apache.org/jira/browse/HBASE-20644
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Priority: Major
> Attachments:
> 101383-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-02.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
>
>
> From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
> {code}
> 2018-05-23 22:14:29,750 ERROR
> [master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
> to become active master
> java.lang.IllegalStateException: Expected the service
> ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
> {code}
> Earlier in the log , the namespace region, 01a7f9ba9fffd691f261d3fbc620da06 ,
> was deemed OPEN on 01-07.hwx.site,16020,1527112194788 which was declared
> not online:
> {code}
> 2018-05-23 21:54:34,786 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.RegionStateStore: Load hbase:meta entry
> region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
> lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
>
> regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
> seqnum=43
> 2018-05-23 21:54:34,787 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: Number of RegionServers=1
> 2018-05-23 21:54:34,788 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: KILL
> RegionServer=ctr-e138-1518143905142-329221-01-07.
> hwx.site,16020,1527112194788 hosting regions but not online.
> {code}
> Later, even though a different instance on 007 registered with master:
> {code}
> 2018-05-23 21:55:13,541 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.ServerManager: Registering
> regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
> ...
> 2018-05-23 21:55:43,881 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
> started=69001 ms ago,cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not
> online on ctr-e138-1518143905142-329221-
> 01-07.hwx.site,16020,1527112506002
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2446)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> {code}
> There was no OPEN request for 01a7f9ba9fffd691f261d3fbc620da06 sent to that
> server instance.
> From
> hbase-hbase-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
> :
> {code}
> 2018-05-23 21:52:27,414 INFO
>
[jira] [Updated] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
[
https://issues.apache.org/jira/browse/HBASE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20644:
---
Affects Version/s: 2.0.0
> Master shutdown due to service ClusterSchemaServiceImpl failing to start
>
>
> Key: HBASE-20644
> URL: https://issues.apache.org/jira/browse/HBASE-20644
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Priority: Major
> Attachments:
> 101383-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-02.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
>
>
> From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
> {code}
> 2018-05-23 22:14:29,750 ERROR
> [master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
> to become active master
> java.lang.IllegalStateException: Expected the service
> ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
> {code}
> Earlier in the log , the namespace region, 01a7f9ba9fffd691f261d3fbc620da06 ,
> was deemed OPEN on 01-07.hwx.site,16020,1527112194788 which was declared
> not online:
> {code}
> 2018-05-23 21:54:34,786 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.RegionStateStore: Load hbase:meta entry
> region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
> lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
>
> regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
> seqnum=43
> 2018-05-23 21:54:34,787 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: Number of RegionServers=1
> 2018-05-23 21:54:34,788 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: KILL
> RegionServer=ctr-e138-1518143905142-329221-01-07.
> hwx.site,16020,1527112194788 hosting regions but not online.
> {code}
> Later, even though a different instance on 007 registered with master:
> {code}
> 2018-05-23 21:55:13,541 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.ServerManager: Registering
> regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
> ...
> 2018-05-23 21:55:43,881 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
> started=69001 ms ago,cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not
> online on ctr-e138-1518143905142-329221-
> 01-07.hwx.site,16020,1527112506002
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2446)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> {code}
> There was no OPEN request for 01a7f9ba9fffd691f261d3fbc620da06 sent to that
> server instance.
> From
> hbase-hbase-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
> :
> {code}
> 2018-05-23 21:52:27,414 INFO
> [RS_CLOSE_REGION-regionserver/ctr-e138-1518143905142-329221-01-07:16020-1]
> regionserver.HRegion: Closed hbase:namespace,,1527099443383.
> 01a7f9ba9fffd691f261d3fbc620da06.
> {code}
> Then region server 007 restarted:
> {code}
> Wed May 23 21:55:03 UTC 2018 Starting regionserver on
> ctr-e138-1518143905142-329221-01-07.hwx.site
> {code}
> After which the region 01a7f9ba9fffd691f261d3fbc620da06 never showed up again
> in log 007
--
This message was sent by Atlassian
[jira] [Commented] (HBASE-20638) nightly source artifact testing should fail the stage if it's going to report an error on jira
[
https://issues.apache.org/jira/browse/HBASE-20638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489940#comment-16489940
]
Hudson commented on HBASE-20638:
Results for branch branch-2.0
[build #343 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/343/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/343//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/343//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/343//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> nightly source artifact testing should fail the stage if it's going to report
> an error on jira
> --
>
> Key: HBASE-20638
> URL: https://issues.apache.org/jira/browse/HBASE-20638
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20638.0.patch
>
>
> Looks like the source artifact testing is properly reporting failures on
> jira, but is marking the stage on jenkins as successful. that makes it much
> hard to track over time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20604) ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled
[
https://issues.apache.org/jira/browse/HBASE-20604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489936#comment-16489936
]
Andrew Purtell commented on HBASE-20604:
So another corner case introduced by CryptoInputStream? Is there a
corresponding HDFS JIRA?
What guarantees we aren't looping back to the beginning ad infinitum? Should
there be an upper bound on retries? Or is a higher level limit in effect that
will bound this?
> ProtobufLogReader#readNext can incorrectly loop to the same position in the
> stream until the the WAL is rolled
> --
>
> Key: HBASE-20604
> URL: https://issues.apache.org/jira/browse/HBASE-20604
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 3.0.0
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
>Priority: Critical
> Attachments: HBASE-20604.002.patch, HBASE-20604.patch
>
>
> Every time we call {{ProtobufLogReader#readNext}} we consume the input stream
> associated to the {{FSDataInputStream}} from the WAL that we are reading.
> Under certain conditions, e.g. when using the encryption at rest
> ({{CryptoInputStream}}) the stream can return partial data which can cause a
> premature EOF that cause {{inputStream.getPos()}} to return to the same
> origina position causing {{ProtobufLogReader#readNext}} to re-try over the
> reads until the WAL is rolled.
> The side effect of this issue is that {{ReplicationSource}} can get stuck
> until the WAL is rolled and causing replication delays up to an hour in some
> cases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20588) Space quota change after quota violation doesn't seem to take in effect
[
https://issues.apache.org/jira/browse/HBASE-20588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20588:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks so much for the patch and your prompt code-review, Nihal!
> Space quota change after quota violation doesn't seem to take in effect
> ---
>
> Key: HBASE-20588
> URL: https://issues.apache.org/jira/browse/HBASE-20588
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20588.master.001.patch,
> HBASE-20588.master.002.patch, HBASE-20588.master.003.patch,
> HBASE-20588.master.004.patch, HBASE-20588.master.005.patch
>
>
> Steps followed
> - Through {{hbase shell}}
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => '2M', POLICY =>
> NO_INSERTS{noformat}
> - Run {{PE}} until the quota is reached
> {noformat}
> hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred
> --rows=2000 sequentialWrite 1{noformat}
> - Through {{HBase}} shell
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => NONE{noformat}
> - Through {{HBase}} shell verify the effective Quotas
> {noformat}
> > list_quotas
> OWNER QUOTAS
>
>
> 0 row(s)
> Took 0.0365 seconds{noformat}
> - Wait for some time (at least 5 mins) and try to add data to the table
> {noformat}
> > put 'TestTable','r1','info0:0','v1'
> ERROR: org.apache.hadoop.hbase.quotas.SpaceLimitingException: NO_INSERTS Puts
> are disallowed due to a space quota.
> at
> org.apache.hadoop.hbase.quotas.policies.NoInsertsViolationPolicyEnforcement.check(NoInsertsViolationPolicyEnforcement.java:47){noformat}
> To resolve the issue, {{RSes}} need to be restarted which points to in memory
> data not getting reset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20588) Space quota change after quota violation doesn't seem to take in effect
[
https://issues.apache.org/jira/browse/HBASE-20588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20588:
---
Fix Version/s: 2.1.0
> Space quota change after quota violation doesn't seem to take in effect
> ---
>
> Key: HBASE-20588
> URL: https://issues.apache.org/jira/browse/HBASE-20588
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20588.master.001.patch,
> HBASE-20588.master.002.patch, HBASE-20588.master.003.patch,
> HBASE-20588.master.004.patch, HBASE-20588.master.005.patch
>
>
> Steps followed
> - Through {{hbase shell}}
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => '2M', POLICY =>
> NO_INSERTS{noformat}
> - Run {{PE}} until the quota is reached
> {noformat}
> hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred
> --rows=2000 sequentialWrite 1{noformat}
> - Through {{HBase}} shell
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => NONE{noformat}
> - Through {{HBase}} shell verify the effective Quotas
> {noformat}
> > list_quotas
> OWNER QUOTAS
>
>
> 0 row(s)
> Took 0.0365 seconds{noformat}
> - Wait for some time (at least 5 mins) and try to add data to the table
> {noformat}
> > put 'TestTable','r1','info0:0','v1'
> ERROR: org.apache.hadoop.hbase.quotas.SpaceLimitingException: NO_INSERTS Puts
> are disallowed due to a space quota.
> at
> org.apache.hadoop.hbase.quotas.policies.NoInsertsViolationPolicyEnforcement.check(NoInsertsViolationPolicyEnforcement.java:47){noformat}
> To resolve the issue, {{RSes}} need to be restarted which points to in memory
> data not getting reset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20331) clean up shaded packaging for 2.1
[
https://issues.apache.org/jira/browse/HBASE-20331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489919#comment-16489919
]
Hudson commented on HBASE-20331:
Results for branch HBASE-20331
[build #19 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20331/19/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20331/19//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20331/19//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20331/19//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20331/19//artifacts/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> clean up shaded packaging for 2.1
> -
>
> Key: HBASE-20331
> URL: https://issues.apache.org/jira/browse/HBASE-20331
> Project: HBase
> Issue Type: Umbrella
> Components: Client, mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
>
> polishing pass on shaded modules for 2.0 based on trying to use them in more
> contexts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20616) TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
[
https://issues.apache.org/jira/browse/HBASE-20616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20616:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Toshihiro
> TruncateTableProcedure is stuck in retry loop in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state
> --
>
> Key: HBASE-20616
> URL: https://issues.apache.org/jira/browse/HBASE-20616
> Project: HBase
> Issue Type: Bug
> Components: amv2
> Environment: HDP-2.5.3
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 20616.master.004.patch, HBASE-20616.master.001.patch,
> HBASE-20616.master.002.patch, HBASE-20616.master.003.patch,
> HBASE-20616.master.004.patch
>
>
> At first, TruncateTableProcedure failed to write some files to HDFS in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state for some reason.
> {code:java}
> 2018-05-15 08:00:25,346 WARN [ProcedureExecutorThread-8]
> procedure.TruncateTableProcedure: Retriable error trying to truncate
> table=: state=TRUNCATE_TABLE_CREATE_FS_LAYOUT
> java.io.IOException: java.util.concurrent.ExecutionException:
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /apps/hbase/data/.tmp/data.regioninfo could
> only be replicated to 0 nodes instead of minReplication (=1). There are number of DNs> datanode(s) running and no node(s) are excluded in this
> operation.
> ...
> {code}
> But at this time, seemed like writing some files to HDFS was successful.
> And then, TruncateTableProcedure was stuck in retry loop in
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state. At this point, the following log
> messages were shown repeatedly in the master log:
> {code:java}
> 2018-05-15 08:00:25,463 WARN [ProcedureExecutorThread-8]
> procedure.TruncateTableProcedure: Retriable error trying to truncate
> table=: state=TRUNCATE_TABLE_CREATE_FS_LAYOUT
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs:///apps/hbase/data/.tmp/data///
> ...
> {code}
> It seems like this is because TruncateTableProcedure tried to write the files
> that were written successfully in the first try.
> I think we need to delete all the files and directories that are written
> successfully in the previous try before retrying the
> TRUNCATE_TABLE_CREATE_FS_LAYOUT state.
> Actually, this issue was observed in HDP-2.5.3, but I think the upstream has
> the same issue. Also, it looks to me that CreateTableProcedure has a similar
> issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20588) Space quota change after quota violation doesn't seem to take in effect
[
https://issues.apache.org/jira/browse/HBASE-20588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489905#comment-16489905
]
Josh Elser commented on HBASE-20588:
No problem, [~stack]. I got you. Thanks!
> Space quota change after quota violation doesn't seem to take in effect
> ---
>
> Key: HBASE-20588
> URL: https://issues.apache.org/jira/browse/HBASE-20588
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.0.1
>
> Attachments: HBASE-20588.master.001.patch,
> HBASE-20588.master.002.patch, HBASE-20588.master.003.patch,
> HBASE-20588.master.004.patch, HBASE-20588.master.005.patch
>
>
> Steps followed
> - Through {{hbase shell}}
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => '2M', POLICY =>
> NO_INSERTS{noformat}
> - Run {{PE}} until the quota is reached
> {noformat}
> hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred
> --rows=2000 sequentialWrite 1{noformat}
> - Through {{HBase}} shell
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => NONE{noformat}
> - Through {{HBase}} shell verify the effective Quotas
> {noformat}
> > list_quotas
> OWNER QUOTAS
>
>
> 0 row(s)
> Took 0.0365 seconds{noformat}
> - Wait for some time (at least 5 mins) and try to add data to the table
> {noformat}
> > put 'TestTable','r1','info0:0','v1'
> ERROR: org.apache.hadoop.hbase.quotas.SpaceLimitingException: NO_INSERTS Puts
> are disallowed due to a space quota.
> at
> org.apache.hadoop.hbase.quotas.policies.NoInsertsViolationPolicyEnforcement.check(NoInsertsViolationPolicyEnforcement.java:47){noformat}
> To resolve the issue, {{RSes}} need to be restarted which points to in memory
> data not getting reset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20638) nightly source artifact testing should fail the stage if it's going to report an error on jira
[
https://issues.apache.org/jira/browse/HBASE-20638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489900#comment-16489900
]
Hudson commented on HBASE-20638:
Results for branch branch-2
[build #778 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> nightly source artifact testing should fail the stage if it's going to report
> an error on jira
> --
>
> Key: HBASE-20638
> URL: https://issues.apache.org/jira/browse/HBASE-20638
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20638.0.patch
>
>
> Looks like the source artifact testing is properly reporting failures on
> jira, but is marking the stage on jenkins as successful. that makes it much
> hard to track over time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20588) Space quota change after quota violation doesn't seem to take in effect
[
https://issues.apache.org/jira/browse/HBASE-20588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489899#comment-16489899
]
Hudson commented on HBASE-20588:
Results for branch branch-2
[build #778 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Space quota change after quota violation doesn't seem to take in effect
> ---
>
> Key: HBASE-20588
> URL: https://issues.apache.org/jira/browse/HBASE-20588
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.0.1
>
> Attachments: HBASE-20588.master.001.patch,
> HBASE-20588.master.002.patch, HBASE-20588.master.003.patch,
> HBASE-20588.master.004.patch, HBASE-20588.master.005.patch
>
>
> Steps followed
> - Through {{hbase shell}}
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => '2M', POLICY =>
> NO_INSERTS{noformat}
> - Run {{PE}} until the quota is reached
> {noformat}
> hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred
> --rows=2000 sequentialWrite 1{noformat}
> - Through {{HBase}} shell
> {noformat}
> set_quota TYPE => SPACE, TABLE => 'TestTable', LIMIT => NONE{noformat}
> - Through {{HBase}} shell verify the effective Quotas
> {noformat}
> > list_quotas
> OWNER QUOTAS
>
>
> 0 row(s)
> Took 0.0365 seconds{noformat}
> - Wait for some time (at least 5 mins) and try to add data to the table
> {noformat}
> > put 'TestTable','r1','info0:0','v1'
> ERROR: org.apache.hadoop.hbase.quotas.SpaceLimitingException: NO_INSERTS Puts
> are disallowed due to a space quota.
> at
> org.apache.hadoop.hbase.quotas.policies.NoInsertsViolationPolicyEnforcement.check(NoInsertsViolationPolicyEnforcement.java:47){noformat}
> To resolve the issue, {{RSes}} need to be restarted which points to in memory
> data not getting reset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20390) IMC Default Parameters for 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489898#comment-16489898
]
Hudson commented on HBASE-20390:
Results for branch branch-2
[build #778 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/778//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> IMC Default Parameters for 2.0.0
>
>
> Key: HBASE-20390
> URL: https://issues.apache.org/jira/browse/HBASE-20390
> Project: HBase
> Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Attachments: HBASE-20390-branch-2.0-01.patch,
> HBASE-20390-branch-2.0-01.patch, HBASE-20390.branch-2.0.002.patch,
> HBASE-20390.branch-2.0.003.patch, HBase 2.0 performance evaluation -
> throughput SSD_HDD.pdf, hits.ihc.png
>
>
> Setting new default parameters for in-memory compaction based on performance
> tests done in HBASE-20188
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489893#comment-16489893 ] Andrew Purtell commented on HBASE-20172: +1 for branch-1, branch-1.2, branch-1.3, and branch-1.4. > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[
https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489891#comment-16489891
]
Hadoop QA commented on HBASE-20642:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
20s{color} | {color:blue} hbase-server in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 49s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} hbase-server: The patch generated 0 new + 148
unchanged - 14 fixed = 148 total (was 162) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 11s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}129m
10s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}177m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20642 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924993/HBASE-20642.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c70bd2282c4a 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1792f541c6 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/12952/artifact/patchprocess/diff-compile-javac-hbase-server.txt
|
| Test
[jira] [Commented] (HBASE-20616) TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
[
https://issues.apache.org/jira/browse/HBASE-20616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489890#comment-16489890
]
Hadoop QA commented on HBASE-20616:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
46s{color} | {color:blue} hbase-server in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} hbase-server: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 4s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}124m
42s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20616 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925005/20616.master.004.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e48ef121c849 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1792f541c6 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12953/testReport/ |
| Max. process+thread count | 5303 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12953/console |
| Powered by | Apache Yetus 0.7.0
[jira] [Commented] (HBASE-20597) Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
[
https://issues.apache.org/jira/browse/HBASE-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489845#comment-16489845
]
Hudson commented on HBASE-20597:
Results for branch branch-1.2
[build #343 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Use a lock to serialize access to a shared reference to ZooKeeperWatcher in
> HBaseReplicationEndpoint
>
>
> Key: HBASE-20597
> URL: https://issues.apache.org/jira/browse/HBASE-20597
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2, 1.4.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20597-branch-1.patch, HBASE-20597.patch
>
>
> The code that closes down a ZKW that fails to initialize when attempting to
> connect to the remote cluster is not MT safe and can in theory leak
> ZooKeeperWatcher instances. The allocation of a new ZKW and store to the
> reference is not atomic. Might have concurrent allocations with only one
> winning store, leading to leaked ZKW instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489838#comment-16489838 ] Ankit Singhal edited comment on HBASE-20172 at 5/24/18 9:34 PM: ping [~stack], [~apurtell] , shouldn't we need this fix to avoid failures if we are still supporting JDK 1.7 for branch-1? One user also seeing something similar:- http://mail-archives.apache.org/mod_mbox/phoenix-user/201805.mbox/%3CCAF1+Vs_TFcZ23zg8nfrZ8TDfpk_4oYYJpu5bFNXybBEFYJ=a...@mail.gmail.com%3E was (Author: an...@apache.org): ping [~stack], [~apurtell] , shouldn't we need this fix to avoid failures if we are still supporting JDK 1.7 for branch-1? > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20638) nightly source artifact testing should fail the stage if it's going to report an error on jira
[
https://issues.apache.org/jira/browse/HBASE-20638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489846#comment-16489846
]
Hudson commented on HBASE-20638:
Results for branch branch-1.2
[build #343 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/343//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> nightly source artifact testing should fail the stage if it's going to report
> an error on jira
> --
>
> Key: HBASE-20638
> URL: https://issues.apache.org/jira/browse/HBASE-20638
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20638.0.patch
>
>
> Looks like the source artifact testing is properly reporting failures on
> jira, but is marking the stage on jenkins as successful. that makes it much
> hard to track over time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489838#comment-16489838 ] Ankit Singhal commented on HBASE-20172: --- ping [~stack], [~apurtell] , shouldn't we need this fix to avoid failures if we are still supporting JDK 1.7 for branch-1? > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20636) Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
[
https://issues.apache.org/jira/browse/HBASE-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489834#comment-16489834
]
Hadoop QA commented on HBASE-20636:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
13s{color} | {color:blue} hbase-server in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} hbase-server: The patch generated 0 new + 224
unchanged - 9 fixed = 224 total (was 233) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 11s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
26s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}184m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
13s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
2s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}268m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.hfile.TestSeekBeforeWithInlineBlocks |
| | hadoop.hbase.regionserver.TestSeekOptimizations |
| | hadoop.hbase.regionserver.TestMultiColumnScanner |
\\
\\
|| Subsystem ||
[jira] [Commented] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[ https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489828#comment-16489828 ] Josh Elser commented on HBASE-20605: ~30 failures due to forked test JVMs crashing, can't imagine they're related but... > Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission > check > - > > Key: HBASE-20605 > URL: https://issues.apache.org/jira/browse/HBASE-20605 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.5.0, 1.3.3, 1.4.5 > > Attachments: HBASE-20605.001.branch-1.patch, > HBASE-20605.002.branch-1.patch > > > Some folks in Hadoop are working on landing a new FileSystem from the Azure > team: HADOOP-15407 > At present, this FileSystem doesn't support permissions which causes the > SecureBulkLoadEndpoint to balk because it the staging directory doesn't have > the proper 711 permissions. > We have a static list of FileSystem schemes which we ignore this check on. I > have a patch on an HBase 1.1ish which: > # Adds the new FileSystem scheme > # Makes this list configurable for the future -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
[
https://issues.apache.org/jira/browse/HBASE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20644:
---
Attachment:
101383-regionserver-ctr-e138-1518143905142-329221-01-02.hwx.site.log
> Master shutdown due to service ClusterSchemaServiceImpl failing to start
>
>
> Key: HBASE-20644
> URL: https://issues.apache.org/jira/browse/HBASE-20644
> Project: HBase
> Issue Type: Bug
>Reporter: Romil Choksi
>Priority: Major
> Attachments:
> 101383-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-02.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
>
>
> From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
> {code}
> 2018-05-23 22:14:29,750 ERROR
> [master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
> to become active master
> java.lang.IllegalStateException: Expected the service
> ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
> {code}
> Earlier in the log , the namespace region, 01a7f9ba9fffd691f261d3fbc620da06 ,
> was deemed OPEN on 01-07.hwx.site,16020,1527112194788 which was declared
> not online:
> {code}
> 2018-05-23 21:54:34,786 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.RegionStateStore: Load hbase:meta entry
> region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
> lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
>
> regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
> seqnum=43
> 2018-05-23 21:54:34,787 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: Number of RegionServers=1
> 2018-05-23 21:54:34,788 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: KILL
> RegionServer=ctr-e138-1518143905142-329221-01-07.
> hwx.site,16020,1527112194788 hosting regions but not online.
> {code}
> Later, even though a different instance on 007 registered with master:
> {code}
> 2018-05-23 21:55:13,541 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.ServerManager: Registering
> regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
> ...
> 2018-05-23 21:55:43,881 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
> started=69001 ms ago,cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not
> online on ctr-e138-1518143905142-329221-
> 01-07.hwx.site,16020,1527112506002
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2446)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> {code}
> There was no OPEN request for 01a7f9ba9fffd691f261d3fbc620da06 sent to that
> server instance.
> From
> hbase-hbase-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
> :
> {code}
> 2018-05-23 21:52:27,414 INFO
> [RS_CLOSE_REGION-regionserver/ctr-e138-1518143905142-329221-01-07:16020-1]
> regionserver.HRegion: Closed hbase:namespace,,1527099443383.
> 01a7f9ba9fffd691f261d3fbc620da06.
> {code}
> Then region server 007 restarted:
> {code}
> Wed May 23 21:55:03 UTC 2018 Starting regionserver on
> ctr-e138-1518143905142-329221-01-07.hwx.site
> {code}
> After which the region 01a7f9ba9fffd691f261d3fbc620da06 never showed up again
> in log 007
--
This
[jira] [Commented] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
[
https://issues.apache.org/jira/browse/HBASE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489817#comment-16489817
]
Ted Yu commented on HBASE-20644:
There were several occurrences in master log in the following form:
{code}
2018-05-23 21:59:39,883 WARN [ProcExecTimeout] assignment.AssignmentManager:
STUCK Region-In-Transition rit=OPENING,
location=ctr-e138-1518143905142-329221-01-02.hwx.site,16020,
1527112463065, table=hbase:namespace, region=01a7f9ba9fffd691f261d3fbc620da06
{code}
Looking at the code in AM:
{code}
private void handleRegionOverStuckWarningThreshold(final RegionInfo
regionInfo) {
final RegionStateNode regionNode =
regionStates.getRegionStateNode(regionInfo);
//if (regionNode.isStuck()) {
LOG.warn("STUCK Region-In-Transition {}", regionNode);
{code}
It seems one potential way of unstuck the region is to send OPEN request to the
region server (with newer start code).
> Master shutdown due to service ClusterSchemaServiceImpl failing to start
>
>
> Key: HBASE-20644
> URL: https://issues.apache.org/jira/browse/HBASE-20644
> Project: HBase
> Issue Type: Bug
>Reporter: Romil Choksi
>Priority: Major
> Attachments:
> 101383-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
>
>
> From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
> {code}
> 2018-05-23 22:14:29,750 ERROR
> [master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
> to become active master
> java.lang.IllegalStateException: Expected the service
> ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
> {code}
> Earlier in the log , the namespace region, 01a7f9ba9fffd691f261d3fbc620da06 ,
> was deemed OPEN on 01-07.hwx.site,16020,1527112194788 which was declared
> not online:
> {code}
> 2018-05-23 21:54:34,786 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.RegionStateStore: Load hbase:meta entry
> region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
> lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
>
> regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
> seqnum=43
> 2018-05-23 21:54:34,787 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: Number of RegionServers=1
> 2018-05-23 21:54:34,788 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: KILL
> RegionServer=ctr-e138-1518143905142-329221-01-07.
> hwx.site,16020,1527112194788 hosting regions but not online.
> {code}
> Later, even though a different instance on 007 registered with master:
> {code}
> 2018-05-23 21:55:13,541 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.ServerManager: Registering
> regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
> ...
> 2018-05-23 21:55:43,881 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
> started=69001 ms ago,cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not
> online on ctr-e138-1518143905142-329221-
> 01-07.hwx.site,16020,1527112506002
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2446)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> {code}
> There was no OPEN request for
[jira] [Updated] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
[
https://issues.apache.org/jira/browse/HBASE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20644:
---
Attachment:
101383-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
101383-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log
> Master shutdown due to service ClusterSchemaServiceImpl failing to start
>
>
> Key: HBASE-20644
> URL: https://issues.apache.org/jira/browse/HBASE-20644
> Project: HBase
> Issue Type: Bug
>Reporter: Romil Choksi
>Priority: Major
> Attachments:
> 101383-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log,
> 101383-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
>
>
> From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
> {code}
> 2018-05-23 22:14:29,750 ERROR
> [master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
> to become active master
> java.lang.IllegalStateException: Expected the service
> ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
> at
> org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
> at
> org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
> {code}
> Earlier in the log , the namespace region, 01a7f9ba9fffd691f261d3fbc620da06 ,
> was deemed OPEN on 01-07.hwx.site,16020,1527112194788 which was declared
> not online:
> {code}
> 2018-05-23 21:54:34,786 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.RegionStateStore: Load hbase:meta entry
> region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
> lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
>
> regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
> seqnum=43
> 2018-05-23 21:54:34,787 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: Number of RegionServers=1
> 2018-05-23 21:54:34,788 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> assignment.AssignmentManager: KILL
> RegionServer=ctr-e138-1518143905142-329221-01-07.
> hwx.site,16020,1527112194788 hosting regions but not online.
> {code}
> Later, even though a different instance on 007 registered with master:
> {code}
> 2018-05-23 21:55:13,541 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
> master.ServerManager: Registering
> regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
> ...
> 2018-05-23 21:55:43,881 INFO
> [master/ctr-e138-1518143905142-329221-01-03:2]
> client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
> started=69001 ms ago,cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not
> online on ctr-e138-1518143905142-329221-
> 01-07.hwx.site,16020,1527112506002
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2446)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> {code}
> There was no OPEN request for 01a7f9ba9fffd691f261d3fbc620da06 sent to that
> server instance.
> From
> hbase-hbase-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log
> :
> {code}
> 2018-05-23 21:52:27,414 INFO
> [RS_CLOSE_REGION-regionserver/ctr-e138-1518143905142-329221-01-07:16020-1]
> regionserver.HRegion: Closed hbase:namespace,,1527099443383.
> 01a7f9ba9fffd691f261d3fbc620da06.
> {code}
> Then region server 007 restarted:
> {code}
> Wed May 23 21:55:03 UTC 2018 Starting regionserver on
> ctr-e138-1518143905142-329221-01-07.hwx.site
> {code}
> After which the region 01a7f9ba9fffd691f261d3fbc620da06 never showed up again
> in log 007
--
This
[jira] [Updated] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
[
https://issues.apache.org/jira/browse/HBASE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20644:
---
Description:
>From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
{code}
2018-05-23 22:14:29,750 ERROR
[master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
to become active master
java.lang.IllegalStateException: Expected the service ClusterSchemaServiceImpl
[FAILED] to be RUNNING, but the service has FAILED
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
at
org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
{code}
Earlier in the log , the namespace region, 01a7f9ba9fffd691f261d3fbc620da06 ,
was deemed OPEN on 01-07.hwx.site,16020,1527112194788 which was declared
not online:
{code}
2018-05-23 21:54:34,786 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.RegionStateStore: Load hbase:meta entry
region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
seqnum=43
2018-05-23 21:54:34,787 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.AssignmentManager: Number of RegionServers=1
2018-05-23 21:54:34,788 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.AssignmentManager: KILL
RegionServer=ctr-e138-1518143905142-329221-01-07.
hwx.site,16020,1527112194788 hosting regions but not online.
{code}
Later, even though a different instance on 007 registered with master:
{code}
2018-05-23 21:55:13,541 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
master.ServerManager: Registering
regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
...
2018-05-23 21:55:43,881 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
started=69001 ms ago,cancelled=false,
msg=org.apache.hadoop.hbase.NotServingRegionException:
hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not online
on ctr-e138-1518143905142-329221- 01-07.hwx.site,16020,1527112506002
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2446)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
{code}
There was no OPEN request for 01a7f9ba9fffd691f261d3fbc620da06 sent to that
server instance.
>From
>hbase-hbase-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log :
{code}
2018-05-23 21:52:27,414 INFO
[RS_CLOSE_REGION-regionserver/ctr-e138-1518143905142-329221-01-07:16020-1]
regionserver.HRegion: Closed hbase:namespace,,1527099443383.
01a7f9ba9fffd691f261d3fbc620da06.
{code}
Then region server 007 restarted:
{code}
Wed May 23 21:55:03 UTC 2018 Starting regionserver on
ctr-e138-1518143905142-329221-01-07.hwx.site
{code}
After which the region 01a7f9ba9fffd691f261d3fbc620da06 never showed up again
in log 007
was:
>From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
{code}
2018-05-23 22:14:29,750 ERROR
[master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
to become active master
java.lang.IllegalStateException: Expected the service ClusterSchemaServiceImpl
[FAILED] to be RUNNING, but the service has FAILED
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
at
org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
at
[jira] [Created] (HBASE-20644) Master shutdown due to service ClusterSchemaServiceImpl failing to start
Ted Yu created HBASE-20644:
--
Summary: Master shutdown due to service ClusterSchemaServiceImpl
failing to start
Key: HBASE-20644
URL: https://issues.apache.org/jira/browse/HBASE-20644
Project: HBase
Issue Type: Bug
Reporter: Romil Choksi
>From hbase-hbase-master-ctr-e138-1518143905142-329221-01-03.hwx.site.log :
{code}
2018-05-23 22:14:29,750 ERROR
[master/ctr-e138-1518143905142-329221-01-03:2] master.HMaster: Failed
to become active master
java.lang.IllegalStateException: Expected the service ClusterSchemaServiceImpl
[FAILED] to be RUNNING, but the service has FAILED
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
at
org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1054)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:918)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2023)
{code}
Earlier in the log , the namespace region was deemed OPEN on
01-07.hwx.site,16020,1527112194788 which was declared not online:
{code}
2018-05-23 21:54:34,786 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.RegionStateStore: Load hbase:meta entry
region=01a7f9ba9fffd691f261d3fbc620da06, regionState=OPEN,
lastHost=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
regionLocation=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112194788,
seqnum=43
2018-05-23 21:54:34,787 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.AssignmentManager: Number of RegionServers=1
2018-05-23 21:54:34,788 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
assignment.AssignmentManager: KILL
RegionServer=ctr-e138-1518143905142-329221-01-07.
hwx.site,16020,1527112194788 hosting regions but not online.
{code}
Later, even though a different instance on 007 registered with master:
{code}
2018-05-23 21:55:13,541 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2]
master.ServerManager: Registering
regionserver=ctr-e138-1518143905142-329221-01-07.hwx.site,16020,1527112506002
...
2018-05-23 21:55:43,881 INFO
[master/ctr-e138-1518143905142-329221-01-03:2]
client.RpcRetryingCallerImpl: Call exception, tries=12, retries=12,
started=69001 ms ago,cancelled=false,
msg=org.apache.hadoop.hbase.NotServingRegionException:
hbase:namespace,,1527099443383.01a7f9ba9fffd691f261d3fbc620da06. is not online
on ctr-e138-1518143905142-329221- 01-07.hwx.site,16020,1527112506002
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3273)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3250)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1414)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2446)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
{code}
There was no OPEN request sent to that instance.
>From
>hbase-hbase-regionserver-ctr-e138-1518143905142-329221-01-07.hwx.site.log :
{code}
2018-05-23 21:52:27,414 INFO
[RS_CLOSE_REGION-regionserver/ctr-e138-1518143905142-329221-01-07:16020-1]
regionserver.HRegion: Closed hbase:namespace,,1527099443383.
01a7f9ba9fffd691f261d3fbc620da06.
{code}
Then region server 007 restarted:
{code}
Wed May 23 21:55:03 UTC 2018 Starting regionserver on
ctr-e138-1518143905142-329221-01-07.hwx.site
{code}
After which the region 01a7f9ba9fffd691f261d3fbc620da06 never showed up again
in log 007
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20595) Remove the concept of 'special tables' from rsgroups
[
https://issues.apache.org/jira/browse/HBASE-20595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489737#comment-16489737
]
Hudson commented on HBASE-20595:
Results for branch branch-1.4
[build #333 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Remove the concept of 'special tables' from rsgroups
>
>
> Key: HBASE-20595
> URL: https://issues.apache.org/jira/browse/HBASE-20595
> Project: HBase
> Issue Type: Task
> Components: Region Assignment, rsgroup
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 2.0.1, 1.4.5
>
> Attachments: HBASE-20595-branch-1.patch, HBASE-20595.patch,
> HBASE-20595.patch
>
>
> Regionserver groups needs to specially handle what it calls "special tables",
> tables upon which core or other modular functionality depends. They need to
> be excluded from normal rsgroup processing during bootstrap to avoid circular
> dependencies or errors due to insufficiently initialized state. I think we
> also want to ensure that such tables are always given a rsgroup assignment
> with nonzero servers. (IIRC another issue already raises that point, we can
> link it later.)
> Special tables include:
> * The system tables in the 'hbase:' namespace
> * The ACL table if the AccessController coprocessor is installed
> * The Labels table if the VisibilityController coprocessor is installed
> * The Quotas table if the FS quotas feature is active
> Either we need a facility where "special tables" can be registered, which
> should be in core. Or, we institute a blanket rule that core and all
> extensions that need a "special table" must put them into the 'hbase:'
> namespace, so the TableName#isSystemTable() test will return TRUE for all,
> and then rsgroups simply needs to test for that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20597) Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
[
https://issues.apache.org/jira/browse/HBASE-20597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489738#comment-16489738
]
Hudson commented on HBASE-20597:
Results for branch branch-1.4
[build #333 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Use a lock to serialize access to a shared reference to ZooKeeperWatcher in
> HBaseReplicationEndpoint
>
>
> Key: HBASE-20597
> URL: https://issues.apache.org/jira/browse/HBASE-20597
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2, 1.4.4
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 2.0.1, 1.4.5
>
> Attachments: HBASE-20597-branch-1.patch, HBASE-20597.patch
>
>
> The code that closes down a ZKW that fails to initialize when attempting to
> connect to the remote cluster is not MT safe and can in theory leak
> ZooKeeperWatcher instances. The allocation of a new ZKW and store to the
> reference is not atomic. Might have concurrent allocations with only one
> winning store, leading to leaked ZKW instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20627) Relocate RS Group pre/post hooks from RSGroupAdminServer to RSGroupAdminEndpoint
[
https://issues.apache.org/jira/browse/HBASE-20627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489739#comment-16489739
]
Hudson commented on HBASE-20627:
Results for branch branch-1.4
[build #333 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/333//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Relocate RS Group pre/post hooks from RSGroupAdminServer to
> RSGroupAdminEndpoint
>
>
> Key: HBASE-20627
> URL: https://issues.apache.org/jira/browse/HBASE-20627
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.1.0, 1.4.5
>
> Attachments: 20627.branch-1.txt, 20627.v1.txt, 20627.v2.txt,
> 20627.v3.txt
>
>
> Currently RS Group pre/post hooks are called from RSGroupAdminServer.
> e.g. RSGroupAdminServer#removeRSGroup :
> {code}
> if (master.getMasterCoprocessorHost() != null) {
> master.getMasterCoprocessorHost().preRemoveRSGroup(name);
> }
> {code}
> RSGroupAdminServer#removeRSGroup is called by RSGroupAdminEndpoint :
> {code}
> checkPermission("removeRSGroup");
> groupAdminServer.removeRSGroup(request.getRSGroupName());
> {code}
> If permission check fails, the pre hook wouldn't be called.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20594) provide utility to compare old and new descriptors
[
https://issues.apache.org/jira/browse/HBASE-20594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489651#comment-16489651
]
Sean Busbey edited comment on HBASE-20594 at 5/24/18 7:34 PM:
--
{code}
// Need to check by element because array equality doesn't work
private void assertSetsEqual(Set s1, Set s2) {
assertEquals(s1.size(), s2.size());
for (T t : s1) {
assertThat(s2, CoreMatchers.hasItem(t));
}
}
{code}
This is just the contract for {{Set.equals}}. It looks like Guava's
{{ImmutableSet}} doesn't honor the contract, which is why we end up doing this
here instead of just {{assertEquals(Set, Set)}}. If we switch to
{{Collections.unmodifiableSet}} then we can also nix this bit and remove the
hamcrest dependency.
was (Author: busbey):
{quote}
// Need to check by element because array equality doesn't work
private void assertSetsEqual(Set s1, Set s2) {
assertEquals(s1.size(), s2.size());
for (T t : s1) {
assertThat(s2, CoreMatchers.hasItem(t));
}
}
{quote}
This is just the contract for {{Set.equals}}. It looks like Guava's
{{ImmutableSet}} doesn't honor the contract, which is why we end up doing this
here instead of just {{assertEquals(Set, Set)}}. If we switch to
{{Collections.unmodifiableSet}} then we can also nix this bit and remove the
hamcrest dependency.
> provide utility to compare old and new descriptors
> --
>
> Key: HBASE-20594
> URL: https://issues.apache.org/jira/browse/HBASE-20594
> Project: HBase
> Issue Type: Improvement
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20594.patch, HBASE-20594.v2.patch,
> HBASE-20594.v3.patch, HBASE-20594.v4.patch, HBASE-20594.v5.patch,
> HBASE-20594.v6.patch
>
>
> HBASE-20567 gives us hooks that give both the old and new descriptor in
> pre/postModify* events, but comparing them is still cumbersome. We should
> provide users some kind of utility for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20594) provide utility to compare old and new descriptors
[
https://issues.apache.org/jira/browse/HBASE-20594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489651#comment-16489651
]
Sean Busbey commented on HBASE-20594:
-
{quote}
// Need to check by element because array equality doesn't work
private void assertSetsEqual(Set s1, Set s2) {
assertEquals(s1.size(), s2.size());
for (T t : s1) {
assertThat(s2, CoreMatchers.hasItem(t));
}
}
{quote}
This is just the contract for {{Set.equals}}. It looks like Guava's
{{ImmutableSet}} doesn't honor the contract, which is why we end up doing this
here instead of just {{assertEquals(Set, Set)}}. If we switch to
{{Collections.unmodifiableSet}} then we can also nix this bit and remove the
hamcrest dependency.
> provide utility to compare old and new descriptors
> --
>
> Key: HBASE-20594
> URL: https://issues.apache.org/jira/browse/HBASE-20594
> Project: HBase
> Issue Type: Improvement
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Attachments: HBASE-20594.patch, HBASE-20594.v2.patch,
> HBASE-20594.v3.patch, HBASE-20594.v4.patch, HBASE-20594.v5.patch,
> HBASE-20594.v6.patch
>
>
> HBASE-20567 gives us hooks that give both the old and new descriptor in
> pre/postModify* events, but comparing them is still cumbersome. We should
> provide users some kind of utility for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
