[jira] [Commented] (HBASE-20009) Add license header to krb5.conf
[ https://issues.apache.org/jira/browse/HBASE-20009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368847#comment-16368847 ] Peter Somogyi commented on HBASE-20009: --- nit: Indentation is different. Starts with 2 spaces and last line + comment have 1 space. > Add license header to krb5.conf > --- > > Key: HBASE-20009 > URL: https://issues.apache.org/jira/browse/HBASE-20009 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Minor > Attachments: 20009.v1.txt > > > hbase-native-client/docker-files/krb5.conf misses Apache license -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19950) Introduce a ColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368819#comment-16368819
]
Anoop Sam John commented on HBASE-19950:
Only on those specific cases, this filter to be used. For other normal cases,
better to do addColumn () stuff to Get/Scan.. Ya pls add enough comments and
documentation.
> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19950.master.001.patch,
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch,
> HBASE-19950.master.004.patch, HBASE-19950.master.005.patch,
> HBASE-19950.master.006.patch, HBASE-19950.master.007.patch,
> HBASE-19950.master.008.patch, HBASE-19950.master.009.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when
> specified condition is matched, this new filter will return the matched cell
> only. There're already some discussions in HBASE-19824.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19401) Add missing security checks in RSRpcServices
[
https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368817#comment-16368817
]
Hadoop QA commented on HBASE-19401:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} hbase-server: The patch generated 0 new + 308
unchanged - 7 fixed = 308 total (was 315) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 26s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m
55s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
25s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19401 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911088/HBASE-19401.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7f502777995d 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f3bb9b9613 |
| maven | version:
[jira] [Commented] (HBASE-19401) Add missing security checks in RSRpcServices
[
https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368810#comment-16368810
]
Hadoop QA commented on HBASE-19401:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} hbase-server: The patch generated 0 new + 308
unchanged - 7 fixed = 308 total (was 315) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 40s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m
57s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
57s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19401 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911086/HBASE-19401.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e8c2b87b69c3 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f3bb9b9613 |
| maven | version:
[jira] [Commented] (HBASE-19400) Add missing security checks in MasterRpcServices
[
https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368809#comment-16368809
]
Hadoop QA commented on HBASE-19400:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} hbase-server: The patch generated 0 new + 309
unchanged - 6 fixed = 309 total (was 315) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}130m 12s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
19s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19400 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911085/HBASE-19400.master.007.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 756fe3f01539 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f3bb9b9613 |
| maven | version: Apache Maven
[jira] [Commented] (HBASE-19402) Add missing security check for RegionServerStatusService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368804#comment-16368804 ] Duo Zhang commented on HBASE-19402: --- For our secure cluster we start master and RS with the same user. And I think this is the common case? If you use secure HBase then usually the HDFS is also secured, and usually the permission would be 700, if you use different users for master and RS then there will be problem accessing HDFS... Thanks, > Add missing security check for RegionServerStatusService RPCs > - > > Key: HBASE-19402 > URL: https://issues.apache.org/jira/browse/HBASE-19402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Priority: Major > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - regionServerStartup > - regionServerReport > - reportRSFatalError > - reportRegionStateTransition > - reportRegionSpaceUse -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19403) Add missing security checks for AdminService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368789#comment-16368789 ] Appy commented on HBASE-19403: -- Idk what's the best for replay() fn. Not sure if it should be just admin. Or create-global would also be sufficient? What will be it's effect on replication, if any? [~Apache9] any ideas? Making following require ADMIN - updateConfiguration - warmupRegion - updateFavoredNodes clearRegionBlockCache > Add missing security checks for AdminService RPCs > - > > Key: HBASE-19403 > URL: https://issues.apache.org/jira/browse/HBASE-19403 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - updateConfiguration > - replay > - warmupRegion > - updateFavoredNodes > - clearRegionBlockCache -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19402) Add missing security check for RegionServerStatusService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19402: - Summary: Add missing security check for RegionServerStatusService RPCs (was: Add missing security hooks for RegionServerStatusService RPCs) > Add missing security check for RegionServerStatusService RPCs > - > > Key: HBASE-19402 > URL: https://issues.apache.org/jira/browse/HBASE-19402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Priority: Major > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - regionServerStartup > - regionServerReport > - reportRSFatalError > - reportRegionStateTransition > - reportRegionSpaceUse -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19403) Add missing security checks for AdminService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19403: - Summary: Add missing security checks for AdminService RPCs (was: Add missing security hooks for AdminService RPCs) > Add missing security checks for AdminService RPCs > - > > Key: HBASE-19403 > URL: https://issues.apache.org/jira/browse/HBASE-19403 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - updateConfiguration > - replay > - warmupRegion > - updateFavoredNodes > - clearRegionBlockCache -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19402) Add missing security hooks for RegionServerStatusService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368786#comment-16368786 ] Appy commented on HBASE-19402: -- Should we make all of these require ADMIN? These are critical functions for a cluster to startup(?) so am trying to understand how will it work? If master starts as user 'hbase', does that user automatically gets certain privileges? If not, how will a cluster boot-up? No RSs will be able to call regionServerStartup(), which means ACL table can't be served, which means no permissions can be granted for to user starting RSs. chicken and egg problem. If yes, that puts a restriction that all hbase services have to be started by same user. I don't see a problem in that restriction, but then i have very limited ops experience. [~stack] [~Apache9] any ideas? > Add missing security hooks for RegionServerStatusService RPCs > - > > Key: HBASE-19402 > URL: https://issues.apache.org/jira/browse/HBASE-19402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Priority: Major > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - regionServerStartup > - regionServerReport > - reportRSFatalError > - reportRegionStateTransition > - reportRegionSpaceUse -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-19403) Add missing security hooks for AdminService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy reassigned HBASE-19403: Assignee: Appy > Add missing security hooks for AdminService RPCs > > > Key: HBASE-19403 > URL: https://issues.apache.org/jira/browse/HBASE-19403 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > > The following RPC methods do not call the observers, therefore they are not > guarded by AccessController: > - updateConfiguration > - replay > - warmupRegion > - updateFavoredNodes > - clearRegionBlockCache -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19401) Add missing security checks in RSRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19401: - Attachment: HBASE-19401.master.002.patch > Add missing security checks in RSRpcServices > > > Key: HBASE-19401 > URL: https://issues.apache.org/jira/browse/HBASE-19401 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Attachments: HBASE-19401.master.001.patch, > HBASE-19401.master.002.patch > > > The following RPC methods in RSRpcServices do not have ACL check for ADMIN > rights. > - execRegionServerService -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19401) Add missing security checks in RSRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19401: - Assignee: Appy Status: Patch Available (was: Open) > Add missing security checks in RSRpcServices > > > Key: HBASE-19401 > URL: https://issues.apache.org/jira/browse/HBASE-19401 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Attachments: HBASE-19401.master.001.patch > > > The following RPC methods in RSRpcServices do not have ACL check for ADMIN > rights. > - execRegionServerService -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19401) Add missing security checks in RSRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368753#comment-16368753 ] Appy commented on HBASE-19401: -- Current patch is built on top of HBASE-19400 which is WIP. When its committed i'll update a final rebased patch. Till then, using hanoi tower to keep making progress. > Add missing security checks in RSRpcServices > > > Key: HBASE-19401 > URL: https://issues.apache.org/jira/browse/HBASE-19401 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Priority: Major > Attachments: HBASE-19401.master.001.patch > > > The following RPC methods in RSRpcServices do not have ACL check for ADMIN > rights. > - execRegionServerService -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19401) Add missing security checks in RSRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19401: - Attachment: HBASE-19401.master.001.patch > Add missing security checks in RSRpcServices > > > Key: HBASE-19401 > URL: https://issues.apache.org/jira/browse/HBASE-19401 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Priority: Major > Attachments: HBASE-19401.master.001.patch > > > The following RPC methods in RSRpcServices do not have ACL check for ADMIN > rights. > - execRegionServerService -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19400) Add missing security checks in MasterRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19400: - Attachment: HBASE-19400.master.007.patch > Add missing security checks in MasterRpcServices > > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19400.master.001.patch, > HBASE-19400.master.002.patch, HBASE-19400.master.003.patch, > HBASE-19400.master.004.patch, HBASE-19400.master.004.patch, > HBASE-19400.master.005.patch, HBASE-19400.master.006.patch, > HBASE-19400.master.007.patch > > > The following RPC methods in MasterRpcServices do not have ACL check for > ADMIN rights. > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19400) Add missing security checks in MasterRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19400: - Summary: Add missing security checks in MasterRpcServices (was: Add missing security hooks for MasterService RPCs) > Add missing security checks in MasterRpcServices > > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19400.master.001.patch, > HBASE-19400.master.002.patch, HBASE-19400.master.003.patch, > HBASE-19400.master.004.patch, HBASE-19400.master.004.patch, > HBASE-19400.master.005.patch, HBASE-19400.master.006.patch > > > The following RPC methods in MasterRpcServices do not have ACL check for > ADMIN rights. > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19401) Add missing security checks in RSRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19401: - Summary: Add missing security checks in RSRpcServices (was: Add missing security hooks for ClientService RPCs) > Add missing security checks in RSRpcServices > > > Key: HBASE-19401 > URL: https://issues.apache.org/jira/browse/HBASE-19401 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Priority: Major > > The following RPC methods in RSRpcServices do not have ACL check for ADMIN > rights. > - execRegionServerService -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[
https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368752#comment-16368752
]
Hadoop QA commented on HBASE-19978:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
7s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} hbase-procedure: The patch generated 0 new + 20
unchanged - 9 fixed = 20 total (was 29) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}107m
58s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19978 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911084/HBASE-19978-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 538e7e60b501 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f3bb9b9613 |
| maven | version: Apache
[jira] [Updated] (HBASE-19400) Add missing security hooks for MasterService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19400: - Description: The following RPC methods in MasterRpcServices do not have ACL check for ADMIN rights. - normalize - setNormalizerRunning - runCatalogScan - enableCatalogJanitor - runCleanerChore - setCleanerChoreRunning - execMasterService - execProcedure - execProcedureWithRet was: The following RPC methods do not have ACL check for ADMIN rights. - normalize - setNormalizerRunning - runCatalogScan - enableCatalogJanitor - runCleanerChore - setCleanerChoreRunning - execMasterService - execProcedure - execProcedureWithRet > Add missing security hooks for MasterService RPCs > - > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19400.master.001.patch, > HBASE-19400.master.002.patch, HBASE-19400.master.003.patch, > HBASE-19400.master.004.patch, HBASE-19400.master.004.patch, > HBASE-19400.master.005.patch, HBASE-19400.master.006.patch > > > The following RPC methods in MasterRpcServices do not have ACL check for > ADMIN rights. > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19401) Add missing security hooks for ClientService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19401: - Description: The following RPC methods in RSRpcServices do not have ACL check for ADMIN rights. - execRegionServerService was: The following RPC method does not call the observers, therefore it is not guarded by AccessController: - execRegionServerService > Add missing security hooks for ClientService RPCs > - > > Key: HBASE-19401 > URL: https://issues.apache.org/jira/browse/HBASE-19401 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Priority: Major > > The following RPC methods in RSRpcServices do not have ACL check for ADMIN > rights. > - execRegionServerService -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19400) Add missing security hooks for MasterService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19400: - Release Note: Added ACL check to following Admin functions: enableCatalogJanitor, runCatalogJanitor, cleanerChoreSwitch, runCleanerChore, execProcedure, execProcedureWithReturn, normalize, normalizerSwitch, coprocessorService. When ACL is enabled, only those with ADMIN rights will be able to invoke these operations successfully. > Add missing security hooks for MasterService RPCs > - > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19400.master.001.patch, > HBASE-19400.master.002.patch, HBASE-19400.master.003.patch, > HBASE-19400.master.004.patch, HBASE-19400.master.004.patch, > HBASE-19400.master.005.patch, HBASE-19400.master.006.patch > > > The following RPC methods do not have ACL check for ADMIN rights. > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19400) Add missing security hooks for MasterService RPCs
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19400: - Description: The following RPC methods do not have ACL check for ADMIN rights. - normalize - setNormalizerRunning - runCatalogScan - enableCatalogJanitor - runCleanerChore - setCleanerChoreRunning - execMasterService - execProcedure - execProcedureWithRet was: The following RPC methods do not call the observers, therefore they are not guarded by AccessController: - normalize - setNormalizerRunning - runCatalogScan - enableCatalogJanitor - runCleanerChore - setCleanerChoreRunning - execMasterService - execProcedure - execProcedureWithRet > Add missing security hooks for MasterService RPCs > - > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19400.master.001.patch, > HBASE-19400.master.002.patch, HBASE-19400.master.003.patch, > HBASE-19400.master.004.patch, HBASE-19400.master.004.patch, > HBASE-19400.master.005.patch, HBASE-19400.master.006.patch > > > The following RPC methods do not have ACL check for ADMIN rights. > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20001) cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
[
https://issues.apache.org/jira/browse/HBASE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368726#comment-16368726
]
Chia-Ping Tsai commented on HBASE-20001:
Could we use MetaTableAccessor#etRegionLocation instead? The passed region name
is converted to HRegionInfo if we use MetaTableAccessor#getRegion. That is a
litter ugly I'd say.
> cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
> -
>
> Key: HBASE-20001
> URL: https://issues.apache.org/jira/browse/HBASE-20001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.1.7
>Reporter: Francis Liu
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.5.0, 1.2.7, 1.4.3
>
> Attachments: HBASE-20001.branch-1.4.001.patch
>
>
> In RegionStates.cleanIfNoMetaEntry()
> {{if (MetaTableAccessor.getRegion(server.getConnection(),
> hri.getEncodedNameAsBytes()) == null) {}}
> {{regionOffline(hri);}}
> {{FSUtils.deleteRegionDir(server.getConfiguration(), hri);}}
> }
> But api expects regionname
> {{public static Pair getRegion(Connection
> connection, byte [] regionName)}}
> So we might end up cleaning good regions.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19147) All branch-2 unit tests pass
[ https://issues.apache.org/jira/browse/HBASE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368725#comment-16368725 ] Duo Zhang commented on HBASE-19147: --- Do we need to go over the tests again to see the ignore ones? Thanks. > All branch-2 unit tests pass > > > Key: HBASE-19147 > URL: https://issues.apache.org/jira/browse/HBASE-19147 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Priority: Blocker > Fix For: 2.0.0-beta-2 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19978: -- Attachment: HBASE-19978-v2.patch > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978-v2.patch, > HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368723#comment-16368723 ] Duo Zhang commented on HBASE-19978: --- And for the core pool size, the default value is the max one of cpu/4 and 16. So user could set it to a really small one if there are not many cpu cores, but I think the users who try to set this value should be advanced users, they should know the bad effects if set it to a very small value... And the intention here is to make the system stabler without doing big changes since 2.0 is closer. I think we can open a new issue which targets to 3.0, and try a more beautiful solution there, coroutine or something. And will go out for another two days... Will upload the new test soon. [~stack] if you think it is OK then please help committing this. I think this is useful, especially the test. In production cluster we have hundreds of tables and an RS could also take hundreds of regions. We need the UT to confirm that we will not be stuck when failover. Thanks. > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368720#comment-16368720 ] Duo Zhang commented on HBASE-19978: --- In the old implementation, any workers can timeout, the only check is that if the number of workers is not greater than for size than we do not timeout. But this is not stable, all the threads can check at the same time and then all time our. The default timeout value is Long.MAX_VALUE so no problem for us now... And also, we only add a new thread in stuck checker only if all the workers are in use. I believe the reason is that, there is no upper limit for thread count, and as said above, we do not have timeout by default, so we should be careful when adding new thread. In the new implementation, I set a upper limit of the number of workers, and the keepalive time is default to 1 minute, so we will add new thread more aggressively since we will stop at the limit, and they will soon be back to normal. I think this could speed up our failover. And the core workers are never timeout, only new worker added by stuck checker can time out. Thanks. > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20015) TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey
[
https://issues.apache.org/jira/browse/HBASE-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368718#comment-16368718
]
stack commented on HBASE-20015:
---
Hmm.. These have dropped off the top list in the flakies dashboard but still on
the bottom set, as though the patch were not in place... Looking.
> TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey
> -
>
> Key: HBASE-20015
> URL: https://issues.apache.org/jira/browse/HBASE-20015
> Project: HBase
> Issue Type: Sub-task
> Components: flakey
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20015.branch-2.001.patch
>
>
> MergeRegionProcedure seems incomplete. The ProcedureExecutor framework can
> run in a test mode such that it kills the Procedure before it can persist
> state and it does this repeatedly to shake out areas where Procedures may not
> be preserving all needed state at each Procedural step. The kill will cause
> the Procedure to 'fail'. It'll then run the rollback procedure. The
> MergeRegionProcedure is not able to roll back the last few steps of Merge
> It throws an UnsupportedException (the hope was that the missing steps would
> be filled in ... but they are hard to complete in that they themselves are
> stepped).
> So
> Well it turns out that Split has a mechanism where it will not fail the
> Procedure if gets to a stage from which it cannot rollback. Instead, it will
> just retry and keep retrying till it succeeds eventually. Merge has this
> facility half-implemented. Merge tests are therefore flakey. They do stuff
> like this:
> {code}
> 2018-02-17 04:04:02,999 WARN [PEWorker-1]
> assignment.MergeTableRegionsProcedure(311): Failed rollback attempt step
> MERGE_TABLE_REGIONS_UPDATE_META for merging the regions
> [485dd0c2a5d14601d61fed791f793158, 8af34a614f064c162ab1d05eac7fca4c] in table
> testRollbackAndDoubleExecution
> java.lang.UnsupportedOperationException: pid=44,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
> exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
> MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
> abort requested; MergeTableRegionsProcedure
> table=testRollbackAndDoubleExecution,
> regions=[485dd0c2a5d14601d61fed791f793158, 8af34a614f064c162ab1d05eac7fca4c],
> forcibly=false unhandled state=MERGE_TABLE_REGIONS_UPDATE_META
> at
> org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:291)
> at
> org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:78)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:199)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:859)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1356)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1312)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1181)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
> 2018-02-17 04:04:03,007 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=44,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
> exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
> MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
> abort requested; MergeTableRegionsProcedure
> table=testRollbackAndDoubleExecution,
> regions=[485dd0c2a5d14601d61fed791f793158, 8af34a614f064c162ab1d05eac7fca4c],
> forcibly=false
> java.lang.UnsupportedOperationException: pid=44,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
> exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
> MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
> abort requested; MergeTableRegionsProcedure
> table=testRollbackAndDoubleExecution,
> regions=[485dd0c2a5d14601d61fed791f793158, 8af34a614f064c162ab1d05eac7fca4c],
> forcibly=false unhandled state=MERGE_TABLE_REGIONS_UPDATE_META
> at
> org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:291)
> at
>
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368719#comment-16368719 ] Duo Zhang commented on HBASE-19978: --- Oh, we could set the keep alive time to a small value and add an asset at the end of the test to confirm that the number of workers is back to normal. > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[
https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368714#comment-16368714
]
Chia-Ping Tsai commented on HBASE-20017:
{quote}what's the affected version(s) here? is this only a problem starting on
branch-1.4?
{quote}
yep, the affected versions includes 1.4.0, 1.4.1, 1.4.2(rc).
> BufferedMutatorImpl submit the same mutation repeatedly
> ---
>
> Key: HBASE-20017
> URL: https://issues.apache.org/jira/browse/HBASE-20017
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.4.0, 1.4.1, 1.4.2
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 2.0.0, 1.5.0, 1.4.3
>
> Attachments: HBASE-20017.branch-1.4.v0.patch.patch,
> HBASE-20017.branch-1.v0.patch.patch, HBASE-20017.v0.patch.patch
>
>
> BMI pass a iter of inner buffer to AsyncProcess to take the undealt
> mutations, AsyncProcess call iter#next to get the mutation and then call
> iter#remove to delete the mutation from inner buffer. Hence, There's a good
> chance that the mutation is processed repeatedly in case there are a bunch
> of threads which are running the flush.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19954) ShutdownHook should check whether shutdown hook is tracked by ShutdownHookManager
[
https://issues.apache.org/jira/browse/HBASE-19954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368715#comment-16368715
]
stack commented on HBASE-19954:
---
Good.
So, we need to reinstall the hook for each test? Or break up this class into
multiple classes, one per test, so fs is reinitialized for each? Or collapse
the tests in this class all together into one test w a comment why the test is
so large?
> ShutdownHook should check whether shutdown hook is tracked by
> ShutdownHookManager
> -
>
> Key: HBASE-19954
> URL: https://issues.apache.org/jira/browse/HBASE-19954
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19954.v1.txt
>
>
> Currently ShutdownHook#suppressHdfsShutdownHook() does the following:
> {code}
> synchronized (fsShutdownHooks) {
> boolean isFSCacheDisabled =
> fs.getConf().getBoolean("fs.hdfs.impl.disable.cache", false);
> if (!isFSCacheDisabled &&
> !fsShutdownHooks.containsKey(hdfsClientFinalizer)
> && !ShutdownHookManager.deleteShutdownHook(hdfsClientFinalizer)) {
> {code}
> There is no check that ShutdownHookManager still tracks the shutdown hook,
> leading to potential RuntimeException (as can be observed in hadoop3 Jenkins
> job).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20017: --- Affects Version/s: 1.4.0 1.4.1 1.4.2 > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.4.0, 1.4.1, 1.4.2 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch, HBASE-20017.v0.patch.patch > > > BMI pass a iter of inner buffer to AsyncProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368712#comment-16368712 ] stack commented on HBASE-19978: --- Oh, nice test too. Have you studied it, the swelling of thread count and then the going back down to core pool size? If not, just say, I could take a look sir. Good stuff. > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368708#comment-16368708 ] stack commented on HBASE-19978: --- The test with 50 tables takes a while to run? You see the PE Workers swell up in number and then back down. Should there be any limit? 4x the passed in threads could be anything. User could set it real small. We could run w/ a real small count but now you've added in keepalive, allow it swell to get over the hump so we don't do the deadlock? +1 on commit if only for the nice refactor. You think this all we need for the last failures? Good stuff [~Apache9] > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368661#comment-16368661 ] Sean Busbey commented on HBASE-20017: - what's the affected version(s) here? is this only a problem starting on branch-1.4? > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch, HBASE-20017.v0.patch.patch > > > BMI pass a iter of inner buffer to AsyncProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20018) Safe online META repair
Andrew Purtell created HBASE-20018: -- Summary: Safe online META repair Key: HBASE-20018 URL: https://issues.apache.org/jira/browse/HBASE-20018 Project: HBase Issue Type: New Feature Components: hbck Reporter: Andrew Purtell HBCK is a tank, or a giant shotgun, or choose the battlefield metaphor you feel is most appropriate. It rolls onto the field and leaves problems crushed in its wake, but if you point it in the wrong direction, it will also crush your production data too. As such it is a means of last resort to fix an ailing cluster. It is also imperative that user request traffic, writes in particular, are stopped before attempting a number of the fixes. It is unlikely the default "-repair" option is what you want - this turns on too many fixes to risk at one time. There are a large number of command line switches for individual checks and fixes which are very useful but also error prone when cobbling together a command line for a cluster fix under pressure. An operations team might hesitate to employ hbck to fix some accumulating bad state, because of the disruption use of it requires, and the risk of compounding the problem if not carefully done. That of course would be bad because the accumulating bad state will eventually have an availability impact. It should be safer to use hbck, but changing hbck also carries risk. We can leave it be as the useful (but dangerous) tool it is and focus on a subset of its functionality to make safer. There are a class of META corruptions of mild to moderate severity which could in theory be handled more safely in an online manner without requiring a suspension of user traffic. Some things hbck does are safe enough to use directly for this. Others need tweaks to do more preflight checks (like checking region states) first. Develop these as a separate tool, maybe even a new HMaster or Admin component. Look for opportunities to share code with existing hbck, via refactor into a shared library. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[
https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368656#comment-16368656
]
Hadoop QA commented on HBASE-20017:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
56s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}152m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20017 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911069/HBASE-20017.v0.patch.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 07f15d68743e 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f3bb9b9613 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11568/testReport/ |
| Max. process+thread count |
[jira] [Commented] (HBASE-19506) Support variable sized chunks from ChunkCreator
[ https://issues.apache.org/jira/browse/HBASE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368617#comment-16368617 ] Anastasia Braginsky commented on HBASE-19506: - Great job, Gali! Thank you! The code looks good to me and there left no issues in the RB. I would like to commit. [~ram_krish], [~anoop.hbase] do you have any last comments? > Support variable sized chunks from ChunkCreator > --- > > Key: HBASE-19506 > URL: https://issues.apache.org/jira/browse/HBASE-19506 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Priority: Major > Attachments: HBASE-19506-V01.patch, HBASE-19506-V02.patch, > HBASE-19506-V03.patch, HBASE-19506-V04.patch, HBASE-19506-V05.patch, > HBASE-19506-V06.patch, HBASE-19506-V07.patch, HBASE-19506-V08.patch > > > When CellChunkMap is created it allocates a special index chunk (or chunks) > where array of cell-representations is stored. When the number of > cell-representations is small, it is preferable to allocate a chunk smaller > than a default value which is 2MB. > On the other hand, those "non-standard size" chunks can not be used in pool. > On-demand allocations in off-heap are costly. So this JIRA is about to > investigate the trade of between memory usage and the final performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20017: --- Attachment: HBASE-20017.v0.patch.patch > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch, HBASE-20017.v0.patch.patch > > > BMI pass a iter of inner buffer to AsyncProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[ https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368599#comment-16368599 ] Chia-Ping Tsai commented on HBASE-20016: [~huaxiang] Would you please take a look? Thanks > TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky > - > > Key: HBASE-20016 > URL: https://issues.apache.org/jira/browse/HBASE-20016 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-20016.branch-1.v0.patch.patch, > HBASE-20016.branch-1.v1.patch.patch > > > It is a time-based test. RegionStates#isRegionOnline will return false if the > target region is in transition. The list of region assignment may not updated > yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19506) Support variable sized chunks from ChunkCreator
[
https://issues.apache.org/jira/browse/HBASE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368586#comment-16368586
]
Hadoop QA commented on HBASE-19506:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} hbase-server: The patch generated 0 new + 32
unchanged - 3 fixed = 32 total (was 35) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 48s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}107m
25s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19506 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911063/HBASE-19506-V08.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6c90c8b30d4e 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f3bb9b9613 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11566/testReport/ |
| Max. process+thread count | 4923 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11566/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Support variable sized chunks from ChunkCreator
> ---
>
[jira] [Commented] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[
https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368576#comment-16368576
]
Hadoop QA commented on HBASE-20016:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
0s{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
4s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
58s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}107m
36s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}131m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:36a7029 |
| JIRA Issue | HBASE-20016 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911062/HBASE-20016.branch-1.v1.patch.patch
|
|
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[
https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368573#comment-16368573
]
Hadoop QA commented on HBASE-19978:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
57s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
35s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} hbase-procedure: The patch generated 0 new + 20
unchanged - 9 fixed = 20 total (was 29) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
44s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
22m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}104m
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}159m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19978 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911061/HBASE-19978-v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 38722c6a0eb2 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f3bb9b9613 |
| maven | version: Apache
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[
https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368561#comment-16368561
]
Hadoop QA commented on HBASE-20017:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
23s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
9s{color} | {color:red} hbase-client in branch-1 has 1 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
10s{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
32s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
25s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
24s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 0s{color}
[jira] [Comment Edited] (HBASE-20003) WALLess HBase on Persistent Memory
[ https://issues.apache.org/jira/browse/HBASE-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368553#comment-16368553 ] Anoop Sam John edited comment on HBASE-20003 at 2/18/18 2:37 PM: - Will come up with the doc soon Andy. The point you raised is valid and we thought. Even today when there is a 3 node replication in HDFS (for WAL file) and all the 3 DN servers (where the data is replicated) goes down, the wal is not available for the replay and so temp data not available.Even when the 3 replica RS is down but the servers are reachable, have to have a way to remote access the pmem and get back the data. Good point raised Andy.. We thought about this already. was (Author: anoop.hbase): Will come up with the doc soon Andy. The point you raised is valid and we thought. Even today when there is a 3 node replication in HDFS (for WAL file) and all the 3 servers (where the data is replicated) goes down, the wal is not available for the replay and so temp data not available. > WALLess HBase on Persistent Memory > -- > > Key: HBASE-20003 > URL: https://issues.apache.org/jira/browse/HBASE-20003 > Project: HBase > Issue Type: New Feature >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > > This JIRA aims to make use of persistent memory (pmem) technologies in HBase. > One such usage is to make the Memstore to reside on pmem. Making a persistent > memstore would remove the need for WAL and paves way for a WALLess HBase. > The existing region replica feature could be used here and ensure the data > written to memstores are synchronously replicated to the replicas and ensure > strong consistency of the data. (pipeline model) > Advantages : > -Data Availability : Since the data across replicas are consistent > (synchronously written) our data is always 100 % available. > -Lower MTTR : It becomes easier/faster to switch over to the replicas on a > primary region failure as there is no WAL replay involved. Building the > memstore map data also is much faster than reading the WAL and replaying the > WAL. > -Possibility of bigger memstores : These pmems are designed to have more > memory than DRAMs so it would also enable us to have bigger sized memstores > which leads to lesser flushes/compaction IO. > -Removes the dependency of HDFS on the write path > Initial PoC has been designed and developed. Testing is underway and we would > publish the PoC results along with the design doc sooner. The PoC doc will > talk about the design decisions, the libraries considered to work with these > pmem devices, pros and cons of those libraries and the performance results. > Note : Next gen memory technologies using 3DXPoint gives persistent memory > feature. Such memory DIMMs are soon to appear in the market. The PoC is done > around Intel's ApachePass (AEP) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20003) WALLess HBase on Persistent Memory
[
https://issues.apache.org/jira/browse/HBASE-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368554#comment-16368554
]
Anoop Sam John commented on HBASE-20003:
{quote}
It would also be interesting to hear about how, for the case where all replicas
go down, and only a subset are recovered, how we know the recovered subset
actually has the latest data. This will depend on the guarantees made by the
means of synchronous replication. A typical majority/consensus approach would
mean out of 3 replicas, 1 might be stale, and if you only bring that 1 back
into service, you'd lose data. If 2 of the replicas are brought back, we could
say most recent view wins? Need to think about that.
{quote}
It would be the last replica commit first and when the primary respond to the
client (for write), the replicas are having committed data. Same way as
present today. It can so happen today that the wal is synced and then the RS
went down before responding to client. But the data is actually committed as
the WAL replay will get it back. Same way when we choose the secondary replica
and turn it as new primary, there will be no data loss.
> WALLess HBase on Persistent Memory
> --
>
> Key: HBASE-20003
> URL: https://issues.apache.org/jira/browse/HBASE-20003
> Project: HBase
> Issue Type: New Feature
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Major
>
> This JIRA aims to make use of persistent memory (pmem) technologies in HBase.
> One such usage is to make the Memstore to reside on pmem. Making a persistent
> memstore would remove the need for WAL and paves way for a WALLess HBase.
> The existing region replica feature could be used here and ensure the data
> written to memstores are synchronously replicated to the replicas and ensure
> strong consistency of the data. (pipeline model)
> Advantages :
> -Data Availability : Since the data across replicas are consistent
> (synchronously written) our data is always 100 % available.
> -Lower MTTR : It becomes easier/faster to switch over to the replicas on a
> primary region failure as there is no WAL replay involved. Building the
> memstore map data also is much faster than reading the WAL and replaying the
> WAL.
> -Possibility of bigger memstores : These pmems are designed to have more
> memory than DRAMs so it would also enable us to have bigger sized memstores
> which leads to lesser flushes/compaction IO.
> -Removes the dependency of HDFS on the write path
> Initial PoC has been designed and developed. Testing is underway and we would
> publish the PoC results along with the design doc sooner. The PoC doc will
> talk about the design decisions, the libraries considered to work with these
> pmem devices, pros and cons of those libraries and the performance results.
> Note : Next gen memory technologies using 3DXPoint gives persistent memory
> feature. Such memory DIMMs are soon to appear in the market. The PoC is done
> around Intel's ApachePass (AEP)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20003) WALLess HBase on Persistent Memory
[ https://issues.apache.org/jira/browse/HBASE-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368553#comment-16368553 ] Anoop Sam John commented on HBASE-20003: Will come up with the doc soon Andy. The point you raised is valid and we thought. Even today when there is a 3 node replication in HDFS (for WAL file) and all the 3 servers (where the data is replicated) goes down, the wal is not available for the replay and so temp data not available. > WALLess HBase on Persistent Memory > -- > > Key: HBASE-20003 > URL: https://issues.apache.org/jira/browse/HBASE-20003 > Project: HBase > Issue Type: New Feature >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > > This JIRA aims to make use of persistent memory (pmem) technologies in HBase. > One such usage is to make the Memstore to reside on pmem. Making a persistent > memstore would remove the need for WAL and paves way for a WALLess HBase. > The existing region replica feature could be used here and ensure the data > written to memstores are synchronously replicated to the replicas and ensure > strong consistency of the data. (pipeline model) > Advantages : > -Data Availability : Since the data across replicas are consistent > (synchronously written) our data is always 100 % available. > -Lower MTTR : It becomes easier/faster to switch over to the replicas on a > primary region failure as there is no WAL replay involved. Building the > memstore map data also is much faster than reading the WAL and replaying the > WAL. > -Possibility of bigger memstores : These pmems are designed to have more > memory than DRAMs so it would also enable us to have bigger sized memstores > which leads to lesser flushes/compaction IO. > -Removes the dependency of HDFS on the write path > Initial PoC has been designed and developed. Testing is underway and we would > publish the PoC results along with the design doc sooner. The PoC doc will > talk about the design decisions, the libraries considered to work with these > pmem devices, pros and cons of those libraries and the performance results. > Note : Next gen memory technologies using 3DXPoint gives persistent memory > feature. Such memory DIMMs are soon to appear in the market. The PoC is done > around Intel's ApachePass (AEP) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368549#comment-16368549 ] Eshcar Hillel commented on HBASE-18294: --- Pushed to master and branch-2. No test failures on master (still waiting for Hudson report on branch-2). Thanks all for the comments and feedback. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, > HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, > HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, > HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, > HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, > HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.branch-2.01.patch, > HBASE-18294.master.01.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19506) Support variable sized chunks from ChunkCreator
[ https://issues.apache.org/jira/browse/HBASE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gali Sheffi updated HBASE-19506: Attachment: HBASE-19506-V08.patch > Support variable sized chunks from ChunkCreator > --- > > Key: HBASE-19506 > URL: https://issues.apache.org/jira/browse/HBASE-19506 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Priority: Major > Attachments: HBASE-19506-V01.patch, HBASE-19506-V02.patch, > HBASE-19506-V03.patch, HBASE-19506-V04.patch, HBASE-19506-V05.patch, > HBASE-19506-V06.patch, HBASE-19506-V07.patch, HBASE-19506-V08.patch > > > When CellChunkMap is created it allocates a special index chunk (or chunks) > where array of cell-representations is stored. When the number of > cell-representations is small, it is preferable to allocate a chunk smaller > than a default value which is 2MB. > On the other hand, those "non-standard size" chunks can not be used in pool. > On-demand allocations in off-heap are costly. So this JIRA is about to > investigate the trade of between memory usage and the final performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368546#comment-16368546 ] Hudson commented on HBASE-18294: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4605 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4605/]) HBASE-18294 Reduce global heap pressure: flush based on heap occupancy (eshcar: rev f3bb9b9613beaefbee0a53be1fb66b94fde1ce19) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLAB.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregateImpl.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientPushback.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreSizing.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableSegment.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALEdit.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AbstractMemStore.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerSideScanMetricsFromClientSide.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestPerColumnFamilyFlush.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableMemStoreLAB.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingMemStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSizeFailures.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestPartialResultsFromClientSide.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionPipeline.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CellArrayImmutableSegment.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAccounting.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java * (edit) hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/util/MapReduceExtendedCell.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompositeImmutableSegment.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestNegativeMemStoreSizeWithSlowCoprocessor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWalAndCompactingMemStoreFlush.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MutableSegment.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWALLockup.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreSnapshot.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreSize.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/FlushLargeStoresPolicy.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServicesForStores.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFlushFromClient.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java * (edit)
[jira] [Updated] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[ https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20016: --- Attachment: HBASE-20016.branch-1.v1.patch.patch > TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky > - > > Key: HBASE-20016 > URL: https://issues.apache.org/jira/browse/HBASE-20016 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-20016.branch-1.v0.patch.patch, > HBASE-20016.branch-1.v1.patch.patch > > > It is a time-based test. RegionStates#isRegionOnline will return false if the > target region is in transition. The list of region assignment may not updated > yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20017: --- Description: BMI pass a iter of inner buffer to AsyncProcess to take the undealt mutations, AsyncProcess call iter#next to get the mutation and then call iter#remove to delete the mutation from inner buffer. Hence, There's a good chance that the mutation is processed repeatedly in case there are a bunch of threads which are running the flush. (was: BMI pass a iter of inner buffer to AccessProcess to take the undealt mutations, AsyncProcess call iter#next to get the mutation and then call iter#remove to delete the mutation from inner buffer. Hence, There's a good chance that the mutation is processed repeatedly in case there are a bunch of threads which are running the flush.) > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch > > > BMI pass a iter of inner buffer to AsyncProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368543#comment-16368543 ] Chia-Ping Tsai commented on HBASE-20017: Thanks [~yuzhih...@gmail.com]. > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch > > > BMI pass a iter of inner buffer to AsyncProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368542#comment-16368542 ] Ted Yu commented on HBASE-20017: Lgtm The AccessProcess in description seems to be typo. > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch > > > BMI pass a iter of inner buffer to AccessProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[ https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368540#comment-16368540 ] Chia-Ping Tsai commented on HBASE-20016: It takes ~10 second on my computer. Given the timeout for the ut is 180 second, I will increase the wait time to 60 second. > TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky > - > > Key: HBASE-20016 > URL: https://issues.apache.org/jira/browse/HBASE-20016 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-20016.branch-1.v0.patch.patch > > > It is a time-based test. RegionStates#isRegionOnline will return false if the > target region is in transition. The list of region assignment may not updated > yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368538#comment-16368538 ] Duo Zhang commented on HBASE-19978: --- Rewrite the UT. Try reverting HBASE-2 on master and this UT will timed out when failover. With HBASE-2 and HBASE-19978 it could pass. [~stack] FYI. > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19978) The keepalive logic is incomplete in ProcedureExecutor
[ https://issues.apache.org/jira/browse/HBASE-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19978: -- Attachment: HBASE-19978-v1.patch > The keepalive logic is incomplete in ProcedureExecutor > -- > > Key: HBASE-19978 > URL: https://issues.apache.org/jira/browse/HBASE-19978 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19978-v1.patch, HBASE-19978.patch > > > The worker thread will just exit after keep alive time, and we never add it > back. The only way to add it back is through the stuck checker, this is not > correct. Here we should start new worker thread if it is under the core pool > size and there are pending procedures. > For now the default keep alive time is Long.MAX_VALUE which means no timeout > so no problem, but we do allow users to set it so we need to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[ https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368537#comment-16368537 ] Ted Yu commented on HBASE-20016: Is 30 second wait long enough ? Otherwise looks good. > TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky > - > > Key: HBASE-20016 > URL: https://issues.apache.org/jira/browse/HBASE-20016 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-20016.branch-1.v0.patch.patch > > > It is a time-based test. RegionStates#isRegionOnline will return false if the > target region is in transition. The list of region assignment may not updated > yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[
https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368536#comment-16368536
]
Hadoop QA commented on HBASE-20016:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 5m
28s{color} | {color:red} Docker failed to build yetus/hbase:36a7029. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20016 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911060/HBASE-20016.branch-1.v0.patch.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11563/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
> -
>
> Key: HBASE-20016
> URL: https://issues.apache.org/jira/browse/HBASE-20016
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 1.5.0, 1.4.3
>
> Attachments: HBASE-20016.branch-1.v0.patch.patch
>
>
> It is a time-based test. RegionStates#isRegionOnline will return false if the
> target region is in transition. The list of region assignment may not updated
> yet.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[ https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20016: --- Attachment: HBASE-20016.branch-1.v0.patch.patch > TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky > - > > Key: HBASE-20016 > URL: https://issues.apache.org/jira/browse/HBASE-20016 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-20016.branch-1.v0.patch.patch > > > It is a time-based test. RegionStates#isRegionOnline will return false if the > target region is in transition. The list of region assignment may not updated > yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
[ https://issues.apache.org/jira/browse/HBASE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20016: --- Status: Patch Available (was: Open) > TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky > - > > Key: HBASE-20016 > URL: https://issues.apache.org/jira/browse/HBASE-20016 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-20016.branch-1.v0.patch.patch > > > It is a time-based test. RegionStates#isRegionOnline will return false if the > target region is in transition. The list of region assignment may not updated > yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20017: --- Attachment: HBASE-20017.branch-1.v0.patch.patch > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch > > > BMI pass a iter of inner buffer to AccessProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368535#comment-16368535 ] Chia-Ping Tsai commented on HBASE-20017: The failed tests pass locally. try the patch for branch-1 > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch > > > BMI pass a iter of inner buffer to AccessProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[
https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368529#comment-16368529
]
Hadoop QA commented on HBASE-20017:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
7s{color} | {color:red} hbase-client in branch-1.4 has 1 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
9s{color} | {color:red} hbase-server in branch-1.4 has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 52s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
16s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 23s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368517#comment-16368517
]
Hadoop QA commented on HBASE-18294:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-18294 does not apply to branch-2. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18294 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911057/HBASE-18294.branch-2.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11562/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch,
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch,
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch,
> HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch,
> HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch,
> HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch,
> HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch,
> HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch,
> HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.branch-2.01.patch,
> HBASE-18294.master.01.patch, HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.branch-2.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, > HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, > HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, > HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, > HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, > HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.branch-2.01.patch, > HBASE-18294.master.01.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20017: --- Status: Patch Available (was: Open) > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch > > > BMI pass a iter of inner buffer to AccessProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368505#comment-16368505 ] Chia-Ping Tsai commented on HBASE-20017: the v0 patch add a trivial change to hbase-server module to trigger full QA > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch > > > BMI pass a iter of inner buffer to AccessProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-20017: --- Attachment: HBASE-20017.branch-1.4.v0.patch.patch > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.5.0, 1.4.3 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch > > > BMI pass a iter of inner buffer to AccessProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
Chia-Ping Tsai created HBASE-20017: -- Summary: BufferedMutatorImpl submit the same mutation repeatedly Key: HBASE-20017 URL: https://issues.apache.org/jira/browse/HBASE-20017 Project: HBase Issue Type: Bug Components: Client Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0, 1.5.0, 1.4.3 BMI pass a iter of inner buffer to AccessProcess to take the undealt mutations, AsyncProcess call iter#next to get the mutation and then call iter#remove to delete the mutation from inner buffer. Hence, There's a good chance that the mutation is processed repeatedly in case there are a bunch of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19400) Add missing security hooks for MasterService RPCs
[
https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368478#comment-16368478
]
Hadoop QA commented on HBASE-19400:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
11s{color} | {color:red} hbase-server: The patch generated 5 new + 311
unchanged - 6 fixed = 316 total (was 317) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} hbase-rsgroup: The patch generated 1 new + 0 unchanged
- 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}109m 0s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
16s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.security.access.TestWithDisabledAuthorization |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19400 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911049/HBASE-19400.master.006.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7a397556c055 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
