[jira] [Commented] (HBASE-22040) Add mergeRegionsAsync with a List of region names method in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-22040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794148#comment-16794148
]
Hadoop QA commented on HBASE-22040:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} hbase-client: The patch generated 0 new + 140
unchanged - 4 fixed = 140 total (was 144) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} hbase-server: The patch generated 0 new + 48
unchanged - 2 fixed = 48 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 50s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
14s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}155m 28s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
50s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}205m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22040 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12962680/HBASE-22040-v3.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 067fcc4d9cd3 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794145#comment-16794145 ] stack commented on HBASE-22052: --- 2.0.003 Fix the UT. Had to do similar exclusions in hbase-rest as done in hbase-it. > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch, > HBASE-22052.branch-2.0.003.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22052: -- Attachment: HBASE-22052.branch-2.0.003.patch > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch, > HBASE-22052.branch-2.0.003.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22051) Expect values are hard-coded in the verifications of TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794143#comment-16794143
]
Hadoop QA commented on HBASE-22051:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
2s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22051 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12962685/HBASE-22051.master.000.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3d42b7c7b154 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f1ebbb928b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16397/testReport/ |
| Max. process+thread count | 4071 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16397/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Expect values
[jira] [Comment Edited] (HBASE-22029) RESTApiClusterManager.java:[250,48] cannot find symbol in hbase-it
[ https://issues.apache.org/jira/browse/HBASE-22029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793416#comment-16793416 ] stack edited comment on HBASE-22029 at 3/16/19 5:22 AM: Yeah, HBASE-22052 will fix this. jersey-core is problematic. It was transitively included from hadoop and polluting our CLASSPATH with an implementation of a 1.x version of the javax.ws.rs.core.Response Interface from jsr311-api when we want the javax.ws.rs-api 2.x version. Unfortunately, the soln. was not simple -- a purge of jersey-core -- because w/o it some mr unit tests fail. See gore over in HBASE-22052 . was (Author: stack): Yeah, HBASE-22029 will fix this. jersey-core is problematic. It was transitively included from hadoop and polluting our CLASSPATH with an implementation of a 1.x version of the javax.ws.rs.core.Response Interface from jsr311-api when we want the javax.ws.rs-api 2.x version. Unfortunately, the soln. was not simple -- a purge of jersey-core -- because w/o it some mr unit tests fail. See gore over in HBASE-22029. > RESTApiClusterManager.java:[250,48] cannot find symbol in hbase-it > -- > > Key: HBASE-22029 > URL: https://issues.apache.org/jira/browse/HBASE-22029 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > > I get this doing a RM build. Can't repro elsewhere. > Picking up an old jaxrs? See > https://stackoverflow.com/questions/34679773/extract-string-from-javax-response > Let me try adding explicit dependency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[
https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794138#comment-16794138
]
Sean Busbey commented on HBASE-21926:
-
I won't have time to make my desired changes until early next week earliest.
$dayjob priorities changed for this last week.
bq. I'm going to ignore further ImportOrder checkstyle warnings, because it
doesn't seem to matter where the import is placed, there is still a warning.
Tired of guessing what is the right answer. As this is the only checkstyle
warning the patch looks good IMHO.
FWIW, there's supposed to be both an IDE formatter config and a write up
somewhere explaining the current configuration. [we define 3 import
"groups"|https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml#L71].
which I think means imports should be "things that aren't thirdparty/shaded
first in alpha order", followed by "thirdparty in alpha order", followed by
"shaded in alpha order".
If I understand the config correctly, in ProfileServlet this line should be
after all of the other imports:
{code}
34 import org.apache.hbase.thirdparty.com.google.common.base.Joiner;
{code}
and in ProcessUtils this line should be before the slf4j imports
{code}
27 import org.apache.yetus.audience.InterfaceAudience;
{code}
> Profiler servlet
>
>
> Key: HBASE-21926
> URL: https://issues.apache.org/jira/browse/HBASE-21926
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, regionserver
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.3.0
>
> Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch,
> HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, HBASE-21926.patch,
> HBASE-21926.patch, HBASE-21926.patch
>
>
> HIVE-20202 describes how Hive added a web endpoint for online in production
> profiling based on async-profiler. The endpoint was added as a servlet to
> httpserver and supports retrieval of flamegraphs compiled from the profiler
> trace. Async profiler
> ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile
> heap allocations, lock contention, and HW performance counters in addition to
> CPU.
> The profiling overhead is pretty low and is safe to run in production. The
> async-profiler project measured and describes CPU and memory overheads on
> these issues:
> [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and
> [https://github.com/jvm-profiling-tools/async-profiler/issues/131]
> We have an httpserver based servlet stack so we can use HIVE-20202 as an
> implementation template for a similar feature for HBase daemons. Ideally we
> achieve these requirements:
> * Retrieve flamegraph SVG generated from latest profile trace.
> * Online enable and disable of profiling activity. (async-profiler does not
> do instrumentation based profiling so this should not cause the code gen
> related perf problems of that other approach and can be safely toggled on and
> off while under production load.)
> * CPU profiling.
> * ALLOCATION profiling.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22032) KeyValue validation should check for null byte array
[
https://issues.apache.org/jira/browse/HBASE-22032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794132#comment-16794132
]
Hudson commented on HBASE-22032:
Results for branch branch-2.1
[build #961 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/961/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/961//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/961//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/961//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> KeyValue validation should check for null byte array
>
>
> Key: HBASE-22032
> URL: https://issues.apache.org/jira/browse/HBASE-22032
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.4, 2.1.3
>Reporter: Geoffrey Jacoby
>Assignee: Geoffrey Jacoby
>Priority: Major
> Fix For: 3.0.0, 2.1.4, 2.2.1
>
> Attachments: HBASE-22032.v01.patch, HBASE-22032.v02.patch,
> HBASE-22032.v03.patch
>
>
> HBASE-21401 added some nice validation checks to throw precise errors if a
> KeyValue is constructed using invalid parameters. However it implicitly
> assumes that the KeyValue buffer is not null. It should validate this
> assumption and alert accordingly rather than throwing an NPE from an
> unrelated check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22021) A small refactoring for NettyServerCall.sendResponseIfReady
[ https://issues.apache.org/jira/browse/HBASE-22021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-22021: --- Status: Open (was: Patch Available) > A small refactoring for NettyServerCall.sendResponseIfReady > --- > > Key: HBASE-22021 > URL: https://issues.apache.org/jira/browse/HBASE-22021 > Project: HBase > Issue Type: Improvement > Components: rpc >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Trivial > Labels: starter > > before: > connection.channel.writeAndFlush(this); > > after: > connection.doRespond(this); -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794112#comment-16794112
]
Xiang Li edited comment on HBASE-22051 at 3/16/19 3:50 AM:
---
Uploaded patch v000. The logic change of it includes
* 3 places mentioned in the "description", to remove the hard-coded
verification and use variables instead.
* More strict verifications at the end of testClearNotProcessedDeadServer().
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments.
[~xucang], would you please review it at your convenience?
was (Author: water):
Uploaded patch v000. The logic change of it includes
* 3 places mentioned in the "description", to remove the hard-coded
verification and use variables instead.
* More strict verifications at the end of testClearNotProcessedDeadServer().
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments.
[~xucang], would you please review it at your convenience?
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[
https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794120#comment-16794120
]
Hadoop QA commented on HBASE-22034:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
25s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} hbase-common: The patch generated 1 new + 240
unchanged - 8 fixed = 241 total (was 248) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} hbase-server: The patch generated 0 new + 16
unchanged - 4 fixed = 16 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
26s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 35s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
31s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}110m 38s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not
[jira] [Comment Edited] (HBASE-22051) Expect values are hard-coded in the verifications of TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794112#comment-16794112
]
Xiang Li edited comment on HBASE-22051 at 3/16/19 3:58 AM:
---
Uploaded patch v000. The logic change of it includes
* 3 places mentioned in the "description", to remove the hard-coded
verification and use variables instead.
* More strict verifications at the end of testClearNotProcessedDeadServer().
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup".
* Add or correct some comments.
[~xucang], would you please review it at your convenience?
was (Author: water):
Uploaded patch v000. The logic change of it includes
* 3 places mentioned in the "description", to remove the hard-coded
verification and use variables instead.
* More strict verifications at the end of testClearNotProcessedDeadServer().
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments.
[~xucang], would you please review it at your convenience?
> Expect values are hard-coded in the verifications of TestRSGroupsBasics
> ---
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22051.master.000.patch
>
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect values are hard-coded in the verifications of TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Status: Patch Available (was: Open)
> Expect values are hard-coded in the verifications of TestRSGroupsBasics
> ---
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22051.master.000.patch
>
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect values are hard-coded in the verifications of TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Attachment: HBASE-22051.master.000.patch
> Expect values are hard-coded in the verifications of TestRSGroupsBasics
> ---
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22051.master.000.patch
>
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect values are hard-coded in the verifications of TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Summary: Expect values are hard-coded in the verifications of
TestRSGroupsBasics (was: Expect values are hard-coded in TestRSGroupsBasics)
> Expect values are hard-coded in the verifications of TestRSGroupsBasics
> ---
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect values are hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Summary: Expect values are hard-coded in TestRSGroupsBasics (was: Expect
value is hard-coded in TestRSGroupsBasics)
> Expect values are hard-coded in TestRSGroupsBasics
> --
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794112#comment-16794112
]
Xiang Li edited comment on HBASE-22051 at 3/16/19 3:50 AM:
---
Uploaded patch v000. The logic change of it includes
* 3 places mentioned in the "description", to remove the hard-coded
verification and use variables instead.
* More strict verifications at the end of testClearNotProcessedDeadServer().
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments.
[~xucang], would you please review it at your convenience?
was (Author: water):
Uploaded patch v000. The logic change of it includes the 3 places mentioned in
the "description", to remove the hard-coded verification and use variables
instead.
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments
[~xucang], would you please review it at your convenience?
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794112#comment-16794112
]
Xiang Li edited comment on HBASE-22051 at 3/16/19 3:36 AM:
---
Uploaded patch v000. The logic change of it includes the 3 places mentioned in
the "description", to remove the hard-coded verification and use variables
instead.
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments
[~xucang], would you please review it at your convenience?
was (Author: water):
Uploaded patch v000. The logic change of it includes the 3 places mentioned in
the "description", to remove the hard-coded verification and use the variables
instead.
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments
[~xucang], would you please review it at your convenience?
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794112#comment-16794112
]
Xiang Li commented on HBASE-22051:
--
Uploaded patch v000. The logic change of it includes the 3 places mentioned in
the "description", to remove the hard-coded verification and use the variables
instead.
The patches also make some changes to improve the readability:
* Rename some variables, like "targetServer" --> "serverToStop", "appInfo" -->
"deadServerGroup"
* Add or correct some comments
[~xucang], would you please review it at your convenience?
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Description:
In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
mini cluster. But in some verifications of TestGroupsBasics, such as in
testBasicStartUp(), the expected value is hard-coded as 4, like:
{code:java}
public void testBasicStartUp() throws IOException {
...
assertEquals(4, defaultInfo.getServers().size());
..
}
{code}
We could also some other places which have hard-coded verifications, as follow:
{code:java}
public void testClearDeadServers() throws Exception {
...
final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()), 3);
...
assertEquals(2, newGroupServers.size());
{code}
and
{code:java}
public void testClearNotProcessedDeadServer() throws Exception {
...
RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
...
assertEquals(1, notClearedServers.size());
{code}
was:
In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
mini cluster. But in some verifications of TestGroupsBasics, such as in
testBasicStartUp(), the expected value is hard-coded as 4, like:
{code:java}
public void testBasicStartUp() throws IOException {
...
assertEquals(4, defaultInfo.getServers().size());
..
}
{code}
We could also see some hard-coded verifications, as follow:
{code:java}
public void testClearDeadServers() throws Exception {
...
final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()), 3);
...
assertEquals(2, newGroupServers.size());
{code}
and
{code:java}
public void testClearNotProcessedDeadServer() throws Exception {
...
RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
...
assertEquals(1, notClearedServers.size());
{code}
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also some other places which have hard-coded verifications, as
> follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Description:
In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
mini cluster. But in some verifications of TestGroupsBasics, such as in
testBasicStartUp(), the expected value is hard-coded as 4, like:
{code:java}
public void testBasicStartUp() throws IOException {
...
assertEquals(4, defaultInfo.getServers().size());
..
}
{code}
We could also see some hard-coded verifications, as follow:
{code:java}
public void testClearDeadServers() throws Exception {
...
final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()), 3);
...
assertEquals(2, newGroupServers.size());
{code}
and
{code:java}
public void testClearNotProcessedDeadServer() throws Exception {
...
RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
...
assertEquals(1, notClearedServers.size());
{code}
was:
In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
mini cluster. But in some verifications of TestGroupsBasics, such as in
testBasicStartUp(), the expected value is hard-coded as 4, like:
{code:java}
public void testBasicStartUp() throws IOException {
...
assertEquals(4, defaultInfo.getServers().size());
..
}
{code}
and
{code:java}
public void testClearDeadServers() throws Exception {
...
final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()), 3);
...
assertEquals(2, newGroupServers.size());
{code}
and
{code:java}
public void testClearNotProcessedDeadServer() throws Exception {
...
RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
...
assertEquals(1, notClearedServers.size());
{code}
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> We could also see some hard-coded verifications, as follow:
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Description:
In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
mini cluster. But in some verifications of TestGroupsBasics, such as in
testBasicStartUp(), the expected value is hard-coded as 4, like:
{code:java}
public void testBasicStartUp() throws IOException {
...
assertEquals(4, defaultInfo.getServers().size());
..
}
{code}
and
{code:java}
public void testClearDeadServers() throws Exception {
...
final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()), 3);
...
assertEquals(2, newGroupServers.size());
{code}
and
{code:java}
public void testClearNotProcessedDeadServer() throws Exception {
...
RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
...
assertEquals(1, notClearedServers.size());
{code}
was:
In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
mini cluster. But in some verifications of TestGroupsBasics, such as in
testBasicStartUp(), the expected value is hard-coded as 4, like:
{code:java}
public void testBasicStartUp() throws IOException {
...
assertEquals(4, defaultInfo.getServers().size());
..
}
{code}
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
> and
> {code:java}
> public void testClearDeadServers() throws Exception {
> ...
> final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
> 3);
> ...
> assertEquals(2, newGroupServers.size());
> {code}
> and
> {code:java}
> public void testClearNotProcessedDeadServer() throws Exception {
> ...
> RSGroupInfo appInfo = addGroup("deadServerGroup", 1);
> ...
> assertEquals(1, notClearedServers.size());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22040) Add mergeRegionsAsync with a List of region names method in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22040: -- Attachment: HBASE-22040-branch-2.patch > Add mergeRegionsAsync with a List of region names method in AsyncAdmin > -- > > Key: HBASE-22040 > URL: https://issues.apache.org/jira/browse/HBASE-22040 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22040-branch-2.patch, HBASE-22040-v1.patch, > HBASE-22040-v2.patch, HBASE-22040-v3.patch, HBASE-22040.patch > > > Although we only support merging two regions until now, but the rpc protocol > does support passing more than two regions to master. > So I think we should provide the methods, but need to add comments to say > that for now we only support merging two regions so do not try to pass more > than two regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22040) Add mergeRegionsAsync with a List of region names method in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22040: -- Attachment: HBASE-22040-v3.patch > Add mergeRegionsAsync with a List of region names method in AsyncAdmin > -- > > Key: HBASE-22040 > URL: https://issues.apache.org/jira/browse/HBASE-22040 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22040-v1.patch, HBASE-22040-v2.patch, > HBASE-22040-v3.patch, HBASE-22040.patch > > > Although we only support merging two regions until now, but the rpc protocol > does support passing more than two regions to master. > So I think we should provide the methods, but need to add comments to say > that for now we only support merging two regions so do not try to pass more > than two regions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22039) Should add the synchronous parameter for the XXXSwitch method in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22039: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2.2+. Thanks [~openinx] for reviewing. > Should add the synchronous parameter for the XXXSwitch method in AsyncAdmin > --- > > Key: HBASE-22039 > URL: https://issues.apache.org/jira/browse/HBASE-22039 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22039-v1.patch, HBASE-22039-v1.patch, > HBASE-22039-v1.patch, HBASE-22039.patch > > > For now we always pass true to HMaster, maybe the decision is that user just > do not need to calling get on the returned Future if it wants asynchronous. > But the problem here is that, the return value is not void, it is a boolean, > which is the previous state of the flag, sometimes users do not need to wait > until the previous transitions or split/merge to complete, but they still > want to get the previous value of the flag. > So we still need to provide the synchronous parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22039) Should add the synchronous parameter for the XXXSwitch method in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22039: -- Release Note: Add drainXXX parameter for balancerSwitch/splitSwitch/mergeSwitch methods in the AsyncAdmin interface, which has the same meaning with the synchronous parameter for these methods in the Admin interface. > Should add the synchronous parameter for the XXXSwitch method in AsyncAdmin > --- > > Key: HBASE-22039 > URL: https://issues.apache.org/jira/browse/HBASE-22039 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22039-v1.patch, HBASE-22039-v1.patch, > HBASE-22039-v1.patch, HBASE-22039.patch > > > For now we always pass true to HMaster, maybe the decision is that user just > do not need to calling get on the returned Future if it wants asynchronous. > But the problem here is that, the return value is not void, it is a boolean, > which is the previous state of the flag, sometimes users do not need to wait > until the previous transitions or split/merge to complete, but they still > want to get the previous value of the flag. > So we still need to provide the synchronous parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22042) Missing @Override annotation for RawAsyncTableImpl.scan
[ https://issues.apache.org/jira/browse/HBASE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794091#comment-16794091 ] Duo Zhang commented on HBASE-22042: --- Thanks [~rishjain]. Will commit soon. > Missing @Override annotation for RawAsyncTableImpl.scan > --- > > Key: HBASE-22042 > URL: https://issues.apache.org/jira/browse/HBASE-22042 > Project: HBase > Issue Type: Task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Rishabh Jain >Priority: Major > Labels: beginner, trivial > Attachments: HBASE-22042.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22042) Missing @Override annotation for RawAsyncTableImpl.scan
[ https://issues.apache.org/jira/browse/HBASE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22042: -- Labels: beginner trivial (was: beginner) > Missing @Override annotation for RawAsyncTableImpl.scan > --- > > Key: HBASE-22042 > URL: https://issues.apache.org/jira/browse/HBASE-22042 > Project: HBase > Issue Type: Task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Rishabh Jain >Priority: Major > Labels: beginner, trivial > Attachments: HBASE-22042.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[ https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794090#comment-16794090 ] Duo Zhang commented on HBASE-21895: --- [~apurtell] I'm not sure how a separated repo works but I believe it will not be much different with a sub module? And it will be easy to break the tie, where we should not run error prone check on the hbase-error-prone module as it will introduce cyclic dependency... Anyway, seems there is no customized checks for now? We can try it later when we actually have some customized checks in the future... Thanks. > Take more care on the error prone plugin and warnings > - > > Key: HBASE-21895 > URL: https://issues.apache.org/jira/browse/HBASE-21895 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21895.master.001.patch > > > As shown in HBASE-21890 , the error prone warnings are truly useful. > But now, the javac warnings section in the pre commit result is messed up, it > always reports unrelated warnings. Need to take a look at it. > And also, we should try out best to clean up the warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[
https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794081#comment-16794081
]
Duo Zhang commented on HBASE-21895:
---
{quote}
{quote}
Good, this is exactly what I missed before. After upgrading to the new error
prone version, it keeps telling me unknown compiler flags...
And the compile is also failed? Now we enable error prone by default after the
patch?
Thanks.
> Take more care on the error prone plugin and warnings
> -
>
> Key: HBASE-21895
> URL: https://issues.apache.org/jira/browse/HBASE-21895
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Attachments: HBASE-21895.master.001.patch
>
>
> As shown in HBASE-21890 , the error prone warnings are truly useful.
> But now, the javac warnings section in the pre commit result is messed up, it
> always reports unrelated warnings. Need to take a look at it.
> And also, we should try out best to clean up the warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[
https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22051:
-
Description:
In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
mini cluster. But in some verifications of TestGroupsBasics, such as in
testBasicStartUp(), the expected value is hard-coded as 4, like:
{code:java}
public void testBasicStartUp() throws IOException {
...
assertEquals(4, defaultInfo.getServers().size());
..
}
{code}
> Expect value is hard-coded in TestRSGroupsBasics
>
>
> Key: HBASE-22051
> URL: https://issues.apache.org/jira/browse/HBASE-22051
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup, test
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In TestRSGroupsBase, we have NUM_SLAVES_BASE = 4, which is used to launch the
> mini cluster. But in some verifications of TestGroupsBasics, such as in
> testBasicStartUp(), the expected value is hard-coded as 4, like:
> {code:java}
> public void testBasicStartUp() throws IOException {
> ...
> assertEquals(4, defaultInfo.getServers().size());
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Expect value is hard-coded in TestRSGroupsBasics
[ https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-22051: - Summary: Expect value is hard-coded in TestRSGroupsBasics (was: Remove the hardcoded expect value in TestRSGroupsBasics) > Expect value is hard-coded in TestRSGroupsBasics > > > Key: HBASE-22051 > URL: https://issues.apache.org/jira/browse/HBASE-22051 > Project: HBase > Issue Type: Improvement > Components: rsgroup, test >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22021) A small refactoring for NettyServerCall.sendResponseIfReady
[ https://issues.apache.org/jira/browse/HBASE-22021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-22021: --- Attachment: (was: HBASE-22021.patch) > A small refactoring for NettyServerCall.sendResponseIfReady > --- > > Key: HBASE-22021 > URL: https://issues.apache.org/jira/browse/HBASE-22021 > Project: HBase > Issue Type: Improvement > Components: rpc >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Trivial > Labels: starter > > before: > connection.channel.writeAndFlush(this); > > after: > connection.doRespond(this); -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[
https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794079#comment-16794079
]
Hadoop QA commented on HBASE-21895:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-build-support hbase-build-configuration . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 4m
8s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 4m 8s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
5s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 55s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-build-configuration . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}279m 2s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
52s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}330m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated |
| | hadoop.hbase.master.procedure.TestSCPWithReplicas |
| | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce
[jira] [Commented] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794077#comment-16794077 ] Andrew Purtell commented on HBASE-22034: Updated patch for checkstyle and javadoc warnings > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch, HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-22034: --- Attachment: HBASE-22034-branch-1.patch > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch, HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794075#comment-16794075 ] Andrew Purtell commented on HBASE-22034: Great, I munged some files so now I own their checkstyle and javadoc problems. :-/ > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[
https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794051#comment-16794051
]
Hadoop QA commented on HBASE-22034:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
2s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
6s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} hbase-common: The patch generated 20 new + 240
unchanged - 8 fixed = 260 total (was 248) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} hbase-server: The patch generated 0 new + 16
unchanged - 4 fixed = 16 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 49s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red} hbase-common-jdk1.8.0_202 with JDK v1.8.0_202
generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
21s{color} | {color:red} hbase-common-jdk1.7.0_211 with JDK v1.7.0_211
generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
45s{color} | {color:green} hbase-server in the patch passed. {color} |
|
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794025#comment-16794025
]
Hudson commented on HBASE-20662:
Results for branch branch-2
[build #1753 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1753/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1753//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1753//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1753//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch,
> HBASE-20662.master.007.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
>
[jira] [Commented] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793955#comment-16793955 ] Andrew Purtell commented on HBASE-22034: branch-1 patch posted to reviewboard as https://reviews.apache.org/r/70220/ > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-22034: --- Attachment: HBASE-22034-branch-1.patch > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-22034: --- Status: Patch Available (was: Open) > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21895) Take more care on the error prone plugin and warnings
[
https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793899#comment-16793899
]
Kevin Risden edited comment on HBASE-21895 at 3/15/19 7:53 PM:
---
Put up a patch that upgrades errorprone and simplifies a bit (removes custom
rule holder since not used currently). Running with "mvn clean verify
-DskipTests -PerrorProne" yields at least one error:
{code:java}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile)
on project hbase-procedure: Compilation failure
[ERROR]
/Users/krisden/repos/apache/hbase/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java:[63,19]
error: [EqualsHashCode] Classes that override equals should also override
hashCode.
[ERROR] (see https://errorprone.info/bugpattern/EqualsHashCode)
{code}
I did not fix failures only made sure that errorProne profile ran. I could go
through and fix the errors identified just wanted to keep the initial patch
simple.
was (Author: risdenk):
Put up a patch that upgrades errorprone and simplifies a bit. Running with "mvn
clean verify -DskipTests -PerrorProne" yields at least one error:
{code:java}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile)
on project hbase-procedure: Compilation failure
[ERROR]
/Users/krisden/repos/apache/hbase/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java:[63,19]
error: [EqualsHashCode] Classes that override equals should also override
hashCode.
[ERROR] (see https://errorprone.info/bugpattern/EqualsHashCode)
{code}
I did not fix failures only made sure that errorProne profile ran.
> Take more care on the error prone plugin and warnings
> -
>
> Key: HBASE-21895
> URL: https://issues.apache.org/jira/browse/HBASE-21895
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Attachments: HBASE-21895.master.001.patch
>
>
> As shown in HBASE-21890 , the error prone warnings are truly useful.
> But now, the javac warnings section in the pre commit result is messed up, it
> always reports unrelated warnings. Need to take a look at it.
> And also, we should try out best to clean up the warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21895) Take more care on the error prone plugin and warnings
[ https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HBASE-21895: - Status: Patch Available (was: Open) > Take more care on the error prone plugin and warnings > - > > Key: HBASE-21895 > URL: https://issues.apache.org/jira/browse/HBASE-21895 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21895.master.001.patch > > > As shown in HBASE-21890 , the error prone warnings are truly useful. > But now, the javac warnings section in the pre commit result is messed up, it > always reports unrelated warnings. Need to take a look at it. > And also, we should try out best to clean up the warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21895) Take more care on the error prone plugin and warnings
[
https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793899#comment-16793899
]
Kevin Risden edited comment on HBASE-21895 at 3/15/19 7:55 PM:
---
Put up a patch that upgrades errorprone and simplifies a bit (removes custom
rule holder since not used currently). Running with "mvn clean verify
-DskipTests -PerrorProne" before the patch on master does not stop the build on
any errors. After the patch "mvn clean verify -DskipTests -PerrorProne" yields
at least one error:
{code:java}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile)
on project hbase-procedure: Compilation failure
[ERROR]
/Users/krisden/repos/apache/hbase/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java:[63,19]
error: [EqualsHashCode] Classes that override equals should also override
hashCode.
[ERROR] (see https://errorprone.info/bugpattern/EqualsHashCode)
{code}
I did not fix failures only made sure that errorProne profile ran. I could go
through and fix the errors identified just wanted to keep the initial patch
simple.
was (Author: risdenk):
Put up a patch that upgrades errorprone and simplifies a bit (removes custom
rule holder since not used currently). Running with "mvn clean verify
-DskipTests -PerrorProne" yields at least one error:
{code:java}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile)
on project hbase-procedure: Compilation failure
[ERROR]
/Users/krisden/repos/apache/hbase/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java:[63,19]
error: [EqualsHashCode] Classes that override equals should also override
hashCode.
[ERROR] (see https://errorprone.info/bugpattern/EqualsHashCode)
{code}
I did not fix failures only made sure that errorProne profile ran. I could go
through and fix the errors identified just wanted to keep the initial patch
simple.
> Take more care on the error prone plugin and warnings
> -
>
> Key: HBASE-21895
> URL: https://issues.apache.org/jira/browse/HBASE-21895
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Attachments: HBASE-21895.master.001.patch
>
>
> As shown in HBASE-21890 , the error prone warnings are truly useful.
> But now, the javac warnings section in the pre commit result is messed up, it
> always reports unrelated warnings. Need to take a look at it.
> And also, we should try out best to clean up the warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-22034: --- Summary: Backport HBASE-21401 and HBASE-22032 to branch-1 (was: Backport HBASE-21404 and HBASE-22032 to branch-1) > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22032) KeyValue validation should check for null byte array
[ https://issues.apache.org/jira/browse/HBASE-22032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-22032: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.2.1 2.1.4 3.0.0 Status: Resolved (was: Patch Available) > KeyValue validation should check for null byte array > > > Key: HBASE-22032 > URL: https://issues.apache.org/jira/browse/HBASE-22032 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.0.4, 2.1.3 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 3.0.0, 2.1.4, 2.2.1 > > Attachments: HBASE-22032.v01.patch, HBASE-22032.v02.patch, > HBASE-22032.v03.patch > > > HBASE-21401 added some nice validation checks to throw precise errors if a > KeyValue is constructed using invalid parameters. However it implicitly > assumes that the KeyValue buffer is not null. It should validate this > assumption and alert accordingly rather than throwing an NPE from an > unrelated check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[
https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793899#comment-16793899
]
Kevin Risden commented on HBASE-21895:
--
Put up a patch that upgrades errorprone and simplifies a bit. Running with "mvn
clean verify -DskipTests -PerrorProne" yields at least one error:
{code:java}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile)
on project hbase-procedure: Compilation failure
[ERROR]
/Users/krisden/repos/apache/hbase/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/util/DelayedUtil.java:[63,19]
error: [EqualsHashCode] Classes that override equals should also override
hashCode.
[ERROR] (see https://errorprone.info/bugpattern/EqualsHashCode)
{code}
I did not fix failures only made sure that errorProne profile ran.
> Take more care on the error prone plugin and warnings
> -
>
> Key: HBASE-21895
> URL: https://issues.apache.org/jira/browse/HBASE-21895
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Attachments: HBASE-21895.master.001.patch
>
>
> As shown in HBASE-21890 , the error prone warnings are truly useful.
> But now, the javac warnings section in the pre commit result is messed up, it
> always reports unrelated warnings. Need to take a look at it.
> And also, we should try out best to clean up the warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21895) Take more care on the error prone plugin and warnings
[ https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HBASE-21895: - Attachment: HBASE-21895.master.001.patch > Take more care on the error prone plugin and warnings > - > > Key: HBASE-21895 > URL: https://issues.apache.org/jira/browse/HBASE-21895 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21895.master.001.patch > > > As shown in HBASE-21890 , the error prone warnings are truly useful. > But now, the javac warnings section in the pre commit result is messed up, it > always reports unrelated warnings. Need to take a look at it. > And also, we should try out best to clean up the warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793876#comment-16793876
]
Nihal Jain commented on HBASE-20662:
Thanks for the review Josh. I willdefinitely submit a patch soon. And regarding
parameterized class, I had done that in [^HBASE-20662.master.002.patch]
itself. Later reverted it for later, citing too much code to review. Will work
on that since this one is commited now. :)
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch,
> HBASE-20662.master.007.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
>
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793851#comment-16793851
]
Josh Elser commented on HBASE-20662:
PS: this did not apply cleanly to branch-2.1, so I only applied it to
newer-branches than that.
I would be in support of this landing on earlier versions if you choose to
submit a patch for other branches.
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch,
> HBASE-20662.master.007.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
>
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20662:
---
Fix Version/s: 2.2.0
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch,
> HBASE-20662.master.007.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
>
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20662:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for continuing to push on this, Nihal. Good work!
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch,
> HBASE-20662.master.007.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[
https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793835#comment-16793835
]
Kevin Risden commented on HBASE-21895:
--
{quote}OK, seems not easy. I do not know why we use such a complicated way to
integrate error prone...
{quote}
[~Apache9] - Do you have more details? I took a quick peek and it looks like
its a maven profile to build with errorprone. I see a
"hbase-build-support/hbase-error-prone" which not entirely sure what it is for,
but the basic build integration part is in "hbase-build-configuration/pom.xml"
PS - Note I haven't looked at the HBase errorprone integration closely but
might be able to help.
> Take more care on the error prone plugin and warnings
> -
>
> Key: HBASE-21895
> URL: https://issues.apache.org/jira/browse/HBASE-21895
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
>
> As shown in HBASE-21890 , the error prone warnings are truly useful.
> But now, the javac warnings section in the pre commit result is messed up, it
> always reports unrelated warnings. Need to take a look at it.
> And also, we should try out best to clean up the warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[
https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793831#comment-16793831
]
Hadoop QA commented on HBASE-22052:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
57s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m
38s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
24s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
5s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 20m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
10s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 52s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}170m 35s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 3m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}249m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.rest.TestSecureRESTServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-22052 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12962616/HBASE-22052.branch-2.0.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile |
| uname | Linux a3b47b1e93b6 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / ac3e9eb7d4 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/16391/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16391/testReport/ |
| Max. process+thread count | 4627 (vs. ulimit of 1) |
| modules | C: hbase-zookeeper hbase-http hbase-server hbase-mapreduce
hbase-endpoint hbase-it hbase-rest . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16391/console
[jira] [Commented] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793827#comment-16793827 ] Josh Elser commented on HBASE-21926: Here's my +1 Excited to take this for a drive. Looks sufficient for a first wag to me! > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793802#comment-16793802 ] Andrew Purtell commented on HBASE-21926: [~busbey] If you want to make adoc changes, please just go ahead and take the latest version of the patch, make your changes on top, and attach the new omnibus patches. Happy to credit both of us upon commit. > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22034) Backport HBASE-21404 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793800#comment-16793800 ] Andrew Purtell commented on HBASE-22034: Let me take this as I am committing HBASE-22032 already > Backport HBASE-21404 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-22034) Backport HBASE-21404 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-22034: -- Assignee: Andrew Purtell (was: Geoffrey Jacoby) > Backport HBASE-21404 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22032) KeyValue validation should check for null byte array
[ https://issues.apache.org/jira/browse/HBASE-22032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793799#comment-16793799 ] Andrew Purtell commented on HBASE-22032: Going to commit this, then. > KeyValue validation should check for null byte array > > > Key: HBASE-22032 > URL: https://issues.apache.org/jira/browse/HBASE-22032 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.0.4, 2.1.3 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Attachments: HBASE-22032.v01.patch, HBASE-22032.v02.patch, > HBASE-22032.v03.patch > > > HBASE-21401 added some nice validation checks to throw precise errors if a > KeyValue is constructed using invalid parameters. However it implicitly > assumes that the KeyValue buffer is not null. It should validate this > assumption and alert accordingly rather than throwing an NPE from an > unrelated check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793798#comment-16793798 ] Andrew Purtell commented on HBASE-21926: I'm going to ignore further ImportOrder checkstyle warnings, because it doesn't seem to matter where the import is placed, there is still a warning. Tired of guessing what is the right answer. As this is the only checkstyle warning the patch looks good IMHO. The javac warning is for a file untouched by this patch, StaticUserWebFilter. The whitespace warning will be fixed on commit with git am --whitespace=fix. The patch has been confirmed to work with manual testing. What else needs to be done here? I think it is ready. I need a +1 please > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements
[ https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793793#comment-16793793 ] Andrew Purtell commented on HBASE-21991: +1 This should go into all branches where MetaMetrics were committed. > Fix MetaMetrics issues - [Race condition, Faulty remove logic], few > improvements > > > Key: HBASE-21991 > URL: https://issues.apache.org/jira/browse/HBASE-21991 > Project: HBase > Issue Type: Bug > Components: Coprocessors, metrics >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Attachments: hbase-21991.master.001.patch, > hbase-21991.master.002.patch, hbase-21991.master.003.patch, > hbase-21991.master.004.patch, hbase-21991.master.005.patch, > hbase-21991.master.006.patch > > > Here is a list of the issues related to the MetaMetrics implementation: > +*Bugs*+: > # [_Lossy counting for top-k_] *Faulty remove logic of non-eligible meters*: > Under certain conditions, we might end up storing/exposing all the meters > rather than top-k-ish > # MetaMetrics can throw NPE resulting in aborting of the RS because of a > *Race Condition*. > +*Improvements*+: > # With high number of regions in the cluster, exposure of metrics for each > region blows up the JMX from ~140 Kbs to 100+ Mbs depending on the number of > regions. It's better to use *lossy counting to maintain top-k for region > metrics* as well. > # As the lossy meters do not represent actual counts, I think, it'll be > better to *rename the meters to include "lossy" in the name*. It would be > more informative while monitoring the metrics and there would be less > confusion regarding actual counts to lossy counts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[ https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793792#comment-16793792 ] Andrew Purtell commented on HBASE-21895: How would a separate repo work? What's the point of error-prone integration if we can't run it on the code we are developing... which would be the main repo... If error-prone integration is unstable then we have to decide if the work to keep it working is worth the returns in the extra coverage it provides. It's a valid choice to remove it. I remember some of the original motivation is we would use it to implement our own static checks for various conventions and invariants. This has not happened. So, in my opinion, much of its possibility has been unrealized and its value is somewhat marginal. > Take more care on the error prone plugin and warnings > - > > Key: HBASE-21895 > URL: https://issues.apache.org/jira/browse/HBASE-21895 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > > As shown in HBASE-21890 , the error prone warnings are truly useful. > But now, the javac warnings section in the pre commit result is messed up, it > always reports unrelated warnings. Need to take a look at it. > And also, we should try out best to clean up the warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22051) Remove the hardcoded expect value in TestRSGroupsBasics
[ https://issues.apache.org/jira/browse/HBASE-22051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-22051: - Priority: Minor (was: Critical) Component/s: (was: master) test rsgroup Issue Type: Improvement (was: Bug) Summary: Remove the hardcoded expect value in TestRSGroupsBasics (was: Prohibit HMaster#getClusterSchema from returning null) > Remove the hardcoded expect value in TestRSGroupsBasics > --- > > Key: HBASE-22051 > URL: https://issues.apache.org/jira/browse/HBASE-22051 > Project: HBase > Issue Type: Improvement > Components: rsgroup, test >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 2:41 PM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I think it
might not be easy to verify it directly, but it is verified indirectly by
(1) {color:#59afe1}TestRSGroupsBasics#testBasicStartUp(){color} for the
condition where there is no servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupsBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region
servers into default rsgroup, as verified */
{code}
and
(2) {color:#59afe1}TestRSGroupsBasics#testClearDeadServers() {color}for the
condition where there are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I think it
might not be easy to verify it directly, but it is verified indirectly by
(1) {color:#59afe1}TestRSGroupBasics#testBasicStartUp(){color} for the
condition where there is no servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) {color:#59afe1}TestRSGroupBasics#testClearDeadServers() {color}for the
condition where there are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch,
> call_stack_getDefaultServers.png
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793656#comment-16793656
]
Josh Elser commented on HBASE-20662:
on v10
* It would be nice to use JUnit parameterization to enumerate all of the new
test classes you added instead of listing them all out by hand. I think you
could move these test methods to its own class to make that more simple.
{code}
- assertTrue("Expected exception message to contain the word '" +
policyToViolate.name()
- + "', but was " + msg,
-msg.contains(policyToViolate.name()));
+ LOG.info("Did not get the expected exception, will sleep and retry");
+ Thread.sleep(2000);
{code}
I'll fix this log message to include the exception text like the if statement
does (the above was the else statement in TestSpaceQuotas). This will help
debug this in the future, lest it breaks.
Going to run through the tests locally and commit if they come back green for
me.
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch,
> HBASE-20662.master.007.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
>
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793653#comment-16793653
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #27 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/27/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/27//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/27//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/27//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793646#comment-16793646
]
Josh Elser commented on HBASE-20662:
bq. Do you mean to say this patch exposes the afore-mentioned bug? If this is
the case I think it's better we get this one resolved first instead of fixing
some more stuff here
I don't think the problem described in 22012 is related to what you're doing
here, Nihal. Just took a long time to actually get a clear explanation as to
what was happening over there :)
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch,
> HBASE-20662.master.007.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793641#comment-16793641
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #136 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/136/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/136//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/136//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/136//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793639#comment-16793639 ] stack commented on HBASE-21935: --- Tests passed. Getting somewhere. Let me get HBASE-22029 in. Then I'll try an RC. > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.0.001.patch, > HBASE-21935.branch-2.0.002.patch, HBASE-21935.branch-2.0.003.patch, > HBASE-21935.branch-2.0.004.patch, HBASE-21935.branch-2.0.005.patch, > HBASE-21935.branch-2.0.006.patch, HBASE-21935.branch-2.0.007.patch, > HBASE-21935.branch-2.0.008.patch, HBASE-21935.branch-2.0.009.patch, > HBASE-21935.branch-2.1.001.patch, HBASE-21935.branch-2.1.002.patch, > HBASE-21935.branch-2.1.003.patch, HBASE-21935.branch-2.1.004.patch, > HBASE-21935.branch-2.1.005.patch, HBASE-21935.branch-2.1.006.patch, > HBASE-21935.branch-2.1.007.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793636#comment-16793636 ] stack commented on HBASE-22052: --- Retry. > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22052: -- Attachment: HBASE-22052.branch-2.0.002.patch > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22012) Space Quota: Policy state is getting changed from disable to Observance after sometime automatically.
[
https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793629#comment-16793629
]
Josh Elser commented on HBASE-22012:
{quote}
But in case of table is disabled, regions are closed and hence region size is
not getting calculated.
Now since region size is not available, as per behaviour, quota policy state
will change to 'in observant' and table will be enabled again.
{quote}
Ah, that makes much more sense.
> Space Quota: Policy state is getting changed from disable to Observance after
> sometime automatically.
> -
>
> Key: HBASE-22012
> URL: https://issues.apache.org/jira/browse/HBASE-22012
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Priority: Minor
>
> pace Quota: Policy state is getting changed from disable to Observance after
> sometime automatically.
> Steps:
> 1: Create a table with space quota policy as Disable
> 2: Put some data so that table state is in space quota violation
> 3: So observe that table state is in violation
> 4: Now wait for some time
> 5: Observe that after some time table state is changing to to Observance
> however table is still disabled
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[ https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-22012: --- Description: pace Quota: Policy state is getting changed from disable to Observance after sometime automatically. Steps: 1: Create a table with space quota policy as Disable 2: Put some data so that table state is in space quota violation 3: So observe that table state is in violation 4: Now wait for some time 5: Observe that after some time table state is changing to to Observance however table is still disabled edit (elserj): The table is automatically moved back from the violation state because of the code added that tried to ride over RITs. When a Region is not online (whether normally or abnormally), the RegionSizeReports are not sent from RS to Master. Eventually, enough Regions are not reported which dips below the acceptable threshold and we automatically move the table back to the "acceptable" space quota state (not in violation). We could skip this failsafe when we're checking for a quota that has the DisableTable violation policy. was: pace Quota: Policy state is getting changed from disable to Observance after sometime automatically. Steps: 1: Create a table with space quota policy as Disable 2: Put some data so that table state is in space quota violation 3: So observe that table state is in violation 4: Now wait for some time 5: Observe that after some time table state is changing to to Observance however table is still disabled > SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable > table > > > Key: HBASE-22012 > URL: https://issues.apache.org/jira/browse/HBASE-22012 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Priority: Minor > > pace Quota: Policy state is getting changed from disable to Observance after > sometime automatically. > Steps: > 1: Create a table with space quota policy as Disable > 2: Put some data so that table state is in space quota violation > 3: So observe that table state is in violation > 4: Now wait for some time > 5: Observe that after some time table state is changing to to Observance > however table is still disabled > edit (elserj): The table is automatically moved back from the violation state > because of the code added that tried to ride over RITs. When a Region is not > online (whether normally or abnormally), the RegionSizeReports are not sent > from RS to Master. Eventually, enough Regions are not reported which dips > below the acceptable threshold and we automatically move the table back to > the "acceptable" space quota state (not in violation). We could skip this > failsafe when we're checking for a quota that has the DisableTable violation > policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[ https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-22012: --- Priority: Major (was: Minor) > SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable > table > > > Key: HBASE-22012 > URL: https://issues.apache.org/jira/browse/HBASE-22012 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Priority: Major > > Space Quota: Policy state is getting changed from disable to Observance after > sometime automatically. > Steps: > 1: Create a table with space quota policy as Disable > 2: Put some data so that table state is in space quota violation > 3: So observe that table state is in violation > 4: Now wait for some time > 5: Observe that after some time table state is changing to to Observance > however table is still disabled > edit (elserj): The table is automatically moved back from the violation state > because of the code added that tried to ride over RITs. When a Region is not > online (whether normally or abnormally), the RegionSizeReports are not sent > from RS to Master. Eventually, enough Regions are not reported which dips > below the acceptable threshold and we automatically move the table back to > the "acceptable" space quota state (not in violation). We could skip this > failsafe when we're checking for a quota that has the DisableTable violation > policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[ https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-22012: --- Description: Space Quota: Policy state is getting changed from disable to Observance after sometime automatically. Steps: 1: Create a table with space quota policy as Disable 2: Put some data so that table state is in space quota violation 3: So observe that table state is in violation 4: Now wait for some time 5: Observe that after some time table state is changing to to Observance however table is still disabled edit (elserj): The table is automatically moved back from the violation state because of the code added that tried to ride over RITs. When a Region is not online (whether normally or abnormally), the RegionSizeReports are not sent from RS to Master. Eventually, enough Regions are not reported which dips below the acceptable threshold and we automatically move the table back to the "acceptable" space quota state (not in violation). We could skip this failsafe when we're checking for a quota that has the DisableTable violation policy. was: pace Quota: Policy state is getting changed from disable to Observance after sometime automatically. Steps: 1: Create a table with space quota policy as Disable 2: Put some data so that table state is in space quota violation 3: So observe that table state is in violation 4: Now wait for some time 5: Observe that after some time table state is changing to to Observance however table is still disabled edit (elserj): The table is automatically moved back from the violation state because of the code added that tried to ride over RITs. When a Region is not online (whether normally or abnormally), the RegionSizeReports are not sent from RS to Master. Eventually, enough Regions are not reported which dips below the acceptable threshold and we automatically move the table back to the "acceptable" space quota state (not in violation). We could skip this failsafe when we're checking for a quota that has the DisableTable violation policy. > SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable > table > > > Key: HBASE-22012 > URL: https://issues.apache.org/jira/browse/HBASE-22012 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Priority: Minor > > Space Quota: Policy state is getting changed from disable to Observance after > sometime automatically. > Steps: > 1: Create a table with space quota policy as Disable > 2: Put some data so that table state is in space quota violation > 3: So observe that table state is in violation > 4: Now wait for some time > 5: Observe that after some time table state is changing to to Observance > however table is still disabled > edit (elserj): The table is automatically moved back from the violation state > because of the code added that tried to ride over RITs. When a Region is not > online (whether normally or abnormally), the RegionSizeReports are not sent > from RS to Master. Eventually, enough Regions are not reported which dips > below the acceptable threshold and we automatically move the table back to > the "acceptable" space quota state (not in violation). We could skip this > failsafe when we're checking for a quota that has the DisableTable violation > policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[ https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-22012: --- Summary: SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table (was: Space Quota: Policy state is getting changed from disable to Observance after sometime automatically.) > SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable > table > > > Key: HBASE-22012 > URL: https://issues.apache.org/jira/browse/HBASE-22012 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Priority: Minor > > pace Quota: Policy state is getting changed from disable to Observance after > sometime automatically. > Steps: > 1: Create a table with space quota policy as Disable > 2: Put some data so that table state is in space quota violation > 3: So observe that table state is in violation > 4: Now wait for some time > 5: Observe that after some time table state is changing to to Observance > however table is still disabled -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22054) Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
Ajeet Rai created HBASE-22054:
-
Summary: Space Quota: Compaction is not working for super user in
case of NO_WRITES_COMPACTIONS
Key: HBASE-22054
URL: https://issues.apache.org/jira/browse/HBASE-22054
Project: HBase
Issue Type: Bug
Reporter: Ajeet Rai
Space Quota: Compaction is not working for super user. Compaction command is
issued successfully at client but actually compaction is not happening.
In debug log below message is printed:
as an active space quota violation policy disallows compaction.
Reference:
[https://lists.apache.org/thread.html/d09aa7abaacf1f0be9d59fa9260515ddc0c17ac0aba9cc0f2ac569bf@%3Cuser.hbase.apache.org%3E]
Actually in requestCompactionInternal method of CompactSplit class ,there is
no check for super user and compcations are disallowed
{noformat}
RegionServerSpaceQuotaManager spaceQuotaManager =
this.server.getRegionServerSpaceQuotaManager();
if (spaceQuotaManager != null &&
spaceQuotaManager.areCompactionsDisabled(region.getTableDescriptor().getTableName()))
{
String reason = "Ignoring compaction request for " + region +
" as an active space quota violation " + " policy disallows
compactions.";
tracker.notExecuted(store, reason);
completeTracker.completed(store);
LOG.debug(reason);
return;
}
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[
https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793561#comment-16793561
]
Hadoop QA commented on HBASE-22052:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
56s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
19s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
10s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
10s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 23s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}170m 4s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 3m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}238m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.rest.TestSecureRESTServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-22052 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12962576/HBASE-22052.branch-2.0.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile |
| uname | Linux 5cfd91124d1a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / ac3e9eb7d4 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/16388/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16388/testReport/ |
| Max. process+thread count | 5138 (vs. ulimit of 1) |
| modules | C: hbase-zookeeper hbase-http hbase-server hbase-mapreduce
hbase-endpoint hbase-it hbase-rest . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16388/console |
|
[jira] [Commented] (HBASE-21965) Fix failed split and merge transactions that have failed to roll back
[
https://issues.apache.org/jira/browse/HBASE-21965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793544#comment-16793544
]
Hadoop QA commented on HBASE-21965:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
43s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 43s{color} |
{color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 43s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
27s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
14s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}133m
29s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
2s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}192m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21965 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793501#comment-16793501
]
Hadoop QA commented on HBASE-22009:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-22009 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-22009 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16390/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch,
> call_stack_getDefaultServers.png
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:52 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I think it
might not be easy to verify it directly, but it is verified indirectly by
(1) {color:#59afe1}TestRSGroupBasics#testBasicStartUp(){color} for the
condition where there is no servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) {color:#59afe1}TestRSGroupBasics#testClearDeadServers() {color}for the
condition where there are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) {color:#59afe1}TestRSGroupBasics#testBasicStartUp(){color} for the
condition where there is no servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) {color:#59afe1}TestRSGroupBasics#testClearDeadServers() {color}for the
condition where there are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22009:
-
Attachment: call_stack_getDefaultServers.png
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch,
> call_stack_getDefaultServers.png
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:43 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:50 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) {color:#59afe1}TestRSGroupBasics#testBasicStartUp(){color} for the
condition where there is no servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) {color:#59afe1}TestRSGroupBasics#testClearDeadServers() {color}for the
condition where there are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) +TestRSGroupBasics#testBasicStartUp()+ for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) +TestRSGroupBasics#testClearDeadServers()+ for the condition where there
are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:49 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) +TestRSGroupBasics#testBasicStartUp()+ for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) +TestRSGroupBasics#testClearDeadServers()+ for the condition where there
are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) *TestRSGroupBasics#testBasicStartUp()* for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) *TestRSGroupBasics#testClearDeadServers() *for the condition where there
are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:49 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) *TestRSGroupBasics#testBasicStartUp()* for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) *TestRSGroupBasics#testClearDeadServers() *for the condition where there
are some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup, as verified */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left, as verified*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:41 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeps 1 server in default rsgroup) and then stop 1
region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:40 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartup() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:36 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() is deeply invoked when HMaster starts (I uploaded the call
stack just for your information), so I am not sure if we could verify it
directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup()
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls he function modified.
getDefaultServers() is deeply invoked when HMaster starts (I uploaded the call
stack just for your information), so I am not sure if we could verify it
directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup()
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:42 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeping 1 server in default rsgroup) and then stop 1
region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartUp() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup into a
specific rsgroup (while keeps 1 server in default rsgroup) and then stop 1
region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:39 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
(1) TestRSGroupBasics#testBasicStartup() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
(2) TestRSGroupBasics#testClearDeadServers() for the condition where there are
some servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
1. TestRSGroupBasics#testBasicStartup() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers() for the condition where there are some
servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:39 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
1. TestRSGroupBasics#testBasicStartup() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers() for the condition where there are some
servers in other groups as well as default group
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:38 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup() for the condition where there is no
servers in other groups (all are in default group)
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup()
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li edited comment on HBASE-22009 at 3/15/19 9:37 AM:
---
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() (addressed by this JIRA) is deeply invoked when HMaster
starts (I uploaded the call stack just for your information), so I am not sure
if we could verify it directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup()
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
was (Author: water):
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls the function modified.
getDefaultServers() is deeply invoked when HMaster starts (I uploaded the call
stack just for your information), so I am not sure if we could verify it
directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup()
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793491#comment-16793491
]
Xiang Li commented on HBASE-22009:
--
Hi [~xucang], sorry, my bad. Have to correct my comments... It is : almost each
test in hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup -verifies-
calls he function modified.
getDefaultServers() is deeply invoked when HMaster starts (I uploaded the call
stack just for your information), so I am not sure if we could verify it
directly, but it is verified indirectly by
TestRSGroupBasics#testBasicStartup()
{code:java}
assertEquals(4, defaultInfo.getServers().size());
/* TestRSGroupBase#setUpTestBeforeClass() puts NUM_SLAVES_BASE=4 region servers
into default rsgroup */
{code}
and
TestRSGroupBasics#testClearDeadServers()
{code}
assertEquals(2, newGroupServers.size());
/* testClearDeadServers() moves 3 region servers from default rsgroup a
specific rsgroup and stop 1 region server, making 2 left*/
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops. And for each server, listRSGroups()
> allocates a new LinkedList and calls Map#values(), both of which are very
> heavy operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22053) zookeeper URL links in documentation are failing with 404
[ https://issues.apache.org/jira/browse/HBASE-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subrat Mishra updated HBASE-22053: -- Attachment: HBASE-22053.master.001.patch > zookeeper URL links in documentation are failing with 404 > - > > Key: HBASE-22053 > URL: https://issues.apache.org/jira/browse/HBASE-22053 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Subrat Mishra >Assignee: Subrat Mishra >Priority: Minor > Attachments: HBASE-22053.master.001.patch > > > zookeeper URL changed from hadoop.apache.org to zookeeper.apache.org > E.g: Below URL failing with 404 > http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions > should be changed to: > https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkSessions -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22049) getReopenStatus() didn't skip counting split parent region
[
https://issues.apache.org/jira/browse/HBASE-22049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793432#comment-16793432
]
Jingyun Tian commented on HBASE-22049:
--
[~Apache9] Failed test is not related to my patch. But the root cause is when
master restart, the flag of isSplit cannot be restore. Maybe we should try to
fix this?
> getReopenStatus() didn't skip counting split parent region
> --
>
> Key: HBASE-22049
> URL: https://issues.apache.org/jira/browse/HBASE-22049
> Project: HBase
> Issue Type: Bug
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Attachments: HBASE-22049.master.001.patch
>
>
> After we modify some attributes of table, hbaseAdmin will getAlterStatus to
> check if all region's attributes updated. It will skip opened region and
> split region as the following code shows.
> {code}
> for (RegionState regionState: states) {
> if (!regionState.isOpened() && !regionState.isSplit()) {
> ritCount++;
> }
> }
> {code}
> But since now the split procedure is to unassign the split parent region,
> thus the state is CLOSED， and the check will hang there until timeout.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21965) Fix failed split and merge transactions that have failed to roll back
[ https://issues.apache.org/jira/browse/HBASE-21965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21965: - Status: Patch Available (was: Reopened) > Fix failed split and merge transactions that have failed to roll back > - > > Key: HBASE-21965 > URL: https://issues.apache.org/jira/browse/HBASE-21965 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-21965.master.001.patch > > > Make HBCK2 be able to fix failed split and merge transactions that have > failed to roll back. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21965) Fix failed split and merge transactions that have failed to roll back
[ https://issues.apache.org/jira/browse/HBASE-21965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21965: - Attachment: HBASE-21965.master.001.patch > Fix failed split and merge transactions that have failed to roll back > - > > Key: HBASE-21965 > URL: https://issues.apache.org/jira/browse/HBASE-21965 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-21965.master.001.patch > > > Make HBCK2 be able to fix failed split and merge transactions that have > failed to roll back. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793418#comment-16793418 ] stack commented on HBASE-21935: --- Will try it after get over this hurdle. 2.0.009 runs site in its own invocation. This seems to work (with HBASE-22029 fix in place). Tests running. > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.0.001.patch, > HBASE-21935.branch-2.0.002.patch, HBASE-21935.branch-2.0.003.patch, > HBASE-21935.branch-2.0.004.patch, HBASE-21935.branch-2.0.005.patch, > HBASE-21935.branch-2.0.006.patch, HBASE-21935.branch-2.0.007.patch, > HBASE-21935.branch-2.0.008.patch, HBASE-21935.branch-2.0.009.patch, > HBASE-21935.branch-2.1.001.patch, HBASE-21935.branch-2.1.002.patch, > HBASE-21935.branch-2.1.003.patch, HBASE-21935.branch-2.1.004.patch, > HBASE-21935.branch-2.1.005.patch, HBASE-21935.branch-2.1.006.patch, > HBASE-21935.branch-2.1.007.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
