[jira] [Commented] (HBASE-18275) Formatting and grammar mistakes in schemadoc chapter
[ https://issues.apache.org/jira/browse/HBASE-18275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065950#comment-16065950 ] Hudson commented on HBASE-18275: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3275 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3275/]) HBASE-18275 Formatting and grammar mistakes in schemadoc chapter (Artem (tedyu: rev b3130435424cad356d32a8a92bd2ef60b4d50f00) * (edit) src/main/asciidoc/_chapters/schema_design.adoc > Formatting and grammar mistakes in schemadoc chapter > > > Key: HBASE-18275 > URL: https://issues.apache.org/jira/browse/HBASE-18275 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Trivial > Labels: newbie > Fix For: 3.0.0 > > Attachments: HBASE-18275-0.patch > > > a grammatical error and formatting error in schema design doc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18284) Update hbasecon asia logo on home page of hbase.apache.org
[ https://issues.apache.org/jira/browse/HBASE-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065951#comment-16065951 ] Hudson commented on HBASE-18284: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3275 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3275/]) HBASE-18284 Update hbasecon asia logo on home page of hbase.apache.org (stack: rev 8bb4d15ba11089879711fe089a985c60df95f6f1) * (edit) src/main/site/resources/images/hbaseconasia2017.png > Update hbasecon asia logo on home page of hbase.apache.org > -- > > Key: HBASE-18284 > URL: https://issues.apache.org/jira/browse/HBASE-18284 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17908) Upgrade guava
[
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065918#comment-16065918
]
Hadoop QA commented on HBASE-17908:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
8s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:red}-1{color} | {color:red} @author {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 5 @author tags which the
community has agreed to not allow in code contributions. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 209 new or modified
test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 5m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-testing-util hbase-assembly . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
35s{color} | {color:red} hbase-common in master has 2 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
21s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
54s{color} | {color:red} hbase-client in master has 4 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
54s{color} | {color:red} hbase-server in master has 10 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
26s{color} | {color:red} hbase-examples in master has 2 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
39s{color} | {color:red} hbase-rest in master has 3 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
9s{color} | {color:red} hbase-hadoop2-compat in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
24s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
15s{color} | {color:red} hbase-endpoint in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
37s{color} | {color:red} hbase-spark in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 33s{color}
| {color:red} hbase-protocol-shaded generated 4 new + 6 unchanged - 2 fixed =
10 total (was 8) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 57s{color}
| {color:red} root generated 4 new + 38 unchanged - 2 fixed = 42 total (was 40)
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 5m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
26s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 13s{color} | {color:green} Patch does not
[jira] [Resolved] (HBASE-18284) Update hbasecon asia logo on home page of hbase.apache.org
[ https://issues.apache.org/jira/browse/HBASE-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-18284. --- Resolution: Fixed Assignee: stack Fix Version/s: 3.0.0 Pushed new logo made by Bijieshan > Update hbasecon asia logo on home page of hbase.apache.org > -- > > Key: HBASE-18284 > URL: https://issues.apache.org/jira/browse/HBASE-18284 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18284) Update hbasecon asia logo on home page of hbase.apache.org
stack created HBASE-18284: - Summary: Update hbasecon asia logo on home page of hbase.apache.org Key: HBASE-18284 URL: https://issues.apache.org/jira/browse/HBASE-18284 Project: HBase Issue Type: Task Reporter: stack -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[
https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065903#comment-16065903
]
Hadoop QA commented on HBASE-18273:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
9s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
8s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
62m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18273 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874797/HBASE-18273.1.patch |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 2eb6dfb2ce14 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b313043 |
| shellcheck | v0.4.6 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7371/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> hbase_rotate_log in hbase-daemon.sh script not working for some JDK
> ---
>
> Key: HBASE-18273
> URL: https://issues.apache.org/jira/browse/HBASE-18273
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch,
> HBASE-18273.2.patch
>
>
> When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate
> GC logs.
> the code looks like this,
> if [ -f "$log" ]; then # rotate logs
> while [ $num -gt 1 ]; do
> prev=`expr $num - 1`
> [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num"
> num=$prev
> done
> But, some version JDK will add a suffix (.0) to the gc file, like
> hbase-xxx.gc.0, rather than hbase-xxx.gc.
> So I add a check before rotate,
> if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0
> if [ -f "$log.0" ]; then
> mv -f "$log.0" "$log";
> fi
> fi
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17893) Allow HBase to build against Hadoop 2.8.0
[
https://issues.apache.org/jira/browse/HBASE-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-17893:
--
Attachment: 17893-1.3-backport-rebased.txt
Thanks [~busbey].
Bit weird then to name it 2.8.0 then :) If it's not ready for production it's
ALPHA, BETA, or RC.
In any case, here's a rebased patch. In case anything wants to compile HBase
1.3 with Hadoop 2.8.0.
> Allow HBase to build against Hadoop 2.8.0
> -
>
> Key: HBASE-17893
> URL: https://issues.apache.org/jira/browse/HBASE-17893
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.5
>Reporter: Lars Hofhansl
> Attachments: 17883-1.2-BROKEN.txt, 17893-1.3-backport-rebased.txt,
> 17893-1.3-backport.txt
>
>
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default)
> on project hbase-assembly: Error rendering velocity resource. Error invoking
> method 'get(java.lang.Integer)' in java.util.ArrayList at
> META-INF/LICENSE.vm[line 1671, column 8]: InvocationTargetException: Index:
> 0, Size: 0 -> [Help 1]
> {code}
> The in the generated LICENSE.
> {code}
> This product includes Nimbus JOSE+JWT licensed under the The Apache Software
> License, Version 2.0.
> ${dep.licenses[0].comments}
> Please check this License for acceptability here:
> https://www.apache.org/legal/resolved
> If it is okay, then update the list named 'non_aggregate_fine' in the
> LICENSE.vm file.
> If it isn't okay, then revert the change that added the dependency.
> More info on the dependency:
> com.nimbusds
> nimbus-jose-jwt
> 3.9
> maven central search
> g:com.nimbusds AND a:nimbus-jose-jwt AND v:3.9
> project website
> https://bitbucket.org/connect2id/nimbus-jose-jwt
> project source
> https://bitbucket.org/connect2id/nimbus-jose-jwt
> {code}
> Maybe the problem is just that it says: Apache _Software_ License
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[
https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065882#comment-16065882
]
Hadoop QA commented on HBASE-18273:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
3s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 49s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18273 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874799/HBASE-18273.2.patch |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 3b078ec6d7d8 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b313043 |
| shellcheck | v0.4.6 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7372/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> hbase_rotate_log in hbase-daemon.sh script not working for some JDK
> ---
>
> Key: HBASE-18273
> URL: https://issues.apache.org/jira/browse/HBASE-18273
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch,
> HBASE-18273.2.patch
>
>
> When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate
> GC logs.
> the code looks like this,
> if [ -f "$log" ]; then # rotate logs
> while [ $num -gt 1 ]; do
> prev=`expr $num - 1`
> [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num"
> num=$prev
> done
> But, some version JDK will add a suffix (.0) to the gc file, like
> hbase-xxx.gc.0, rather than hbase-xxx.gc.
> So I add a check before rotate,
> if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0
> if [ -f "$log.0" ]; then
> mv -f "$log.0" "$log";
> fi
> fi
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17931) Assign system tables to servers with highest version
[
https://issues.apache.org/jira/browse/HBASE-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065874#comment-16065874
]
Hadoop QA commented on HBASE-17931:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
53s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} hbase-common in branch-1 failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} hbase-server in branch-1 failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} hbase-server in branch-1 failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} hbase-common in the patch passed with JDK
v1.8.0_131. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} hbase-server-jdk1.8.0_131 with JDK v1.8.0_131
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} hbase-common in the patch passed with JDK
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} hbase-server-jdk1.7.0_131 with JDK v1.7.0_131
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 38s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} hbase-common in the patch passed with JDK
v1.8.0_131. {color} |
|
[jira] [Updated] (HBASE-18182) Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start error
[
https://issues.apache.org/jira/browse/HBASE-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fangyuan Deng updated HBASE-18182:
--
Status: Patch Available (was: Open)
> Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start
> error
>
>
> Key: HBASE-18182
> URL: https://issues.apache.org/jira/browse/HBASE-18182
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.6, 1.3.1, 1.1.6, 2.0.0
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 2.1.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-18182.0.patch
>
>
> when we set in hbase-env.sh to use G1GC
> HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:+UseG1GC -Xms96g
> -Xmx96g -XX:MaxGCPauseMillis=100"
> then run ./hbase-daemon.sh start regionserver
> Conflicting collector combinations in option list; please refer to the
> release notes for the combinations allowed
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> because the HBASE_OPTS default use CMS and is conflicted with
> HBASE_REGIONSERVER_OPTS
> so I add a patch to modify bin/hbase like this
> elif [ "$COMMAND" = "regionserver" ] ; then
> CLASS='org.apache.hadoop.hbase.regionserver.HRegionServer'
> if [ "$1" != "stop" ] ; then
> HBASE_OPTS="$HBASE_OPTS $HBASE_REGIONSERVER_OPTS"
> if [[ $HBASE_REGIONSERVER_OPTS == *UseG1GC* ]] ; then
> HBASE_OPTS=${HBASE_OPTS/'-XX:+UseConcMarkSweepGC'/''}
> fi
> fi
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Work stopped] (HBASE-18182) Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start error
[
https://issues.apache.org/jira/browse/HBASE-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-18182 stopped by Fangyuan Deng.
-
> Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start
> error
>
>
> Key: HBASE-18182
> URL: https://issues.apache.org/jira/browse/HBASE-18182
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.6
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 2.1.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-18182.0.patch
>
>
> when we set in hbase-env.sh to use G1GC
> HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:+UseG1GC -Xms96g
> -Xmx96g -XX:MaxGCPauseMillis=100"
> then run ./hbase-daemon.sh start regionserver
> Conflicting collector combinations in option list; please refer to the
> release notes for the combinations allowed
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> because the HBASE_OPTS default use CMS and is conflicted with
> HBASE_REGIONSERVER_OPTS
> so I add a patch to modify bin/hbase like this
> elif [ "$COMMAND" = "regionserver" ] ; then
> CLASS='org.apache.hadoop.hbase.regionserver.HRegionServer'
> if [ "$1" != "stop" ] ; then
> HBASE_OPTS="$HBASE_OPTS $HBASE_REGIONSERVER_OPTS"
> if [[ $HBASE_REGIONSERVER_OPTS == *UseG1GC* ]] ; then
> HBASE_OPTS=${HBASE_OPTS/'-XX:+UseConcMarkSweepGC'/''}
> fi
> fi
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18182) Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start error
[
https://issues.apache.org/jira/browse/HBASE-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fangyuan Deng updated HBASE-18182:
--
Status: In Progress (was: Patch Available)
> Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start
> error
>
>
> Key: HBASE-18182
> URL: https://issues.apache.org/jira/browse/HBASE-18182
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.2.6, 1.3.1, 1.1.6, 2.0.0
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 2.1.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-18182.0.patch
>
>
> when we set in hbase-env.sh to use G1GC
> HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:+UseG1GC -Xms96g
> -Xmx96g -XX:MaxGCPauseMillis=100"
> then run ./hbase-daemon.sh start regionserver
> Conflicting collector combinations in option list; please refer to the
> release notes for the combinations allowed
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> because the HBASE_OPTS default use CMS and is conflicted with
> HBASE_REGIONSERVER_OPTS
> so I add a patch to modify bin/hbase like this
> elif [ "$COMMAND" = "regionserver" ] ; then
> CLASS='org.apache.hadoop.hbase.regionserver.HRegionServer'
> if [ "$1" != "stop" ] ; then
> HBASE_OPTS="$HBASE_OPTS $HBASE_REGIONSERVER_OPTS"
> if [[ $HBASE_REGIONSERVER_OPTS == *UseG1GC* ]] ; then
> HBASE_OPTS=${HBASE_OPTS/'-XX:+UseConcMarkSweepGC'/''}
> fi
> fi
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18182) Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start error
[
https://issues.apache.org/jira/browse/HBASE-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fangyuan Deng reassigned HBASE-18182:
-
Assignee: Fangyuan Deng
> Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start
> error
>
>
> Key: HBASE-18182
> URL: https://issues.apache.org/jira/browse/HBASE-18182
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.6
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 2.1.0, 1.3.2, 1.2.7
>
> Attachments: HBASE-18182.0.patch
>
>
> when we set in hbase-env.sh to use G1GC
> HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:+UseG1GC -Xms96g
> -Xmx96g -XX:MaxGCPauseMillis=100"
> then run ./hbase-daemon.sh start regionserver
> Conflicting collector combinations in option list; please refer to the
> release notes for the combinations allowed
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> because the HBASE_OPTS default use CMS and is conflicted with
> HBASE_REGIONSERVER_OPTS
> so I add a patch to modify bin/hbase like this
> elif [ "$COMMAND" = "regionserver" ] ; then
> CLASS='org.apache.hadoop.hbase.regionserver.HRegionServer'
> if [ "$1" != "stop" ] ; then
> HBASE_OPTS="$HBASE_OPTS $HBASE_REGIONSERVER_OPTS"
> if [[ $HBASE_REGIONSERVER_OPTS == *UseG1GC* ]] ; then
> HBASE_OPTS=${HBASE_OPTS/'-XX:+UseConcMarkSweepGC'/''}
> fi
> fi
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[ https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065868#comment-16065868 ] Ted Yu commented on HBASE-18281: Please squash the two commits into one patch. > Performance update in StoreFileWriter.java for string replacement > - > > Key: HBASE-18281 > URL: https://issues.apache.org/jira/browse/HBASE-18281 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Ben Epstein >Priority: Trivial > Attachments: 0001-HBASE-18281-fixed-performance-using-Pattern.patch, > 0002-created-static-Pattern-for-performance-upgrade.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Change the replaceAll() function using regex with a static Pattern for > performance upgrade -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangyuan Deng reassigned HBASE-18273: - Assignee: Fangyuan Deng > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng >Assignee: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch, > HBASE-18273.2.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a suffix (.0) to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangyuan Deng updated HBASE-18273: -- Attachment: HBASE-18273.2.patch > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch, > HBASE-18273.2.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a suffix (.0) to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065867#comment-16065867 ] Fangyuan Deng commented on HBASE-18273: --- thx Dima, like this ? [ -f "$log.0" ] && mv -f "$log.0" "$log" #for some jdk, gc log has a suffix 0 > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch, > HBASE-18273.2.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a suffix (.0) to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[
https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065861#comment-16065861
]
Hadoop QA commented on HBASE-18281:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
35s{color} | {color:red} hbase-server in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 21s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}146m 10s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
39s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}315m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18281 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874758/0002-created-static-Pattern-for-performance-upgrade.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux fec290f804f3 3.13.0-119-generic #166-Ubuntu SMP Wed
[jira] [Commented] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065860#comment-16065860 ] Dima Spivak commented on HBASE-18273: - Why not just check for the file with the suffixed name alone? No need to have it be a compound statement. > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a suffix (.0) to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangyuan Deng updated HBASE-18273: -- Description: When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate GC logs. the code looks like this, if [ -f "$log" ]; then # rotate logs while [ $num -gt 1 ]; do prev=`expr $num - 1` [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" num=$prev done But, some version JDK will add a suffix (.0) to the gc file, like hbase-xxx.gc.0, rather than hbase-xxx.gc. So I add a check before rotate, if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 if [ -f "$log.0" ]; then mv -f "$log.0" "$log"; fi fi was: When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate GC logs. the code looks like this, if [ -f "$log" ]; then # rotate logs while [ $num -gt 1 ]; do prev=`expr $num - 1` [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" num=$prev done But, some version JDK will add a postfix .0 to the gc file, like hbase-xxx.gc.0, rather than hbase-xxx.gc. So I add a check before rotate, if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 if [ -f "$log.0" ]; then mv -f "$log.0" "$log"; fi fi > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a suffix (.0) to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18280) Manual Array Copy Cleanup: Automated
[ https://issues.apache.org/jira/browse/HBASE-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065859#comment-16065859 ] Mike Drob commented on HBASE-18280: --- The changes in HBaseAdmin and Bytes could also use Arrays.copyOf to encapsulate the array create logic as well. > Manual Array Copy Cleanup: Automated > > > Key: HBASE-18280 > URL: https://issues.apache.org/jira/browse/HBASE-18280 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > Attachments: 0001-HBASE-18280-Remove-Manual-Array-Copies.patch > > > Remove Manual Array Copies: Style Issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065857#comment-16065857 ] Fangyuan Deng commented on HBASE-18273: --- simplify the check, if [ ! -f "$log" ] && [ -f "$log.0" ]; then #for some jdk, gc log has a postfix 0 mv -f "$log.0" "$log"; fi > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a postfix .0 to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangyuan Deng updated HBASE-18273: -- Attachment: HBASE-18273.1.patch > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a postfix .0 to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065857#comment-16065857 ] Fangyuan Deng edited comment on HBASE-18273 at 6/28/17 3:01 AM: simplify the check, if [ ! -f "$log" ] && [ -f "$log.0" ]; then #for some jdk, gc log has a suffix 0 mv -f "$log.0" "$log"; fi was (Author: whisper_deng): simplify the check, if [ ! -f "$log" ] && [ -f "$log.0" ]; then #for some jdk, gc log has a postfix 0 mv -f "$log.0" "$log"; fi > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug > Components: hbase >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a postfix .0 to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18229) create new Async Split API to embrace AM v2
[
https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065845#comment-16065845
]
Hadoop QA commented on HBASE-18229:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
57s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
45s{color} | {color:red} hbase-client in master has 4 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
25s{color} | {color:red} hbase-server in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 88m 36s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterProcedureWalLease |
| Timed out junit tests |
org.apache.hadoop.hbase.backup.TestIncrementalBackupDeleteTable |
| |
[jira] [Updated] (HBASE-18275) Formatting and grammar mistakes in schemadoc chapter
[ https://issues.apache.org/jira/browse/HBASE-18275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18275: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 2.0.0-alpha-2) Status: Resolved (was: Patch Available) Thanks for the patch, Artem. > Formatting and grammar mistakes in schemadoc chapter > > > Key: HBASE-18275 > URL: https://issues.apache.org/jira/browse/HBASE-18275 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Trivial > Labels: newbie > Fix For: 3.0.0 > > Attachments: HBASE-18275-0.patch > > > a grammatical error and formatting error in schema design doc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18275) Formatting and grammar mistakes in schemadoc chapter
[ https://issues.apache.org/jira/browse/HBASE-18275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18275: --- Summary: Formatting and grammar mistakes in schemadoc chapter (was: formatting and grammar mistakes in schemadoc chapter) > Formatting and grammar mistakes in schemadoc chapter > > > Key: HBASE-18275 > URL: https://issues.apache.org/jira/browse/HBASE-18275 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18275-0.patch > > > a grammatical error and formatting error in schema design doc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18268) Eliminate the findbugs warnings for hbase-client
[ https://issues.apache.org/jira/browse/HBASE-18268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065816#comment-16065816 ] Chia-Ping Tsai commented on HBASE-18268: bq. Can we only add the spotbugs dependency when jdk8 is active? Do we plan to upgrade the java for branch-1? If not, IMHO we shouldn't add the spotbugs which may be never used. > Eliminate the findbugs warnings for hbase-client > > > Key: HBASE-18268 > URL: https://issues.apache.org/jira/browse/HBASE-18268 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18268.branch-1.v0.patch, HBASE-18268.v0.patch, > HBASE-18268.v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18275) formatting and grammar mistakes in schemadoc chapter
[ https://issues.apache.org/jira/browse/HBASE-18275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065802#comment-16065802 ] Artem Ervits commented on HBASE-18275: -- failing test is unrelated, most likely it's due to https://issues.apache.org/jira/browse/HBASE-18244 > formatting and grammar mistakes in schemadoc chapter > > > Key: HBASE-18275 > URL: https://issues.apache.org/jira/browse/HBASE-18275 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18275-0.patch > > > a grammatical error and formatting error in schema design doc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-16542) Skip full backup in selected backup tests
[ https://issues.apache.org/jira/browse/HBASE-16542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-16542. Resolution: Later > Skip full backup in selected backup tests > - > > Key: HBASE-16542 > URL: https://issues.apache.org/jira/browse/HBASE-16542 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Priority: Minor > Labels: backup > Fix For: HBASE-7912 > > > Since automatic mode is always used (HBASE-16037), some tests take longer > time to run: > 1. restore full backup > 2. restore incremental backup > Action 2 would execute action 1 again. > We can selectively skip full backup in backup / restore tests where > incremental backup is involved. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-15103) hadoopcheck test should provide diff file showing what's new
[ https://issues.apache.org/jira/browse/HBASE-15103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-15103. Resolution: Later > hadoopcheck test should provide diff file showing what's new > > > Key: HBASE-15103 > URL: https://issues.apache.org/jira/browse/HBASE-15103 > Project: HBase > Issue Type: Test > Components: test >Reporter: Ted Yu >Priority: Minor > Labels: test > > Currently developer has to go to 'build artifacts' folder to read output from > hadoopcheck test. > e.g. > https://builds.apache.org/job/PreCommit-HBASE-Build/98/artifact/patchprocess/patch-javac-2.6.1.txt > hadoopcheck test should provide diff file showing what exactly is new > Thanks to Sean for offline discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18052) Add document for async admin
[ https://issues.apache.org/jira/browse/HBASE-18052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065778#comment-16065778 ] Guanghao Zhang commented on HBASE-18052: Open HBASE-18283 for it. And do we need provide setRpcTimeout and setOperationTimeout for user? It is easy for user to use admin operations. So need a AdminBuilder, too? > Add document for async admin > > > Key: HBASE-18052 > URL: https://issues.apache.org/jira/browse/HBASE-18052 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18052.master.001.patch, > HBASE-18052.master.002.patch, HBASE-18052.master.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18283) Provide a construct method which accept a thread pool for AsyncAdmin
Guanghao Zhang created HBASE-18283: -- Summary: Provide a construct method which accept a thread pool for AsyncAdmin Key: HBASE-18283 URL: https://issues.apache.org/jira/browse/HBASE-18283 Project: HBase Issue Type: Sub-task Affects Versions: 3.0.0, 2.0.0-alpha-2 Reporter: Guanghao Zhang Assignee: Guanghao Zhang Similar to AsyncTable, provide a construct method which accept a thread pool for normal user. User need provide a thread pool to get a AsyncAdmin instance. Then the callbacks registered to the returned CompletableFuture can be executed in that thread pool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18278) [AMv2] Enable and fix uni test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[
https://issues.apache.org/jira/browse/HBASE-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065766#comment-16065766
]
Hadoop QA commented on HBASE-18278:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
27s{color} | {color:red} hbase-server in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
62m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}166m 42s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}260m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestPerColumnFamilyFlush |
| | hadoop.hbase.replication.TestReplicationSmallTests |
| Timed out junit tests | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.util.TestHBaseFsckEncryption |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded |
| | org.apache.hadoop.hbase.util.TestRegionMover |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18278 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874742/HBASE-18278.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ed9b5a92f17d 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8da6f06 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7364/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/7364/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Updated] (HBASE-17931) Assign system tables to servers with highest version
[ https://issues.apache.org/jira/browse/HBASE-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-17931: -- Attachment: HBASE-17931.branch-1.v05.patch > Assign system tables to servers with highest version > > > Key: HBASE-17931 > URL: https://issues.apache.org/jira/browse/HBASE-17931 > Project: HBase > Issue Type: Bug > Components: Region Assignment, scan >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Blocker > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17931.branch-1.v01.patch, > HBASE-17931.branch-1.v02.patch, HBASE-17931.branch-1.v03.patch, > HBASE-17931.branch-1.v04.patch, HBASE-17931.branch-1.v04.patch, > HBASE-17931.branch-1.v05.patch, HBASE-17931.branch-1.v05.patch, > HBASE-17931.v01.patch, HBASE-17931.v02.patch, HBASE-17931.v03.patch > > > In branch-1 and master we have some improvement and new features on scanning > which is not compatible. > A client of old version to a server of new version is compatible (must be a > bug if not, maybe need some test?). > A client of new version may not be able to read from a server of old version > correctly (because of scan limit, moreResults flag, etc), which is ok for > major/minor upgrade and we can tell users to upgrade server before upgrading > client. But RS also use scan to read meta. If meta table is in RS of old > version, all RSs of new version may have trouble while scanning meta table. > So we should make sure meta table always in servers of new version. Force > meta table in Master and upgrade Master first, or assign meta table in region > servers with latest version? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18052) Add document for async admin
[ https://issues.apache.org/jira/browse/HBASE-18052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065754#comment-16065754 ] Duo Zhang commented on HBASE-18052: --- [~zghaobac] Let's open a new issue to address the problem for creating AsyncAdmin first? I think the documentation should go last. Thanks. > Add document for async admin > > > Key: HBASE-18052 > URL: https://issues.apache.org/jira/browse/HBASE-18052 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18052.master.001.patch, > HBASE-18052.master.002.patch, HBASE-18052.master.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18164) Much faster locality cost function and candidate generator
[ https://issues.apache.org/jira/browse/HBASE-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065691#comment-16065691 ] Hudson commented on HBASE-18164: SUCCESS: Integrated in Jenkins build HBase-2.0 #121 (See [https://builds.apache.org/job/HBase-2.0/121/]) Revert "HBASE-18164 Fast locality computation in balancer - addendum (busbey: rev 0ad8f26662412fd9a7e61fa00e13ab29cb1e203d) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java HBASE-18164 Fast locality computation in balancer - addendum handles NaN (busbey: rev 812b38332557b497560655005c839a92543a80ec) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java > Much faster locality cost function and candidate generator > -- > > Key: HBASE-18164 > URL: https://issues.apache.org/jira/browse/HBASE-18164 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Critical > Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-2 > > Attachments: 18164.branch-1.addendum.txt, HBASE-18164-00.patch, > HBASE-18164-01.patch, HBASE-18164-02.patch, HBASE-18164-04.patch, > HBASE-18164-05.patch, HBASE-18164-06.patch, HBASE-18164-07.patch, > HBASE-18164-08.patch > > > We noticed that during the stochastic load balancer was not scaling well with > cluster size. That is to say that on our smaller clusters (~17 tables, ~12 > region servers, ~5k regions), the balancer considers ~100,000 cluster > configurations in 60s per balancer run, but only ~5,000 per 60s on our bigger > clusters (~82 tables, ~160 region servers, ~13k regions) . > Because of this, our bigger clusters are not able to converge on balance as > quickly for things like table skew, region load, etc. because the balancer > does not have enough time to "think". > We have re-written the locality cost function to be incremental, meaning it > only recomputes cost based on the most recent region move proposed by the > balancer, rather than recomputing the cost across all regions/servers every > iteration. > Further, we also cache the locality of every region on every server at the > beginning of the balancer's execution for both the LocalityBasedCostFunction > and the LocalityCandidateGenerator to reference. This way, they need not > collect all HDFS blocks of every region at each iteration of the balancer. > The changes have been running in all 6 of our production clusters and all 4 > QA clusters without issue. The speed improvements we noticed are massive. Our > big clusters now consider 20x more cluster configurations. > One design decision I made is to consider locality cost as the difference > between the best locality that is possible given the current cluster state, > and the currently measured locality. The old locality computation would > measure the locality cost as the difference from the current locality and > 100% locality, but this new computation instead takes the difference between > the current locality for a given region and the best locality for that region > in the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18161) Incremental Load support for Multiple-Table HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065690#comment-16065690 ] Hudson commented on HBASE-18161: SUCCESS: Integrated in Jenkins build HBase-2.0 #121 (See [https://builds.apache.org/job/HBase-2.0/121/]) HBASE-18161 Incremental Load support for Multiple-Table (tedyu: rev d5e206dfa06513c0ca26e7f2c93fc38f33703494) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableHFileOutputFormat.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableHFileOutputFormat.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java > Incremental Load support for Multiple-Table HFileOutputFormat > - > > Key: HBASE-18161 > URL: https://issues.apache.org/jira/browse/HBASE-18161 > Project: HBase > Issue Type: New Feature >Reporter: Densel Santhmayor >Assignee: Densel Santhmayor >Priority: Minor > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: MultiHFileOutputFormatSupport_HBASE_18161.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v10.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v11.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v2.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v3.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v4.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v5.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v6.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v7.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v8.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v9.patch > > > h2. Introduction > MapReduce currently supports the ability to write HBase records in bulk to > HFiles for a single table. The file(s) can then be uploaded to the relevant > RegionServers information with reasonable latency. This feature is useful to > make a large set of data available for queries at the same time as well as > provides a way to efficiently process very large input into HBase without > affecting query latencies. > There is, however, no support to write variations of the same record key to > HFiles belonging to multiple HBase tables from within the same MapReduce job. > > h2. Goal > The goal of this JIRA is to extend HFileOutputFormat2 to support writing to > HFiles for different tables within the same MapReduce job while single-table > HFile features backwards-compatible. > For our use case, we needed to write a record key to a smaller HBase table > for quicker access, and the same record key with a date appended to a larger > table for longer term storage with chronological access. Each of these tables > would have different TTL and other settings to support their respective > access patterns. We also needed to be able to bulk write records to multiple > tables with different subsets of very large input as efficiently as possible. > Rather than run the MapReduce job multiple times (one for each table or > record structure), it would be useful to be able to parse the input a single > time and write to multiple tables simultaneously. > Additionally, we'd like to maintain backwards compatibility with the existing > heavily-used HFileOutputFormat2 interface to allow benefits such as locality > sensitivity (that was introduced long after we implemented support for > multiple tables) to support both single table and multi table hfile writes. > h2. Proposal > * Backwards compatibility for existing single table support in > HFileOutputFormat2 will be maintained and in this case, mappers will need to > emit the table rowkey as before. However, a new class - > MultiHFileOutputFormat - will provide a helper function to generate a rowkey > for mappers that prefixes the desired tablename to the existing rowkey as > well as provides configureIncrementalLoad support for multiple tables. > * HFileOutputFormat2 will be updated in the following way: > ** configureIncrementalLoad will now accept multiple table descriptor and > region locator pairs, analogous to the single pair currently accepted by > HFileOutputFormat2. > ** Compression, Block Size, Bloom Type and Datablock settings PER column > family that are set in the Configuration object are now indexed and retrieved > by tablename AND column family > ** getRegionStartKeys will now support multiple regionlocators and calculate > split points and therefore partitions collectively for all tables. Similarly, > now the eventual number of Reducers will be equal to the total number of > partitions across all tables. > ** The RecordWriter class will be able to process rowkeys either with or > without the tablename
[jira] [Commented] (HBASE-18054) log when we add/remove failed servers in client
[
https://issues.apache.org/jira/browse/HBASE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065683#comment-16065683
]
Hadoop QA commented on HBASE-18054:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red} hbase-client in master has 4 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
44s{color} | {color:red} hbase-server in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
20s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}114m
46s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874749/HBASE-18054.v5.master.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 09903b4b969d 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8da6f06 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7365/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
|
| findbugs |
[jira] [Updated] (HBASE-18229) create new Async Split API to embrace AM v2
[ https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18229: - Attachment: hbase-18229-master-v4.patch > create new Async Split API to embrace AM v2 > --- > > Key: HBASE-18229 > URL: https://issues.apache.org/jira/browse/HBASE-18229 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Yi Liang >Assignee: Yi Liang > Attachments: HBase-18229-master-v1.patch, > hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch, > hbase-18229-master-v4.patch > > > See HBASE-18105 for related information, this jira will change the logic of > Path 1 in splitProcedure, the execution path will be: > *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> > MasterRpcServices.splitRegion -> HMaster.splitRegion-> > AssignmentManager.submitProcedure* > Master Will no longer ask Rs and then Rs turn around to ask master to do the > split operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[
https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065679#comment-16065679
]
Hadoop QA commented on HBASE-18281:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
42s{color} | {color:red} hbase-server in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}114m
5s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18281 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874750/0001-HBASE-18281-fixed-performance-using-Pattern.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 9792e824bbb7 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 8da6f06 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7366/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7366/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7366/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Performance update in StoreFileWriter.java for string replacement
> -
>
>
[jira] [Commented] (HBASE-18164) Much faster locality cost function and candidate generator
[ https://issues.apache.org/jira/browse/HBASE-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065677#comment-16065677 ] Hudson commented on HBASE-18164: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3273 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3273/]) Revert "HBASE-18164 Fast locality computation in balancer - addendum (busbey: rev 141482512a617a2a05c5dcf0c84fd81385d5b008) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java HBASE-18164 Fast locality computation in balancer - addendum handles NaN (busbey: rev 8da6f069c307d7cf0b8d56281189e84108c7430b) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java > Much faster locality cost function and candidate generator > -- > > Key: HBASE-18164 > URL: https://issues.apache.org/jira/browse/HBASE-18164 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Critical > Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-2 > > Attachments: 18164.branch-1.addendum.txt, HBASE-18164-00.patch, > HBASE-18164-01.patch, HBASE-18164-02.patch, HBASE-18164-04.patch, > HBASE-18164-05.patch, HBASE-18164-06.patch, HBASE-18164-07.patch, > HBASE-18164-08.patch > > > We noticed that during the stochastic load balancer was not scaling well with > cluster size. That is to say that on our smaller clusters (~17 tables, ~12 > region servers, ~5k regions), the balancer considers ~100,000 cluster > configurations in 60s per balancer run, but only ~5,000 per 60s on our bigger > clusters (~82 tables, ~160 region servers, ~13k regions) . > Because of this, our bigger clusters are not able to converge on balance as > quickly for things like table skew, region load, etc. because the balancer > does not have enough time to "think". > We have re-written the locality cost function to be incremental, meaning it > only recomputes cost based on the most recent region move proposed by the > balancer, rather than recomputing the cost across all regions/servers every > iteration. > Further, we also cache the locality of every region on every server at the > beginning of the balancer's execution for both the LocalityBasedCostFunction > and the LocalityCandidateGenerator to reference. This way, they need not > collect all HDFS blocks of every region at each iteration of the balancer. > The changes have been running in all 6 of our production clusters and all 4 > QA clusters without issue. The speed improvements we noticed are massive. Our > big clusters now consider 20x more cluster configurations. > One design decision I made is to consider locality cost as the difference > between the best locality that is possible given the current cluster state, > and the currently measured locality. The old locality computation would > measure the locality cost as the difference from the current locality and > 100% locality, but this new computation instead takes the difference between > the current locality for a given region and the best locality for that region > in the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18161) Incremental Load support for Multiple-Table HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065676#comment-16065676 ] Hudson commented on HBASE-18161: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3273 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3273/]) HBASE-18161 Incremental Load support for Multiple-Table (tedyu: rev 293cb87d52d6ccc98f0d03387f9dc07dc4522042) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableHFileOutputFormat.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableHFileOutputFormat.java > Incremental Load support for Multiple-Table HFileOutputFormat > - > > Key: HBASE-18161 > URL: https://issues.apache.org/jira/browse/HBASE-18161 > Project: HBase > Issue Type: New Feature >Reporter: Densel Santhmayor >Assignee: Densel Santhmayor >Priority: Minor > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: MultiHFileOutputFormatSupport_HBASE_18161.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v10.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v11.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v2.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v3.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v4.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v5.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v6.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v7.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v8.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v9.patch > > > h2. Introduction > MapReduce currently supports the ability to write HBase records in bulk to > HFiles for a single table. The file(s) can then be uploaded to the relevant > RegionServers information with reasonable latency. This feature is useful to > make a large set of data available for queries at the same time as well as > provides a way to efficiently process very large input into HBase without > affecting query latencies. > There is, however, no support to write variations of the same record key to > HFiles belonging to multiple HBase tables from within the same MapReduce job. > > h2. Goal > The goal of this JIRA is to extend HFileOutputFormat2 to support writing to > HFiles for different tables within the same MapReduce job while single-table > HFile features backwards-compatible. > For our use case, we needed to write a record key to a smaller HBase table > for quicker access, and the same record key with a date appended to a larger > table for longer term storage with chronological access. Each of these tables > would have different TTL and other settings to support their respective > access patterns. We also needed to be able to bulk write records to multiple > tables with different subsets of very large input as efficiently as possible. > Rather than run the MapReduce job multiple times (one for each table or > record structure), it would be useful to be able to parse the input a single > time and write to multiple tables simultaneously. > Additionally, we'd like to maintain backwards compatibility with the existing > heavily-used HFileOutputFormat2 interface to allow benefits such as locality > sensitivity (that was introduced long after we implemented support for > multiple tables) to support both single table and multi table hfile writes. > h2. Proposal > * Backwards compatibility for existing single table support in > HFileOutputFormat2 will be maintained and in this case, mappers will need to > emit the table rowkey as before. However, a new class - > MultiHFileOutputFormat - will provide a helper function to generate a rowkey > for mappers that prefixes the desired tablename to the existing rowkey as > well as provides configureIncrementalLoad support for multiple tables. > * HFileOutputFormat2 will be updated in the following way: > ** configureIncrementalLoad will now accept multiple table descriptor and > region locator pairs, analogous to the single pair currently accepted by > HFileOutputFormat2. > ** Compression, Block Size, Bloom Type and Datablock settings PER column > family that are set in the Configuration object are now indexed and retrieved > by tablename AND column family > ** getRegionStartKeys will now support multiple regionlocators and calculate > split points and therefore partitions collectively for all tables. Similarly, > now the eventual number of Reducers will be equal to the total number of > partitions across all tables. > ** The RecordWriter class will be able to process rowkeys either with or >
[jira] [Updated] (HBASE-18229) create new Async Split API to embrace AM v2
[ https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18229: - Description: See HBASE-18105 for related information, this jira will change the logic of Path 1 in splitProcedure, the execution path will be: *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> MasterRpcServices.splitRegion -> HMaster.splitRegion-> AssignmentManager.submitProcedure* Master Will no longer ask Rs and then Rs turn around to ask master to do the split operation. was: See HBASE-18105 for related information, this jira will change the logic of Path 1 in splitProcedure, the execution path will be: *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> MasterRpcServices.splitRegion -> HMaster.splitRegion-> AssignmentManager.submitProcedure* Master Will no longer as Rs and then Rs turn around to ask master to do the split operation. > create new Async Split API to embrace AM v2 > --- > > Key: HBASE-18229 > URL: https://issues.apache.org/jira/browse/HBASE-18229 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Yi Liang >Assignee: Yi Liang > Attachments: HBase-18229-master-v1.patch, > hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch > > > See HBASE-18105 for related information, this jira will change the logic of > Path 1 in splitProcedure, the execution path will be: > *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> > MasterRpcServices.splitRegion -> HMaster.splitRegion-> > AssignmentManager.submitProcedure* > Master Will no longer ask Rs and then Rs turn around to ask master to do the > split operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18282) ReplicationLogCleaner can delete WALs not yet replicated in case of a KeeperException
Ashu Pachauri created HBASE-18282:
-
Summary: ReplicationLogCleaner can delete WALs not yet replicated
in case of a KeeperException
Key: HBASE-18282
URL: https://issues.apache.org/jira/browse/HBASE-18282
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 2.0.0-alpha-1, 1.1.11, 1.2.6, 1.3.1
Reporter: Ashu Pachauri
Assignee: Ashu Pachauri
Priority: Critical
ReplicationStateZKBase#getListOfReplicators does not rethrow a KeeperException
and returns null in such a case. ReplicationLogCleaner just assumes that there
are no replicators and deletes everything.
ReplicationStateZKBase:
{code:java}
public List getListOfReplicators() {
List result = null;
try {
result = ZKUtil.listChildrenNoWatch(this.zookeeper, this.queuesZNode);
} catch (KeeperException e) {
this.abortable.abort("Failed to get list of replicators", e);
}
return result;
}
{code}
ReplicationLogCleaner:
{code:java}
private Set loadWALsFromQueues() throws KeeperException {
for (int retry = 0; ; retry++) {
int v0 = replicationQueues.getQueuesZNodeCversion();
List rss = replicationQueues.getListOfReplicators();
if (rss == null) {
LOG.debug("Didn't find any region server that replicates, won't prevent
any deletions.");
return ImmutableSet.of();
}
...
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18164) Much faster locality cost function and candidate generator
[ https://issues.apache.org/jira/browse/HBASE-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065601#comment-16065601 ] Hudson commented on HBASE-18164: FAILURE: Integrated in Jenkins build HBase-1.4 #792 (See [https://builds.apache.org/job/HBase-1.4/792/]) Revert "HBASE-18164 Fast locality computation in balancer - addendum (busbey: rev 386a6e87fecd2971354028cfebd4d19fd9dfe599) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java HBASE-18164 Fast locality computation in balancer - addendum handles NaN (busbey: rev bb181ff50bd60120bcda463d6d71db8505170e88) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java > Much faster locality cost function and candidate generator > -- > > Key: HBASE-18164 > URL: https://issues.apache.org/jira/browse/HBASE-18164 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Critical > Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-2 > > Attachments: 18164.branch-1.addendum.txt, HBASE-18164-00.patch, > HBASE-18164-01.patch, HBASE-18164-02.patch, HBASE-18164-04.patch, > HBASE-18164-05.patch, HBASE-18164-06.patch, HBASE-18164-07.patch, > HBASE-18164-08.patch > > > We noticed that during the stochastic load balancer was not scaling well with > cluster size. That is to say that on our smaller clusters (~17 tables, ~12 > region servers, ~5k regions), the balancer considers ~100,000 cluster > configurations in 60s per balancer run, but only ~5,000 per 60s on our bigger > clusters (~82 tables, ~160 region servers, ~13k regions) . > Because of this, our bigger clusters are not able to converge on balance as > quickly for things like table skew, region load, etc. because the balancer > does not have enough time to "think". > We have re-written the locality cost function to be incremental, meaning it > only recomputes cost based on the most recent region move proposed by the > balancer, rather than recomputing the cost across all regions/servers every > iteration. > Further, we also cache the locality of every region on every server at the > beginning of the balancer's execution for both the LocalityBasedCostFunction > and the LocalityCandidateGenerator to reference. This way, they need not > collect all HDFS blocks of every region at each iteration of the balancer. > The changes have been running in all 6 of our production clusters and all 4 > QA clusters without issue. The speed improvements we noticed are massive. Our > big clusters now consider 20x more cluster configurations. > One design decision I made is to consider locality cost as the difference > between the best locality that is possible given the current cluster state, > and the currently measured locality. The old locality computation would > measure the locality cost as the difference from the current locality and > 100% locality, but this new computation instead takes the difference between > the current locality for a given region and the best locality for that region > in the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[ https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065554#comment-16065554 ] Mike Drob commented on HBASE-18281: --- Your second patch includes the first patchfile as part of it, maybe a mistaken `git commit -a`? (smile) Check out the developer guide - http://hbase.apache.org/book.html#submitting.patches - maybe it can make your workflow easier. > Performance update in StoreFileWriter.java for string replacement > - > > Key: HBASE-18281 > URL: https://issues.apache.org/jira/browse/HBASE-18281 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Ben Epstein >Priority: Trivial > Attachments: 0001-HBASE-18281-fixed-performance-using-Pattern.patch, > 0002-created-static-Pattern-for-performance-upgrade.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Change the replaceAll() function using regex with a static Pattern for > performance upgrade -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065549#comment-16065549
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #157 (See
[https://builds.apache.org/job/HBase-1.2-JDK7/157/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
d0308825d290774a7dbd243d962434cf15bb51bf)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065542#comment-16065542
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #153 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/153/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
d0308825d290774a7dbd243d962434cf15bb51bf)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17056) Remove checked in PB generated files
[ https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17056: -- Status: Patch Available (was: In Progress) > Remove checked in PB generated files > - > > Key: HBASE-17056 > URL: https://issues.apache.org/jira/browse/HBASE-17056 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Enis Soztutar >Assignee: stack > Fix For: 2.0.0 > > Attachments: > 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, > HBASE-17056.master.001.patch > > > Now that we have the new PB maven plugin, there is no need to have the PB > files checked in to the repo. The reason we did that was to ease up developer > env setup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17056) Remove checked in PB generated files
[ https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17056: -- Status: In Progress (was: Patch Available) > Remove checked in PB generated files > - > > Key: HBASE-17056 > URL: https://issues.apache.org/jira/browse/HBASE-17056 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Enis Soztutar >Assignee: stack > Fix For: 2.0.0 > > Attachments: > 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, > HBASE-17056.master.001.patch > > > Now that we have the new PB maven plugin, there is no need to have the PB > files checked in to the repo. The reason we did that was to ease up developer > env setup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065528#comment-16065528
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #202 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/202/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
1f78b288cf67798e665e7a58682e4ce80754bd60)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[ https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Epstein updated HBASE-18281: Attachment: 0002-created-static-Pattern-for-performance-upgrade.patch Changed Pattern to static > Performance update in StoreFileWriter.java for string replacement > - > > Key: HBASE-18281 > URL: https://issues.apache.org/jira/browse/HBASE-18281 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Ben Epstein >Priority: Trivial > Attachments: 0001-HBASE-18281-fixed-performance-using-Pattern.patch, > 0002-created-static-Pattern-for-performance-upgrade.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Change the replaceAll() function using regex with a static Pattern for > performance upgrade -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065521#comment-16065521
]
Hudson commented on HBASE-18230:
FAILURE: Integrated in Jenkins build HBase-1.1-JDK8 #1968 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1968/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
fd067809b69ebb5db5b9889567de6488ef25398b)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065517#comment-16065517
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1885 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1885/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
fd067809b69ebb5db5b9889567de6488ef25398b)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17908) Upgrade guava
[ https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065512#comment-16065512 ] stack commented on HBASE-17908: --- Fixed bad scanner id manufacture. Seems to be responsible for the failures above. > Upgrade guava > - > > Key: HBASE-17908 > URL: https://issues.apache.org/jira/browse/HBASE-17908 > Project: HBase > Issue Type: Sub-task > Components: dependencies >Reporter: Balazs Meszaros >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-17908.master.001.patch, > HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, > HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, > HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, > HBASE-17908.master.008.patch > > > Currently we are using guava 12.0.1, but the latest version is 21.0. > Upgrading guava is always a hassle because it is not always backward > compatible with itself. > Currently I think there are to approaches: > 1. Upgrade guava to the newest version (21.0) and shade it. > 2. Upgrade guava to a version which does not break or builds (15.0). > If we can update it, some dependencies should be removed: > commons-collections, commons-codec, ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18086) Create native client which creates load on selected cluster
[ https://issues.apache.org/jira/browse/HBASE-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18086: --- Attachment: 18086.v6.txt Patch v6 adds support of Increment. > Create native client which creates load on selected cluster > --- > > Key: HBASE-18086 > URL: https://issues.apache.org/jira/browse/HBASE-18086 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18086.v1.txt, 18086.v3.txt, 18086.v4.txt, 18086.v5.txt, > 18086.v6.txt > > > This task is to create a client which uses multiple threads to conduct Puts > followed by Gets against selected cluster. > Default is to run the tool against local cluster. > This would give us some idea on the characteristics of native client in terms > of handling high load. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17908) Upgrade guava
[ https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17908: -- Attachment: HBASE-17908.master.008.patch > Upgrade guava > - > > Key: HBASE-17908 > URL: https://issues.apache.org/jira/browse/HBASE-17908 > Project: HBase > Issue Type: Sub-task > Components: dependencies >Reporter: Balazs Meszaros >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-17908.master.001.patch, > HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, > HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, > HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, > HBASE-17908.master.008.patch > > > Currently we are using guava 12.0.1, but the latest version is 21.0. > Upgrading guava is always a hassle because it is not always backward > compatible with itself. > Currently I think there are to approaches: > 1. Upgrade guava to the newest version (21.0) and shade it. > 2. Upgrade guava to a version which does not break or builds (15.0). > If we can update it, some dependencies should be removed: > commons-collections, commons-codec, ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[ https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065499#comment-16065499 ] Mike Drob commented on HBASE-18281: --- Would be even better to make that a static member variable instead of a field so that we only have to compile the pattern once at class init instead of once every time we call. > Performance update in StoreFileWriter.java for string replacement > - > > Key: HBASE-18281 > URL: https://issues.apache.org/jira/browse/HBASE-18281 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Ben Epstein >Priority: Trivial > Attachments: 0001-HBASE-18281-fixed-performance-using-Pattern.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Change the replaceAll() function using regex with a static Pattern for > performance upgrade -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065492#comment-16065492 ] Alex Leblang commented on HBASE-18147: -- That patch looks good to me [~busbey] I'll give it a + > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey > Attachments: HBASE-18147.0.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065489#comment-16065489 ] Yi Liang commented on HBASE-18175: -- Hi [~mdrob], It finally pass the findbugs warnig, ;), And in the new patch, I also carry your last comments about diamond operator and reuse of the static classes from ITBulkLoad. Thanks for reviewing. > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0 > > Attachments: hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18023) Log multi-* requests for more than threshold number of rows
[ https://issues.apache.org/jira/browse/HBASE-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18023: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Closing for now. Can re-open if there's interest for a branch-1 backport. Thanks again for the patch, David. > Log multi-* requests for more than threshold number of rows > --- > > Key: HBASE-18023 > URL: https://issues.apache.org/jira/browse/HBASE-18023 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Clay B. >Assignee: David Harju >Priority: Minor > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18023.master.001.patch, > HBASE-18023.master.002.patch, HBASE-18023.master.003.patch, > HBASE-18023.master.004.patch > > > Today, if a user happens to do something like a large multi-put, they can get > through request throttling (e.g. it is one request) but still crash a region > server with a garbage storm. We have seen regionservers hit this issue and it > is silent and deadly. The RS will report nothing more than a mysterious > garbage collection and exit out. > Ideally, we could report a large multi-* request before starting it, in case > it happens to be deadly. Knowing the client, user and how many rows are > affected would be a good start to tracking down painful users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[ https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Epstein updated HBASE-18281: Status: Patch Available (was: Open) Patch attached for fix > Performance update in StoreFileWriter.java for string replacement > - > > Key: HBASE-18281 > URL: https://issues.apache.org/jira/browse/HBASE-18281 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Ben Epstein >Priority: Trivial > Attachments: 0001-HBASE-18281-fixed-performance-using-Pattern.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Change the replaceAll() function using regex with a static Pattern for > performance upgrade -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
[ https://issues.apache.org/jira/browse/HBASE-18281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Epstein updated HBASE-18281: Attachment: 0001-HBASE-18281-fixed-performance-using-Pattern.patch Fix for regex to Pattern > Performance update in StoreFileWriter.java for string replacement > - > > Key: HBASE-18281 > URL: https://issues.apache.org/jira/browse/HBASE-18281 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Ben Epstein >Priority: Trivial > Attachments: 0001-HBASE-18281-fixed-performance-using-Pattern.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Change the replaceAll() function using regex with a static Pattern for > performance upgrade -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18054) log when we add/remove failed servers in client
[
https://issues.apache.org/jira/browse/HBASE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ali updated HBASE-18054:
Status: Patch Available (was: Open)
> log when we add/remove failed servers in client
> ---
>
> Key: HBASE-18054
> URL: https://issues.apache.org/jira/browse/HBASE-18054
> Project: HBase
> Issue Type: Bug
> Components: Client, Operability
>Affects Versions: 1.3.0
>Reporter: Sean Busbey
>Assignee: Ali
> Attachments: HBASE-18054.patch, HBASE-18054.v2.master.patch,
> HBASE-18054.v3.master.patch, HBASE-18054.v4.master.patch,
> HBASE-18054.v5.master.patch
>
>
> Currently we log if a server is in the failed server list when we go to
> connect to it, but we don't log anything about when the server got into the
> list.
> This means we have to search the log for errors involving the same server
> name that (hopefully) managed to get into the log within
> {{FAILED_SERVER_EXPIRY_KEY}} milliseconds earlier (default 2 seconds).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18054) log when we add/remove failed servers in client
[
https://issues.apache.org/jira/browse/HBASE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065473#comment-16065473
]
Ali commented on HBASE-18054:
-
thank you [~mdrob] for your feedback. i have made the changes in v5 patch
> log when we add/remove failed servers in client
> ---
>
> Key: HBASE-18054
> URL: https://issues.apache.org/jira/browse/HBASE-18054
> Project: HBase
> Issue Type: Bug
> Components: Client, Operability
>Affects Versions: 1.3.0
>Reporter: Sean Busbey
>Assignee: Ali
> Attachments: HBASE-18054.patch, HBASE-18054.v2.master.patch,
> HBASE-18054.v3.master.patch, HBASE-18054.v4.master.patch,
> HBASE-18054.v5.master.patch
>
>
> Currently we log if a server is in the failed server list when we go to
> connect to it, but we don't log anything about when the server got into the
> list.
> This means we have to search the log for errors involving the same server
> name that (hopefully) managed to get into the log within
> {{FAILED_SERVER_EXPIRY_KEY}} milliseconds earlier (default 2 seconds).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18054) log when we add/remove failed servers in client
[
https://issues.apache.org/jira/browse/HBASE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ali updated HBASE-18054:
Status: Open (was: Patch Available)
> log when we add/remove failed servers in client
> ---
>
> Key: HBASE-18054
> URL: https://issues.apache.org/jira/browse/HBASE-18054
> Project: HBase
> Issue Type: Bug
> Components: Client, Operability
>Affects Versions: 1.3.0
>Reporter: Sean Busbey
>Assignee: Ali
> Attachments: HBASE-18054.patch, HBASE-18054.v2.master.patch,
> HBASE-18054.v3.master.patch, HBASE-18054.v4.master.patch,
> HBASE-18054.v5.master.patch
>
>
> Currently we log if a server is in the failed server list when we go to
> connect to it, but we don't log anything about when the server got into the
> list.
> This means we have to search the log for errors involving the same server
> name that (hopefully) managed to get into the log within
> {{FAILED_SERVER_EXPIRY_KEY}} milliseconds earlier (default 2 seconds).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18054) log when we add/remove failed servers in client
[
https://issues.apache.org/jira/browse/HBASE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ali updated HBASE-18054:
Attachment: HBASE-18054.v5.master.patch
> log when we add/remove failed servers in client
> ---
>
> Key: HBASE-18054
> URL: https://issues.apache.org/jira/browse/HBASE-18054
> Project: HBase
> Issue Type: Bug
> Components: Client, Operability
>Affects Versions: 1.3.0
>Reporter: Sean Busbey
>Assignee: Ali
> Attachments: HBASE-18054.patch, HBASE-18054.v2.master.patch,
> HBASE-18054.v3.master.patch, HBASE-18054.v4.master.patch,
> HBASE-18054.v5.master.patch
>
>
> Currently we log if a server is in the failed server list when we go to
> connect to it, but we don't log anything about when the server got into the
> list.
> This means we have to search the log for errors involving the same server
> name that (hopefully) managed to get into the log within
> {{FAILED_SERVER_EXPIRY_KEY}} milliseconds earlier (default 2 seconds).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[
https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065467#comment-16065467
]
Hadoop QA commented on HBASE-18175:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-assembly hbase-spark/hbase-spark-it . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hbase-spark-it in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}138m 17s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
58s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}191m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.rsgroup.TestShellRSGroups |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18175 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874717/hbase-18175-master-v4.patch
|
| Optional Tests | asflicense javac javadoc unit xml compile findbugs
hadoopcheck
[jira] [Created] (HBASE-18281) Performance update in StoreFileWriter.java for string replacement
Ben Epstein created HBASE-18281: --- Summary: Performance update in StoreFileWriter.java for string replacement Key: HBASE-18281 URL: https://issues.apache.org/jira/browse/HBASE-18281 Project: HBase Issue Type: Improvement Components: community Reporter: Ben Epstein Priority: Trivial Change the replaceAll() function using regex with a static Pattern for performance upgrade -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18278) [AMv2] Enable and fix uni test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18278: - Attachment: HBASE-18278.master.002.patch Retry with minor change. Previous failure is for a flaky test. > [AMv2] Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18278 > URL: https://issues.apache.org/jira/browse/HBASE-18278 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0-alpha-2 > > Attachments: HBASE-18278.master.001.patch, > HBASE-18278.master.002.patch > > > Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18278) [AMv2] Enable and fix uni test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[
https://issues.apache.org/jira/browse/HBASE-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065421#comment-16065421
]
Hadoop QA commented on HBASE-18278:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
36s{color} | {color:red} hbase-server in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
59m 40s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 33m 43s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}123m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.locking.TestLockProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18278 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874721/HBASE-18278.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 049dd088 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 14957d4 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7361/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/7361/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7361/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7361/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> [AMv2] Enable and fix uni test
>
[jira] [Commented] (HBASE-18272) Fix issue about RSGroupBasedLoadBalancer#roundRobinAssignment where BOGUS_SERVER_NAME is involved in two groups
[ https://issues.apache.org/jira/browse/HBASE-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065400#comment-16065400 ] Hudson commented on HBASE-18272: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3272 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3272/]) HBASE-18272 Fix issue about (tedyu: rev 0f4ea611e18f15e8eeafb312e5b2f7f356c7d0a6) * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRSGroupBasedLoadBalancer.java > Fix issue about RSGroupBasedLoadBalancer#roundRobinAssignment where > BOGUS_SERVER_NAME is involved in two groups > --- > > Key: HBASE-18272 > URL: https://issues.apache.org/jira/browse/HBASE-18272 > Project: HBase > Issue Type: Sub-task > Components: Balancer >Reporter: chenxu >Assignee: chenxu > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18272-master-v1.patch, HBASE-18272-master-v2.patch > > > if two groups both include BOGUS_SERVER_NAME, assignments.putAll will > overwrite the previous data -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18259) HBase book link to "beginner" issues includes resolved issues
[ https://issues.apache.org/jira/browse/HBASE-18259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065403#comment-16065403 ] Hudson commented on HBASE-18259: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3272 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3272/]) HBASE-18259 HBase book link to "beginner" issues includes resolved (busbey: rev 14957d4a7dbfe506cb263cf6705511776d08bba3) * (edit) src/main/asciidoc/_chapters/developer.adoc > HBase book link to "beginner" issues includes resolved issues > - > > Key: HBASE-18259 > URL: https://issues.apache.org/jira/browse/HBASE-18259 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Mike Drob >Assignee: Peter Somogyi > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-18259.master.001.patch > > > The link at http://hbase.apache.org/book.html#getting.involved for beginner > issues is > https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20labels%20in%20(beginner) > but this includes resolved issues as well, which is not useful to folks > looking for new issues to cut their teeth on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065402#comment-16065402
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3272 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3272/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
b3ae7a29b8dfc3b1247deb27fdb852816b95d36d)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18274) hbase autorestart will overwrite the gc log
[
https://issues.apache.org/jira/browse/HBASE-18274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065401#comment-16065401
]
Hudson commented on HBASE-18274:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3272 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3272/])
HBASE-18274 hbase autorestart will overwrite the gc log (Fangyuan Deng) (tedyu:
rev c015abe5076d73ea90cd875cdfa61d2ced4e6ddb)
* (edit) bin/hbase-daemon.sh
> hbase autorestart will overwrite the gc log
> ---
>
> Key: HBASE-18274
> URL: https://issues.apache.org/jira/browse/HBASE-18274
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18274.0.patch
>
>
> While using hbase autorestart , the gc log will be overwirited after the
> process(hmaster or hregionserver) retarting.
> This is because the autorestart loop is in internal_autostart function ( in
> hbase-daemon.sh), but we only rotate the gc log in autorestart function.
> (internal_autostart)
> ONE_HOUR_IN_SECS=3600
> autostartWindowStartDate=`date +%s`
> autostartCount=0
> touch "$HBASE_AUTOSTART_FILE"
> # keep starting the command until asked to stop. Reloop on software crash
> while true
> do
> code # restart the hbase process util the cluster is shut down
> fi
> (autorestart)
> echo running $command, logging to $HBASE_LOGOUT
> # stop the command
> $thiscmd --config "${HBASE_CONF_DIR}" stop $command $args &
> wait_until_done $!
> # wait a user-specified sleep period
> sp=${HBASE_RESTART_SLEEP:-3}
> if [ $sp -gt 0 ]; then
> sleep $sp
> fi
> check_before_start
> hbase_rotate_log $HBASE_LOGOUT
> hbase_rotate_log $HBASE_LOGGC
> nohup $thiscmd --config "${HBASE_CONF_DIR}" --autostart-window-size
> ${AUTOSTART_WINDOW_SIZE} --autostart-window-retry-limit
> ${AUTOSTART_WINDOW_RETRY_LIMIT} \
> internal_autostart $command $args < /dev/null > ${HBASE_LOGOUT} 2>&1
> &
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18164) Much faster locality cost function and candidate generator
[
https://issues.apache.org/jira/browse/HBASE-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065388#comment-16065388
]
Sean Busbey commented on HBASE-18164:
-
reverted and reapplied with the changes I requested back at v7:
{quote}
I plan to amend the v7 patch to a) use 1.0 for the bestLocality instead of 1
and b) update the commit message. Let me know if folks want the modified patch
posted here.
{quote}
(the change in (a) to avoid implicitly casting from int to double was the
important bit)
> Much faster locality cost function and candidate generator
> --
>
> Key: HBASE-18164
> URL: https://issues.apache.org/jira/browse/HBASE-18164
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Reporter: Kahlil Oppenheimer
>Assignee: Kahlil Oppenheimer
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-2
>
> Attachments: 18164.branch-1.addendum.txt, HBASE-18164-00.patch,
> HBASE-18164-01.patch, HBASE-18164-02.patch, HBASE-18164-04.patch,
> HBASE-18164-05.patch, HBASE-18164-06.patch, HBASE-18164-07.patch,
> HBASE-18164-08.patch
>
>
> We noticed that during the stochastic load balancer was not scaling well with
> cluster size. That is to say that on our smaller clusters (~17 tables, ~12
> region servers, ~5k regions), the balancer considers ~100,000 cluster
> configurations in 60s per balancer run, but only ~5,000 per 60s on our bigger
> clusters (~82 tables, ~160 region servers, ~13k regions) .
> Because of this, our bigger clusters are not able to converge on balance as
> quickly for things like table skew, region load, etc. because the balancer
> does not have enough time to "think".
> We have re-written the locality cost function to be incremental, meaning it
> only recomputes cost based on the most recent region move proposed by the
> balancer, rather than recomputing the cost across all regions/servers every
> iteration.
> Further, we also cache the locality of every region on every server at the
> beginning of the balancer's execution for both the LocalityBasedCostFunction
> and the LocalityCandidateGenerator to reference. This way, they need not
> collect all HDFS blocks of every region at each iteration of the balancer.
> The changes have been running in all 6 of our production clusters and all 4
> QA clusters without issue. The speed improvements we noticed are massive. Our
> big clusters now consider 20x more cluster configurations.
> One design decision I made is to consider locality cost as the difference
> between the best locality that is possible given the current cluster state,
> and the currently measured locality. The old locality computation would
> measure the locality cost as the difference from the current locality and
> 100% locality, but this new computation instead takes the difference between
> the current locality for a given region and the best locality for that region
> in the cluster.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18161) Incremental Load support for Multiple-Table HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-18161: -- Assignee: Densel Santhmayor Hadoop Flags: Reviewed Fix Version/s: 2.0.0-alpha-2 3.0.0 > Incremental Load support for Multiple-Table HFileOutputFormat > - > > Key: HBASE-18161 > URL: https://issues.apache.org/jira/browse/HBASE-18161 > Project: HBase > Issue Type: New Feature >Reporter: Densel Santhmayor >Assignee: Densel Santhmayor >Priority: Minor > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: MultiHFileOutputFormatSupport_HBASE_18161.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v10.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v11.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v2.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v3.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v4.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v5.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v6.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v7.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v8.patch, > MultiHFileOutputFormatSupport_HBASE_18161_v9.patch > > > h2. Introduction > MapReduce currently supports the ability to write HBase records in bulk to > HFiles for a single table. The file(s) can then be uploaded to the relevant > RegionServers information with reasonable latency. This feature is useful to > make a large set of data available for queries at the same time as well as > provides a way to efficiently process very large input into HBase without > affecting query latencies. > There is, however, no support to write variations of the same record key to > HFiles belonging to multiple HBase tables from within the same MapReduce job. > > h2. Goal > The goal of this JIRA is to extend HFileOutputFormat2 to support writing to > HFiles for different tables within the same MapReduce job while single-table > HFile features backwards-compatible. > For our use case, we needed to write a record key to a smaller HBase table > for quicker access, and the same record key with a date appended to a larger > table for longer term storage with chronological access. Each of these tables > would have different TTL and other settings to support their respective > access patterns. We also needed to be able to bulk write records to multiple > tables with different subsets of very large input as efficiently as possible. > Rather than run the MapReduce job multiple times (one for each table or > record structure), it would be useful to be able to parse the input a single > time and write to multiple tables simultaneously. > Additionally, we'd like to maintain backwards compatibility with the existing > heavily-used HFileOutputFormat2 interface to allow benefits such as locality > sensitivity (that was introduced long after we implemented support for > multiple tables) to support both single table and multi table hfile writes. > h2. Proposal > * Backwards compatibility for existing single table support in > HFileOutputFormat2 will be maintained and in this case, mappers will need to > emit the table rowkey as before. However, a new class - > MultiHFileOutputFormat - will provide a helper function to generate a rowkey > for mappers that prefixes the desired tablename to the existing rowkey as > well as provides configureIncrementalLoad support for multiple tables. > * HFileOutputFormat2 will be updated in the following way: > ** configureIncrementalLoad will now accept multiple table descriptor and > region locator pairs, analogous to the single pair currently accepted by > HFileOutputFormat2. > ** Compression, Block Size, Bloom Type and Datablock settings PER column > family that are set in the Configuration object are now indexed and retrieved > by tablename AND column family > ** getRegionStartKeys will now support multiple regionlocators and calculate > split points and therefore partitions collectively for all tables. Similarly, > now the eventual number of Reducers will be equal to the total number of > partitions across all tables. > ** The RecordWriter class will be able to process rowkeys either with or > without the tablename prepended depending on how configureIncrementalLoad was > configured with MultiHFileOutputFormat or HFileOutputFormat2. > * The use of MultiHFileOutputFormat will write the output into HFiles which > will match the output format of HFileOutputFormat2. However, while the > default use case will keep the existing directory structure with column > family name as the directory and HFiles within that directory, in the case of > MultiHFileOutputFormat, it will output HFiles in the output directory with > the following relative paths:
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065371#comment-16065371
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-1.4 #791 (See
[https://builds.apache.org/job/HBase-1.4/791/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
a88310e9ae68e4146847c39087fbfeda19cbd8ea)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065352#comment-16065352
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #890 (See
[https://builds.apache.org/job/HBase-1.2-IT/890/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
d0308825d290774a7dbd243d962434cf15bb51bf)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18272) Fix issue about RSGroupBasedLoadBalancer#roundRobinAssignment where BOGUS_SERVER_NAME is involved in two groups
[ https://issues.apache.org/jira/browse/HBASE-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065342#comment-16065342 ] Hudson commented on HBASE-18272: FAILURE: Integrated in Jenkins build HBase-2.0 #120 (See [https://builds.apache.org/job/HBase-2.0/120/]) HBASE-18272 Fix issue about (tedyu: rev 6e1907919ab4904e03d0b28d940f0826c201a7c7) * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRSGroupBasedLoadBalancer.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java > Fix issue about RSGroupBasedLoadBalancer#roundRobinAssignment where > BOGUS_SERVER_NAME is involved in two groups > --- > > Key: HBASE-18272 > URL: https://issues.apache.org/jira/browse/HBASE-18272 > Project: HBase > Issue Type: Sub-task > Components: Balancer >Reporter: chenxu >Assignee: chenxu > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18272-master-v1.patch, HBASE-18272-master-v2.patch > > > if two groups both include BOGUS_SERVER_NAME, assignments.putAll will > overwrite the previous data -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18274) hbase autorestart will overwrite the gc log
[
https://issues.apache.org/jira/browse/HBASE-18274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065343#comment-16065343
]
Hudson commented on HBASE-18274:
FAILURE: Integrated in Jenkins build HBase-2.0 #120 (See
[https://builds.apache.org/job/HBase-2.0/120/])
HBASE-18274 hbase autorestart will overwrite the gc log (Fangyuan Deng) (tedyu:
rev 2cf3622b47178c5a1bdb9d81839cb1ae76fda992)
* (edit) bin/hbase-daemon.sh
> hbase autorestart will overwrite the gc log
> ---
>
> Key: HBASE-18274
> URL: https://issues.apache.org/jira/browse/HBASE-18274
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18274.0.patch
>
>
> While using hbase autorestart , the gc log will be overwirited after the
> process(hmaster or hregionserver) retarting.
> This is because the autorestart loop is in internal_autostart function ( in
> hbase-daemon.sh), but we only rotate the gc log in autorestart function.
> (internal_autostart)
> ONE_HOUR_IN_SECS=3600
> autostartWindowStartDate=`date +%s`
> autostartCount=0
> touch "$HBASE_AUTOSTART_FILE"
> # keep starting the command until asked to stop. Reloop on software crash
> while true
> do
> code # restart the hbase process util the cluster is shut down
> fi
> (autorestart)
> echo running $command, logging to $HBASE_LOGOUT
> # stop the command
> $thiscmd --config "${HBASE_CONF_DIR}" stop $command $args &
> wait_until_done $!
> # wait a user-specified sleep period
> sp=${HBASE_RESTART_SLEEP:-3}
> if [ $sp -gt 0 ]; then
> sleep $sp
> fi
> check_before_start
> hbase_rotate_log $HBASE_LOGOUT
> hbase_rotate_log $HBASE_LOGGC
> nohup $thiscmd --config "${HBASE_CONF_DIR}" --autostart-window-size
> ${AUTOSTART_WINDOW_SIZE} --autostart-window-retry-limit
> ${AUTOSTART_WINDOW_RETRY_LIMIT} \
> internal_autostart $command $args < /dev/null > ${HBASE_LOGOUT} 2>&1
> &
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065344#comment-16065344
]
Hudson commented on HBASE-18230:
FAILURE: Integrated in Jenkins build HBase-2.0 #120 (See
[https://builds.apache.org/job/HBase-2.0/120/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
0a4794aabb0f9e97bc2e8b5f5429ab2fe43cda3c)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18275) formatting and grammar mistakes in schemadoc chapter
[
https://issues.apache.org/jira/browse/HBASE-18275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065339#comment-16065339
]
Hadoop QA commented on HBASE-18275:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 46s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 17s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 183m 15s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 230m 35s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874701/HBASE-18275-0.patch |
| JIRA Issue | HBASE-18275 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux a9940f393c7b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c015abe |
| Default Java | 1.8.0_131 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/7359/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7359/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7359/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7359/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> formatting and grammar mistakes in schemadoc chapter
>
>
> Key: HBASE-18275
> URL: https://issues.apache.org/jira/browse/HBASE-18275
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Trivial
> Labels: newbie
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18275-0.patch
>
>
> a grammatical error and formatting error in schema design doc.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065330#comment-16065330
]
Hudson commented on HBASE-18230:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #70 (See
[https://builds.apache.org/job/HBase-1.3-IT/70/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
1f78b288cf67798e665e7a58682e4ce80754bd60)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18280) Manual Array Copy Cleanup: Automated
[ https://issues.apache.org/jira/browse/HBASE-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18280: Status: Patch Available (was: Open) > Manual Array Copy Cleanup: Automated > > > Key: HBASE-18280 > URL: https://issues.apache.org/jira/browse/HBASE-18280 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > Attachments: 0001-HBASE-18280-Remove-Manual-Array-Copies.patch > > > Remove Manual Array Copies: Style Issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18280) Manual Array Copy Cleanup: Automated
[ https://issues.apache.org/jira/browse/HBASE-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065321#comment-16065321 ] Mike Drob commented on HBASE-18280: --- +1 LTGM. I think this is more than a style issue though, System.arrayCopy probably has some native optimizations going for it too. > Manual Array Copy Cleanup: Automated > > > Key: HBASE-18280 > URL: https://issues.apache.org/jira/browse/HBASE-18280 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > Attachments: 0001-HBASE-18280-Remove-Manual-Array-Copies.patch > > > Remove Manual Array Copies: Style Issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18279) Manual Array to Collection Copy: Automated
[
https://issues.apache.org/jira/browse/HBASE-18279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065311#comment-16065311
]
Hadoop QA commented on HBASE-18279:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 3m
25s{color} | {color:red} Docker failed to build yetus/hbase:757bf37. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18279 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12874724/0001-HBASE-18279-Fix-Manual-Array-to-Collection-Copy.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7362/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Manual Array to Collection Copy: Automated
> --
>
> Key: HBASE-18279
> URL: https://issues.apache.org/jira/browse/HBASE-18279
> Project: HBase
> Issue Type: Improvement
>Reporter: John Leach
>Assignee: John Leach
>Priority: Trivial
> Attachments:
> 0001-HBASE-18279-Fix-Manual-Array-to-Collection-Copy.patch
>
>
> Style Cleanup.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18280) Manual Array Copy Cleanup: Automated
[ https://issues.apache.org/jira/browse/HBASE-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Leach updated HBASE-18280: --- Attachment: 0001-HBASE-18280-Remove-Manual-Array-Copies.patch > Manual Array Copy Cleanup: Automated > > > Key: HBASE-18280 > URL: https://issues.apache.org/jira/browse/HBASE-18280 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > Attachments: 0001-HBASE-18280-Remove-Manual-Array-Copies.patch > > > Remove Manual Array Copies: Style Issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18280) Manual Array Copy Cleanup: Automated
[ https://issues.apache.org/jira/browse/HBASE-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065296#comment-16065296 ] John Leach commented on HBASE-18280: Style Change > Manual Array Copy Cleanup: Automated > > > Key: HBASE-18280 > URL: https://issues.apache.org/jira/browse/HBASE-18280 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > Attachments: 0001-HBASE-18280-Remove-Manual-Array-Copies.patch > > > Remove Manual Array Copies: Style Issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18280) Manual Array Copy Cleanup: Automated
John Leach created HBASE-18280: -- Summary: Manual Array Copy Cleanup: Automated Key: HBASE-18280 URL: https://issues.apache.org/jira/browse/HBASE-18280 Project: HBase Issue Type: Improvement Reporter: John Leach Assignee: John Leach Priority: Trivial Remove Manual Array Copies: Style Issue -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065285#comment-16065285
]
Hudson commented on HBASE-18230:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #188 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/188/])
HBASE-18230 Remove Velocity vars from LICENSE (Addendum) (busbey: rev
1f78b288cf67798e665e7a58682e4ce80754bd60)
* (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18279) Manual Array to Collection Copy: Automated
[ https://issues.apache.org/jira/browse/HBASE-18279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Leach updated HBASE-18279: --- Status: Patch Available (was: Open) > Manual Array to Collection Copy: Automated > -- > > Key: HBASE-18279 > URL: https://issues.apache.org/jira/browse/HBASE-18279 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > Attachments: > 0001-HBASE-18279-Fix-Manual-Array-to-Collection-Copy.patch > > > Style Cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18279) Manual Array to Collection Copy: Automated
[ https://issues.apache.org/jira/browse/HBASE-18279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Leach updated HBASE-18279: --- Attachment: 0001-HBASE-18279-Fix-Manual-Array-to-Collection-Copy.patch > Manual Array to Collection Copy: Automated > -- > > Key: HBASE-18279 > URL: https://issues.apache.org/jira/browse/HBASE-18279 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > Attachments: > 0001-HBASE-18279-Fix-Manual-Array-to-Collection-Copy.patch > > > Style Cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18279) Manual Array to Collection Copy: Automated
John Leach created HBASE-18279: -- Summary: Manual Array to Collection Copy: Automated Key: HBASE-18279 URL: https://issues.apache.org/jira/browse/HBASE-18279 Project: HBase Issue Type: Improvement Reporter: John Leach Assignee: John Leach Style Cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18279) Manual Array to Collection Copy: Automated
[ https://issues.apache.org/jira/browse/HBASE-18279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Leach updated HBASE-18279: --- Priority: Trivial (was: Major) > Manual Array to Collection Copy: Automated > -- > > Key: HBASE-18279 > URL: https://issues.apache.org/jira/browse/HBASE-18279 > Project: HBase > Issue Type: Improvement >Reporter: John Leach >Assignee: John Leach >Priority: Trivial > > Style Cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18278) [AMv2] Enable and fix uni test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18278: - Status: Patch Available (was: In Progress) > [AMv2] Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18278 > URL: https://issues.apache.org/jira/browse/HBASE-18278 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0-alpha-2 > > Attachments: HBASE-18278.master.001.patch > > > Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18278) [AMv2] Enable and fix uni test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18278: - Attachment: HBASE-18278.master.001.patch Verified by running unit test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta() 20 times in a loop with my dev setup. It failed 4/20 times without patch. With patch succeeded all 20 times. > [AMv2] Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18278 > URL: https://issues.apache.org/jira/browse/HBASE-18278 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0-alpha-2 > > Attachments: HBASE-18278.master.001.patch > > > Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18230) Generated LICENSE file includes unsubstituted Velocity variables
[
https://issues.apache.org/jira/browse/HBASE-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18230:
Resolution: Fixed
Status: Resolved (was: Patch Available)
addendum is on all branches now.
> Generated LICENSE file includes unsubstituted Velocity variables
>
>
> Key: HBASE-18230
> URL: https://issues.apache.org/jira/browse/HBASE-18230
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18230.addendum.patch, HBASE-18230.patch
>
>
> From the release vote:
> {quote}
> we have a ton of places where we have velocity variables instead of
> copyright years, but IIRC that's a problem on branch-1 right now too.
> {quote}
> This is referring to lines like these:
> {noformat}
> * javax.annotation-api, ${dep.licenses[0].comments}
> * javax.servlet-api, ${dep.licenses[0].comments}
> * jetty-schemas, ${dep.licenses[0].comments}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Work started] (HBASE-18278) [AMv2] Enable and fix uni test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18278 started by Umesh Agashe. > [AMv2] Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18278 > URL: https://issues.apache.org/jira/browse/HBASE-18278 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0-alpha-2 > > > Enable and fix uni test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta -- This message was sent by Atlassian JIRA (v6.4.14#64029)
