[jira] [Commented] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605296#comment-16605296
]
Hudson commented on HBASE-21153:
Results for branch branch-2
[build #1207 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21107) add a metrics for netty direct memory
[
https://issues.apache.org/jira/browse/HBASE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605297#comment-16605297
]
Hudson commented on HBASE-21107:
Results for branch branch-2
[build #1207 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1207//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> add a metrics for netty direct memory
> -
>
> Key: HBASE-21107
> URL: https://issues.apache.org/jira/browse/HBASE-21107
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
>Affects Versions: 2.0.1
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21107-master-v1.patch,
> HBASE-21107-master-v1.patch, HBASE-21107-master-v2.patch,
> HBASE-21107-master-v2.patch, HBASE-21107-master-v2.patch,
> HBASE-21107-master-v3.patch, Screen Shot 2018-08-23 at 4.55.01 PM.png
>
>
> This is separated from HBASE-20913 as I realized that netty direct memory
> usage needs to show up in metrics instead of the web page.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21157) Split TableInputFormatScan to individual tests
[ https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605283#comment-16605283 ] Guanghao Zhang commented on HBASE-21157: +1. > Split TableInputFormatScan to individual tests > -- > > Key: HBASE-21157 > URL: https://issues.apache.org/jira/browse/HBASE-21157 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Minor > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21157.patch > > > We have done a split in HBASE-8326, which split the test to two parts. But it > is still a bit slow, split it into several tests can increase the parallelism > and make the 'mvn test' run faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[
https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605281#comment-16605281
]
Allan Yang commented on HBASE-21023:
{quote}
I think that would mean firing up another tool, like we used to do running
hbck1. What you thinking?
{quote}
Make sense to me.
> Add bypassProcedureToCompletion() API to HbckService
>
>
> Key: HBASE-21023
> URL: https://issues.apache.org/jira/browse/HBASE-21023
> Project: HBase
> Issue Type: Sub-task
> Components: hbck2
>Affects Versions: 2.0.1
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.2.0
>
>
> completeProcedure/s(): some procedures do not support abort at every step.
> When these procedures get stuck then they can not be aborted or make further
> progress. Corrective action is to bypass these procedures to completion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[ https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605280#comment-16605280 ] stack commented on HBASE-21155: --- .003 Rolled in some javadoc and commentary I had lying around the place. > Save on a few log strings and some churn in wal splitter by skipping out > early if no logs in dir > > > Key: HBASE-21155 > URL: https://issues.apache.org/jira/browse/HBASE-21155 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Trivial > Fix For: 2.1.1 > > Attachments: HBASE-21155.branch-2.1.001.patch, > HBASE-21155.branch-2.1.002.patch, HBASE-21155.branch-2.1.002.patch, > HBASE-21155.branch-2.1.003.patch > > > Trivial change to splitlogmanager that saves us a log line at least per WAL > dir when it goes to split. Also saves some not-needed churn in SLM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[ https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21155: -- Attachment: HBASE-21155.branch-2.1.003.patch > Save on a few log strings and some churn in wal splitter by skipping out > early if no logs in dir > > > Key: HBASE-21155 > URL: https://issues.apache.org/jira/browse/HBASE-21155 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Trivial > Fix For: 2.1.1 > > Attachments: HBASE-21155.branch-2.1.001.patch, > HBASE-21155.branch-2.1.002.patch, HBASE-21155.branch-2.1.002.patch, > HBASE-21155.branch-2.1.003.patch > > > Trivial change to splitlogmanager that saves us a log line at least per WAL > dir when it goes to split. Also saves some not-needed churn in SLM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-18956) ruby-lint need to be configured to understand that we're using jruby
[
https://issues.apache.org/jira/browse/HBASE-18956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605267#comment-16605267
]
Sean Busbey edited comment on HBASE-18956 at 9/6/18 4:56 AM:
-
ugh. looks like even though ruby-lint supports jruby it doesn't have anything
to handle the jruby java bridge stuff. To do this we'd need to make a ruby-lint
definition file:
http://code.yorickpeterse.com/ruby-lint/latest/file.definitions.html
and then add that definition via a {{ruby-lint.yml}} configuration file:
http://code.yorickpeterse.com/ruby-lint/latest/file.configuration.html#requires
was (Author: busbey):
ugh. looks like even though ruby-lint support jruby it doesn't have anything to
handle the jruby java bridge stuff. To do this we'd need to make a ruby-lint
definition file:
http://code.yorickpeterse.com/ruby-lint/latest/file.definitions.html
and then add that definition via a {{ruby-lint.yml}} configuration file:
http://code.yorickpeterse.com/ruby-lint/latest/file.configuration.html#requires
> ruby-lint need to be configured to understand that we're using jruby
>
>
> Key: HBASE-18956
> URL: https://issues.apache.org/jira/browse/HBASE-18956
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
> In HBASE-18909, I add a new java_import for admin.rb. But the ruby-lint
> report "undefined method java_import".
> {code}
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:22:13: undefined
> method java
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:23:1: undefined
> method java_import
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:49:29: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:273:27: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:287:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:317:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1009:30: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1015:35: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1016:9: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1022:28: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1028:33: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1029:9: undefined
> constant Pattern
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18956) ruby-lint need to be configured to understand that we're using jruby
[
https://issues.apache.org/jira/browse/HBASE-18956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605267#comment-16605267
]
Sean Busbey commented on HBASE-18956:
-
ugh. looks like even though ruby-lint support jruby it doesn't have anything to
handle the jruby java bridge stuff. To do this we'd need to make a ruby-lint
definition file:
http://code.yorickpeterse.com/ruby-lint/latest/file.definitions.html
and then add that definition via a {{ruby-lint.yml}} configuration file:
http://code.yorickpeterse.com/ruby-lint/latest/file.configuration.html#requires
> ruby-lint need to be configured to understand that we're using jruby
>
>
> Key: HBASE-18956
> URL: https://issues.apache.org/jira/browse/HBASE-18956
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
> In HBASE-18909, I add a new java_import for admin.rb. But the ruby-lint
> report "undefined method java_import".
> {code}
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:22:13: undefined
> method java
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:23:1: undefined
> method java_import
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:49:29: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:273:27: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:287:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:317:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1009:30: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1015:35: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1016:9: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1022:28: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1028:33: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1029:9: undefined
> constant Pattern
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20857) JMX - add Balancer status = enabled / disabled
[ https://issues.apache.org/jira/browse/HBASE-20857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kiran Kumar Maturi updated HBASE-20857: --- Attachment: HBASE-20857.branch-1.4.001.patch > JMX - add Balancer status = enabled / disabled > -- > > Key: HBASE-20857 > URL: https://issues.apache.org/jira/browse/HBASE-20857 > Project: HBase > Issue Type: Improvement > Components: API, master, metrics, REST, tooling, Usability >Reporter: Hari Sekhon >Assignee: Kiran Kumar Maturi >Priority: Major > Attachments: HBASE-20857.branch-1.4.001.patch > > > Add HBase Balancer enabled/disabled status to JMX API on HMaster. > Right now the HMaster will give a warning near the top of HMaster UI if > balancer is disabled, but scraping this is for monitoring integration is not > nice, it should be available in JMX API as there is already a > Master,sub=Balancer bean with metrics for the balancer ops etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605261#comment-16605261
]
Hadoop QA commented on HBASE-21138:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
9s{color} | {color:red} root in master failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed with JDK v1.8.0_181 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
9s{color} | {color:red} hbase-server in master failed with JDK v1.7.0_191.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
7s{color} | {color:red} branch has 7 errors when building our shaded downstream
artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed with JDK v1.8.0_181 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
9s{color} | {color:red} hbase-server in master failed with JDK v1.7.0_191.
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
7s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
10s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_191.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 10s{color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_191. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
8s{color} | {color:red} patch has 7 errors when building our shaded downstream
artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
7s{color} | {color:red} The patch causes 7 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
15s{color} | {color:red} The patch causes 7 errors with Hadoop v3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
10s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_191.
{color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 9s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21138 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938581/HBASE-21138.004.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname |
[jira] [Commented] (HBASE-21157) Split TableInputFormatScan to individual tests
[
https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605259#comment-16605259
]
Hadoop QA commented on HBASE-21157:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 15 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 30s{color}
| {color:red} hbase-mapreduce generated 1 new + 157 unchanged - 1 fixed = 158
total (was 158) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} hbase-mapreduce: The patch generated 0 new + 6
unchanged - 70 fixed = 6 total (was 76) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 55s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
54s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21157 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938574/HBASE-21157.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0e7ba533102d 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5a672b9da9 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/14324/artifact/patchprocess/diff-compile-javac-hbase-mapreduce.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14324/testReport/ |
| Max. process+thread count | 2473 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605258#comment-16605258 ] stack commented on HBASE-21023: --- [~allan163] Just starting to think about this now. It would certainly be 'easier' adding to shell but we were thinking of keeping hbck ops apart from general user items. I think that would mean firing up another tool, like we used to do running hbck1. What you thinking? > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605254#comment-16605254 ] Allan Yang commented on HBASE-21023: I prefer bypassing should be exposed to shell rather than HBCK2. In HBCK, we should figure out a way to take care of all situations, not a single procedure stuck. > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605253#comment-16605253
]
Hudson commented on HBASE-21153:
Results for branch branch-2.1
[build #284 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/284/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/284//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/284//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/284//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20786) Table create with thousands of regions takes too long
[ https://issues.apache.org/jira/browse/HBASE-20786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605251#comment-16605251 ] Allan Yang commented on HBASE-20786: Looks like RegionLocationFinder.refreshAndWait is doing some hdfs locality calculation so it can better balance the region later. I think we can add this process to additional thread, and doing it background, do not block the master start up. > Table create with thousands of regions takes too long > - > > Key: HBASE-20786 > URL: https://issues.apache.org/jira/browse/HBASE-20786 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Priority: Major > > Internal testing has create of a table with 33k regions taking 18 minutes. > Let me provide more info below. We have an executor with default ten threads > handling the creation of the regions in HDFS which helps distribute out the > load but its not enough. This cluster had >600 servers. Let me add detail. > Need to spend some time on speeding up create/assigns. Made this an umbrella > issue so can pick off pieces of the problem as subtasks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HBASE-21138:
--
Attachment: HBASE-21138.004.patch
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch, HBASE-21138.004.patch,
> HBASE-21138.branch-1.004.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HBASE-21138:
--
Attachment: HBASE-21138.branch-1.004.patch
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch, HBASE-21138.branch-1.004.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605248#comment-16605248
]
Allan Yang commented on HBASE-21154:
+1 for folding it to hbase:meta. For now, we must wait before namespace region
online, sometimes, it will block the master to finish init.
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18956) ruby-lint need to be configured to understand that we're using jruby
[
https://issues.apache.org/jira/browse/HBASE-18956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605246#comment-16605246
]
Sean Busbey commented on HBASE-18956:
-
this is because our ruby-lint configs don't tell it that we're using jruby so
it doesn't understand any of the jruby-specific stuff.
let me see if I can find a pointer on how to do this.
> ruby-lint need to be configured to understand that we're using jruby
>
>
> Key: HBASE-18956
> URL: https://issues.apache.org/jira/browse/HBASE-18956
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
> In HBASE-18909, I add a new java_import for admin.rb. But the ruby-lint
> report "undefined method java_import".
> {code}
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:22:13: undefined
> method java
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:23:1: undefined
> method java_import
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:49:29: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:273:27: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:287:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:317:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1009:30: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1015:35: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1016:9: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1022:28: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1028:33: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1029:9: undefined
> constant Pattern
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18956) ruby-lint need to be configured to understand that we're using jruby
[
https://issues.apache.org/jira/browse/HBASE-18956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18956:
Summary: ruby-lint need to be configured to understand that we're using
jruby (was: ruby-lint report "undefined method java_import")
> ruby-lint need to be configured to understand that we're using jruby
>
>
> Key: HBASE-18956
> URL: https://issues.apache.org/jira/browse/HBASE-18956
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
> In HBASE-18909, I add a new java_import for admin.rb. But the ruby-lint
> report "undefined method java_import".
> {code}
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:22:13: undefined
> method java
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:23:1: undefined
> method java_import
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:49:29: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:273:27: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:287:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:317:17: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1009:30: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1015:35: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1016:9: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1022:28: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1028:33: undefined
> constant Pattern
> /testptch/hbase/hbase-shell/src/main/ruby/hbase/admin.rb:E:1029:9: undefined
> constant Pattern
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21143) Update findbugs-maven-plugin to 3.0.4
[
https://issues.apache.org/jira/browse/HBASE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605241#comment-16605241
]
Mike Drob commented on HBASE-21143:
---
+1
> Update findbugs-maven-plugin to 3.0.4
> -
>
> Key: HBASE-21143
> URL: https://issues.apache.org/jira/browse/HBASE-21143
> Project: HBase
> Issue Type: Bug
> Components: pom
>Affects Versions: 3.0.0, 2.1.0, 2.2.0, 2.0.2
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21143.master.001.patch
>
>
> {code}
> Failed to execute goal org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs
> (default) on project hbase: Execution default of goal
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs failed: Plugin
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0 or one of its dependencies
> could not be resolved: Failed to collect dependencies at
> org.codehaus.mojo:findbugs-maven-plugin:jar:3.0.0 ->
> org.codehaus.groovy:groovy-all:jar:1.7.4: Failed to read artifact descriptor
> for org.codehaus.groovy:groovy-all:jar:1.7.4: Could not transfer artifact
> org.codehaus.groovy:groovy-all:pom:1.7.4 from/to mirror
> (http://xxx..xxx/nexus/content/groups/public): Failed to transfer file:
> http://xxx..xxx/nexus/content/groups/public/org/codehaus/groovy/groovy-all/1.7.4/groovy-all-1.7.4.pom.
> Return code is: 418 , ReasonPhrase:Artifact is in Tencent Blacklist! Please
> update to the safe version, more information:
> http://xxx..xxx/?tab=blackList.
> {code}
> Recently, when I compile HBase with a new machine, I got the above error.
> Since the machine could not connect to the external network, we visited our
> internal Maven repository, but org.codehaus.groovy:groovy-all:jar:1.7.4 was
> added to the blacklist and could not be downloaded. See details,
> org.codehaus.groovy:groovy-all:jar:1.7.4 is marked as vulnerable by
> [CVE-2015-3253|https://www.cvedetails.com/cve/CVE-2015-3253], so we should
> upgrade the version.
> {code:xml}
> org.codehaus.mojo
> findbugs-maven-plugin
> 3.0.0
>
>
>
> ${project.basedir}/../dev-support/findbugs-exclude.xml
> {code}
> Look at the history commit record, findbugs-maven-plugin has been upgraded to
> 3.0.4 in HBASE-18264, but one place is missing which still using the version
> of 3.0.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[ https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21155: -- Attachment: HBASE-21155.branch-2.1.002.patch > Save on a few log strings and some churn in wal splitter by skipping out > early if no logs in dir > > > Key: HBASE-21155 > URL: https://issues.apache.org/jira/browse/HBASE-21155 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Trivial > Fix For: 2.1.1 > > Attachments: HBASE-21155.branch-2.1.001.patch, > HBASE-21155.branch-2.1.002.patch, HBASE-21155.branch-2.1.002.patch > > > Trivial change to splitlogmanager that saves us a log line at least per WAL > dir when it goes to split. Also saves some not-needed churn in SLM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605238#comment-16605238 ] stack commented on HBASE-21156: --- [~busbey] Those don't block startup so should be fixable post launch... But let me make sure. > [hbck2] Queue an assign of hbase:meta > - > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605232#comment-16605232 ] Sean Busbey commented on HBASE-21156: - Maybe system tables generally? like rsgroup? visibility labels? > [hbck2] Queue an assign of hbase:meta > - > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21001) ReplicationObserver fails to load in HBase 2.0.0
[
https://issues.apache.org/jira/browse/HBASE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605230#comment-16605230
]
Guangxu Cheng commented on HBASE-21001:
---
All tests pass locally. Checked the test report, these failed tests are
unrelated to this jira.
So if no objections I will commit the patch to branch-2.0+ later.Thanks
> ReplicationObserver fails to load in HBase 2.0.0
>
>
> Key: HBASE-21001
> URL: https://issues.apache.org/jira/browse/HBASE-21001
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4, 2.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Guangxu Cheng
>Priority: Major
> Labels: replication
> Attachments: HBASE-21001.master.001.patch,
> HBASE-21001.master.001.patch
>
>
> ReplicationObserver was added in HBASE-17290 to prevent "Potential loss of
> data for replication of bulk loaded hfiles".
> I tried to enable bulk loading replication feature
> (hbase.replication.bulkload.enabled=true and configure
> hbase.replication.cluster.id) on a HBase 2.0.0 cluster, but the RegionServer
> started with the following error:
> {quote}
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: System
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 INFO
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost: Table
> coprocessor loading is enabled
> 2018-08-02 18:20:36,365 ERROR
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationObserver is not
> of type
> RegionServerCoprocessor. Check the configuration of
> hbase.coprocessor.regionserver.classes
> 2018-08-02 18:20:36,366 ERROR
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Cannot load coprocessor
> ReplicationObserver
> {quote}
> It looks like this was broken by HBASE-17732 to me, but I could be wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21143) Update findbugs-maven-plugin to 3.0.4
[
https://issues.apache.org/jira/browse/HBASE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605225#comment-16605225
]
Guangxu Cheng commented on HBASE-21143:
---
[~busbey] [~mdrob] mind taking a look at it ? thanks
> Update findbugs-maven-plugin to 3.0.4
> -
>
> Key: HBASE-21143
> URL: https://issues.apache.org/jira/browse/HBASE-21143
> Project: HBase
> Issue Type: Bug
> Components: pom
>Affects Versions: 3.0.0, 2.1.0, 2.2.0, 2.0.2
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21143.master.001.patch
>
>
> {code}
> Failed to execute goal org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs
> (default) on project hbase: Execution default of goal
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs failed: Plugin
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0 or one of its dependencies
> could not be resolved: Failed to collect dependencies at
> org.codehaus.mojo:findbugs-maven-plugin:jar:3.0.0 ->
> org.codehaus.groovy:groovy-all:jar:1.7.4: Failed to read artifact descriptor
> for org.codehaus.groovy:groovy-all:jar:1.7.4: Could not transfer artifact
> org.codehaus.groovy:groovy-all:pom:1.7.4 from/to mirror
> (http://xxx..xxx/nexus/content/groups/public): Failed to transfer file:
> http://xxx..xxx/nexus/content/groups/public/org/codehaus/groovy/groovy-all/1.7.4/groovy-all-1.7.4.pom.
> Return code is: 418 , ReasonPhrase:Artifact is in Tencent Blacklist! Please
> update to the safe version, more information:
> http://xxx..xxx/?tab=blackList.
> {code}
> Recently, when I compile HBase with a new machine, I got the above error.
> Since the machine could not connect to the external network, we visited our
> internal Maven repository, but org.codehaus.groovy:groovy-all:jar:1.7.4 was
> added to the blacklist and could not be downloaded. See details,
> org.codehaus.groovy:groovy-all:jar:1.7.4 is marked as vulnerable by
> [CVE-2015-3253|https://www.cvedetails.com/cve/CVE-2015-3253], so we should
> upgrade the version.
> {code:xml}
> org.codehaus.mojo
> findbugs-maven-plugin
> 3.0.0
>
>
>
> ${project.basedir}/../dev-support/findbugs-exclude.xml
> {code}
> Look at the history commit record, findbugs-maven-plugin has been upgraded to
> 3.0.4 in HBASE-18264, but one place is missing which still using the version
> of 3.0.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605224#comment-16605224
]
Hadoop QA commented on HBASE-21138:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 50s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
11s{color} | {color:red} hbase-server: The patch generated 1 new + 28 unchanged
- 23 fixed = 29 total (was 51) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}218m
18s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}261m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21138 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938557/HBASE-21138.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a85bbe4ba9f8 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15
17:03:53 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bdc168713d |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/14323/artifact/patchprocess/diff-compile-javac-hbase-server.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14323/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-21157) Split TableInputFormatScan to individual tests
[ https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605222#comment-16605222 ] Duo Zhang commented on HBASE-21157: --- Most works are just copy-paste. > Split TableInputFormatScan to individual tests > -- > > Key: HBASE-21157 > URL: https://issues.apache.org/jira/browse/HBASE-21157 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Minor > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21157.patch > > > We have done a split in HBASE-8326, which split the test to two parts. But it > is still a bit slow, split it into several tests can increase the parallelism > and make the 'mvn test' run faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21157) Split TableInputFormatScan to individual tests
[ https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21157: -- Fix Version/s: 2.0.3 2.1.1 2.2.0 3.0.0 Status: Patch Available (was: Open) > Split TableInputFormatScan to individual tests > -- > > Key: HBASE-21157 > URL: https://issues.apache.org/jira/browse/HBASE-21157 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Minor > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21157.patch > > > We have done a split in HBASE-8326, which split the test to two parts. But it > is still a bit slow, split it into several tests can increase the parallelism > and make the 'mvn test' run faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21157) Split TableInputFormatScan to individual tests
[ https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21157: -- Attachment: HBASE-21157.patch > Split TableInputFormatScan to individual tests > -- > > Key: HBASE-21157 > URL: https://issues.apache.org/jira/browse/HBASE-21157 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Minor > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21157.patch > > > We have done a split in HBASE-8326, which split the test to two parts. But it > is still a bit slow, split it into several tests can increase the parallelism > and make the 'mvn test' run faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21157) Split TableInputFormatScan to individual tests
[ https://issues.apache.org/jira/browse/HBASE-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21157: -- Issue Type: Improvement (was: Bug) > Split TableInputFormatScan to individual tests > -- > > Key: HBASE-21157 > URL: https://issues.apache.org/jira/browse/HBASE-21157 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Minor > > We have done a split in HBASE-8326, which split the test to two parts. But it > is still a bit slow, split it into several tests can increase the parallelism > and make the 'mvn test' run faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21157) Split TableInputFormatScan to individual tests
Duo Zhang created HBASE-21157: - Summary: Split TableInputFormatScan to individual tests Key: HBASE-21157 URL: https://issues.apache.org/jira/browse/HBASE-21157 Project: HBase Issue Type: Bug Components: test Reporter: Duo Zhang Assignee: Duo Zhang We have done a split in HBASE-8326, which split the test to two parts. But it is still a bit slow, split it into several tests can increase the parallelism and make the 'mvn test' run faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21091) Update Hadoop compatibility table
[
https://issues.apache.org/jira/browse/HBASE-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605205#comment-16605205
]
Josh Elser commented on HBASE-21091:
{quote}They may have started out in anticipation of problems, but we've already
had reports of failures and told folks to stick to Java 8 as a result. (e.g.
HBASE-21109) We need stronger language than "we're not sure".
{quote}
Great context, thanks for sharing. Would agree with you. I'll try to remember
to incorporate that when I get the time to pick up this PDF debugging.
> Update Hadoop compatibility table
> -
>
> Key: HBASE-21091
> URL: https://issues.apache.org/jira/browse/HBASE-21091
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-20264.001.patch, HBASE-20264.002.patch,
> hbase-20264-emoji.png, hbase-20264-html.png
>
>
> [https://lists.apache.org/thread.html/7016d322a07e96dccdb071041c37238e43d3df4f93e9515d52ccfafc@%3Cdev.hbase.apache.org%3E]
> covers some discussion around our Hadoop Version Compatibility table.
>
> A "leading" suggestion to make this more clear is to use a green/yellow/red
> (traffic-signal) style marking, instead of using specifics words/phrases (as
> they're often dependent on the interpretation of the reader).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605198#comment-16605198
]
Hadoop QA commented on HBASE-21098:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
13s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} hbase-server: The patch generated 0 new + 283
unchanged - 1 fixed = 283 total (was 284) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
15s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}129m 57s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 28s{color}
| {color:red}
[jira] [Commented] (HBASE-15728) Add remaining per-table region / store / flush / compaction related metrics
[ https://issues.apache.org/jira/browse/HBASE-15728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605187#comment-16605187 ] Sean Busbey commented on HBASE-15728: - this jira got reverted on branch-1 yesterday, which is why another nightly result showed up. I believe the plan is to incorporate the changes in this jira for branch-1 as a part of HBASE-21140. > Add remaining per-table region / store / flush / compaction related metrics > > > Key: HBASE-15728 > URL: https://issues.apache.org/jira/browse/HBASE-15728 > Project: HBase > Issue Type: Sub-task > Components: metrics >Reporter: Enis Soztutar >Assignee: Xu Cang >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.2.0 > > Attachments: HBASE-15728.branch-1.001.patch, > HBASE-15728.branch-1.001.patch, HBASE-15728.branch-1.002.patch, > HBASE-15728.master.001.patch, hbase-15728_v1.patch > > > Continuing on the work for per-table metrics, HBASE-15518 and HBASE-15671. > We need to add some remaining metrics at the per-table level, so that we will > have the same metrics reported at the per-regionserver, per-region and > per-table levels. > After this patch, most of the metrics at the RS and all of the per-region > level are also reported at the per-table level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[
https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605184#comment-16605184
]
Hadoop QA commented on HBASE-21155:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
31s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
15s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 41s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}179m 37s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}238m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
| | hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21155 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938551/HBASE-21155.branch-2.1.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 78b7352a866d 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.1 / 4d7221a68f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605182#comment-16605182
]
Duo Zhang commented on HBASE-21154:
---
What about the acl table? Do we want to fold into meta table too?
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21091) Update Hadoop compatibility table
[
https://issues.apache.org/jira/browse/HBASE-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605165#comment-16605165
]
Sean Busbey commented on HBASE-21091:
-
{quote}
bq. the java 9 and 10 columns should be , no?
Good question: my interpretation was that we warned against them (because
they're non LTS), but we don't know of any discrete problems in using them.
{quote}
They may have started out in anticipation of problems, but we've already had
reports of failures and told folks to stick to Java 8 as a result. (e.g.
HBASE-21109) We need stronger language than "we're not sure".
> Update Hadoop compatibility table
> -
>
> Key: HBASE-21091
> URL: https://issues.apache.org/jira/browse/HBASE-21091
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-20264.001.patch, HBASE-20264.002.patch,
> hbase-20264-emoji.png, hbase-20264-html.png
>
>
> [https://lists.apache.org/thread.html/7016d322a07e96dccdb071041c37238e43d3df4f93e9515d52ccfafc@%3Cdev.hbase.apache.org%3E]
> covers some discussion around our Hadoop Version Compatibility table.
>
> A "leading" suggestion to make this more clear is to use a green/yellow/red
> (traffic-signal) style marking, instead of using specifics words/phrases (as
> they're often dependent on the interpretation of the reader).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605135#comment-16605135
]
Hudson commented on HBASE-21088:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #468 (See
[https://builds.apache.org/job/HBase-1.3-IT/468/])
HBASE-21088 HStoreFile should be closed in HStore#hasReferences (apurtell: rev
36ef36ccc5babbf1e398097855ded3e2e1b9dc34)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 2.1.1, 2.0.2
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure
[
https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605134#comment-16605134
]
Vladimir Rodionov commented on HBASE-21149:
---
Yes, it seems that the reason for a failure is test's time out. I can confirm,
that despite the optimization we have in TestBackupBase, HBase cluster starts
up and shuts down in every unit test we run because, maven launches new JVM per
test class. We need to reuse JVM to shorten execution time.
> TestIncrementalBackupWithBulkLoad may fail due to file copy failure
> ---
>
> Key: HBASE-21149
> URL: https://issues.apache.org/jira/browse/HBASE-21149
> Project: HBase
> Issue Type: Test
> Components: backuprestore
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-21149-v1.patch
>
>
> From
> https://builds.apache.org/job/HBase%20Nightly/job/master/471/testReport/junit/org.apache.hadoop.hbase.backup/TestIncrementalBackupWithBulkLoad/TestIncBackupDeleteTable/
> :
> {code}
> 2018-09-03 11:54:30,526 ERROR [Time-limited test]
> impl.TableBackupClient(235): Unexpected Exception : Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> java.io.IOException: Failed copy from
> hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/0f626c66493649daaf84057b8dd71a30_SeqId_205_,hdfs://localhost:53075/user/jenkins/test-data/ecd40bd0-cb93-91e0-90b5-7bfd5bb2c566/data/default/test-1535975627781/773f5709b645b46bd3840f9cfb549c5a/f/ad8df6415bd9459d9b3df76c588d79df_SeqId_205_
> to hdfs://localhost:53075/backupUT/backup_1535975655488
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.incrementalCopyHFiles(IncrementalTableBackupClient.java:351)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.copyBulkLoadedFiles(IncrementalTableBackupClient.java:219)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.handleBulkLoad(IncrementalTableBackupClient.java:198)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:320)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:605)
> at
> org.apache.hadoop.hbase.backup.TestIncrementalBackupWithBulkLoad.TestIncBackupDeleteTable(TestIncrementalBackupWithBulkLoad.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> {code}
> However, some part of the test output was lost:
> {code}
> 2018-09-03 11:53:36,793 DEBUG [RS:0;765c9ca5ea28:36357] regions
> ...[truncated 398396 chars]...
> 8)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21088) HStoreFile should be closed in HStore#hasReferences
[
https://issues.apache.org/jira/browse/HBASE-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21088:
---
Fix Version/s: 1.4.8
1.3.3
1.5.0
> HStoreFile should be closed in HStore#hasReferences
> ---
>
> Key: HBASE-21088
> URL: https://issues.apache.org/jira/browse/HBASE-21088
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.8, 2.1.1, 2.0.2
>
> Attachments: 21088.v1.txt, 21088.v2.txt, 21088.v2.txt, 21088.v3.txt,
> 21088.v4.txt
>
>
> {code}
> reloadedStoreFiles = loadStoreFiles();
> return StoreUtils.hasReferences(reloadedStoreFiles);
> {code}
> The intention of obtaining the HStoreFile's is to check for references.
> The loaded HStoreFile's should be closed prior to return to prevent leak.
> I noticed the increase in open files when running test suite. After checking
> recently modified code, I came to this particular method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21156) [hbck2] Queue an assign of hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21156: -- Priority: Critical (was: Major) > [hbck2] Queue an assign of hbase:meta > - > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21035) Meta Table should be able to online even if all procedures are lost
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21035: -- Status: In Progress (was: Patch Available) > Meta Table should be able to online even if all procedures are lost > --- > > Key: HBASE-21035 > URL: https://issues.apache.org/jira/browse/HBASE-21035 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.1.0 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Attachments: HBASE-21035.branch-2.0.001.patch > > > After HBASE-20708, we changed the way we init after master starts. It will > only check WAL dirs and compare to Zookeeper RS nodes to decide which server > need to expire. For servers which's dir is ending with 'SPLITTING', we assure > that there will be a SCP for it. > But, if the server with the meta region crashed before master restarts, and > if all the procedure wals are lost (due to bug, or deleted manually, > whatever), the new restarted master will be stuck when initing. Since no one > will bring meta region online. > Although it is an anomaly case, but I think no matter what happens, we need > to online meta region. Otherwise, we are sitting ducks, noting can be done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605123#comment-16605123 ] stack commented on HBASE-21035: --- Late to the party I've been playing with removing the procedure WALs and doing other damage to the cluster to see how well we recover. I ran into this issue here that [~allan163] talks of where there is no assign for meta region on startup; in my case, I'd removed the procedure WAL dirs as Allan does in his test but different from Allan, there was no WAL dir for the server that had been carrying meta; I think it'd been removed across restarts (I didn't check) but I was able to repro by removing the empty WAL dir manually (empty because there'd been a clean shutdown). After reading the above healthy back and forth, while the system seems pretty robust as is -- I had trouble breaking it removing stuff -- and Allan's patch would catch a particular case not covered now, I agree that we need the "assign meta" fix-it in our hbck2 vocabulary. Let me add scheduling of a meta assign (and log search and recovery) as a hbck2 option to the list of fix-its we need in hbck2 as we discussed in person a few weeks back. In my investigations, it seems like we need similar for hbase:namespace table. It can get banjaxed similarly and if not online the cluster is a mess. Master initialization gets stuck trying to read from meta to populate the TableStates. It later gets stuck trying to initialize the TableNamespaceManager if the namespace table is not online. I filed HBASE-21156 > Meta Table should be able to online even if all procedures are lost > --- > > Key: HBASE-21035 > URL: https://issues.apache.org/jira/browse/HBASE-21035 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.1.0 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Attachments: HBASE-21035.branch-2.0.001.patch > > > After HBASE-20708, we changed the way we init after master starts. It will > only check WAL dirs and compare to Zookeeper RS nodes to decide which server > need to expire. For servers which's dir is ending with 'SPLITTING', we assure > that there will be a SCP for it. > But, if the server with the meta region crashed before master restarts, and > if all the procedure wals are lost (due to bug, or deleted manually, > whatever), the new restarted master will be stuck when initing. Since no one > will bring meta region online. > Although it is an anomaly case, but I think no matter what happens, we need > to online meta region. Otherwise, we are sitting ducks, noting can be done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605119#comment-16605119 ] stack commented on HBASE-21156: --- Oh, may need to do same for hbase:namespace. > [hbck2] Queue an assign of hbase:meta > - > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.1.1 > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605122#comment-16605122
]
Josh Elser commented on HBASE-20952:
{quote}I'd think this is how we'd lead off? In a write-up. The shiny new WAL
interface that has had all FS/Path/File purged from it. Tah-dah!
{quote}
{quote}A write-up that talked up how existing Provider notion is being
doubled-down on would help with navigation.
{quote}
{quote}the high-level design was more ecumenical
{quote}
{quote}I defer to you fellows that are doing the hard work. I'd like to benefit
from what you've learned though from reviewing existing implementations. I
don't think I will get this reading a fat patch only.
{quote}
Fantastic. Thanks much, Stack, for penning this. I think this is all "in
vision". We can draw a better line between things that we need to build in
HBase and things that a WAL would provide.
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21156) [hbck2] Queue an assign of hbase:meta
stack created HBASE-21156: - Summary: [hbck2] Queue an assign of hbase:meta Key: HBASE-21156 URL: https://issues.apache.org/jira/browse/HBASE-21156 Project: HBase Issue Type: Sub-task Components: hbck2 Affects Versions: 2.1.0 Reporter: stack Assignee: stack Fix For: 2.1.1 We need this to effect repair when damage. If procedure WALs AND a server WAL dir are lost or cleaned or we crashed during partial split (unlikely scenarios but nonetheless possible), a Master can be stuck unable to become active because there is no assign procedure for hbase:meta in the system. The reasonable argument over in HBASE-21035 has it that attempts at auto-repair under these extremes could cause other issues so at least until we learn more, we for now punt to the operator for fix-up. To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s test). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21091) Update Hadoop compatibility table
[
https://issues.apache.org/jira/browse/HBASE-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605113#comment-16605113
]
Josh Elser commented on HBASE-21091:
{quote}the java 9 and 10 columns should be (x), no?
{quote}
Good question: my interpretation was that we warned against them (because
they're non LTS), but we don't know of any discrete problems in using them.
{quote}both the yellow warning sign and the bad rendering of the pdf smell like
font problems, fwiw.
{quote}
Interesting. At least the formatting of the content isn't borked like they were
with the SVGs. I'm out of my element to debug that, but I'll have to google and
see what I can find..
> Update Hadoop compatibility table
> -
>
> Key: HBASE-21091
> URL: https://issues.apache.org/jira/browse/HBASE-21091
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-20264.001.patch, HBASE-20264.002.patch,
> hbase-20264-emoji.png, hbase-20264-html.png
>
>
> [https://lists.apache.org/thread.html/7016d322a07e96dccdb071041c37238e43d3df4f93e9515d52ccfafc@%3Cdev.hbase.apache.org%3E]
> covers some discussion around our Hadoop Version Compatibility table.
>
> A "leading" suggestion to make this more clear is to use a green/yellow/red
> (traffic-signal) style marking, instead of using specifics words/phrases (as
> they're often dependent on the interpretation of the reader).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[
https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605110#comment-16605110
]
Hadoop QA commented on HBASE-20226:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
15s{color} | {color:red} hbase-server: The patch generated 13 new + 5 unchanged
- 0 fixed = 18 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 41s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
12s{color} | {color:red} hbase-server generated 3 new + 0 unchanged - 0 fixed =
3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 21m 51s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 64m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | v2Regions could be null and is guaranteed to be dereferenced in
org.apache.hadoop.hbase.snapshot.SnapshotManifest.convertToV2SingleManifest()
Dereferenced at SnapshotManifest.java:is guaranteed to be dereferenced in
org.apache.hadoop.hbase.snapshot.SnapshotManifest.convertToV2SingleManifest()
Dereferenced at SnapshotManifest.java:[line 516] |
| | Possible null pointer dereference of v1Regions in
org.apache.hadoop.hbase.snapshot.SnapshotManifest.convertToV2SingleManifest()
Dereferenced at SnapshotManifest.java:v1Regions in
org.apache.hadoop.hbase.snapshot.SnapshotManifest.convertToV2SingleManifest()
Dereferenced at SnapshotManifest.java:[line 516] |
| | Nullcheck of v1Regions at line 516 of value previously dereferenced in
org.apache.hadoop.hbase.snapshot.SnapshotManifest.convertToV2SingleManifest()
At SnapshotManifest.java:516 of value previously dereferenced in
[jira] [Updated] (HBASE-21140) Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to branch-1 . (and backport HBASE-15728 for branch-1)
[ https://issues.apache.org/jira/browse/HBASE-21140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-21140: Status: In Progress (was: Patch Available) > Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to > branch-1 . (and backport HBASE-15728 for branch-1) > --- > > Key: HBASE-21140 > URL: https://issues.apache.org/jira/browse/HBASE-21140 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Duo Zhang >Assignee: Xu Cang >Priority: Major > Fix For: 1.5.0 > > Attachments: > HBASE-21140.diff_against_cf198a65e8d704d28538c4c165a941b9e5bac678.branch-1.001.patch > > > There is no computeIfAbsent method on branch-1 as we still need to support > JDK7, so the fix will be different with branch-2+. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21140) Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to branch-1 . (and backport HBASE-15728 for branch-1)
[ https://issues.apache.org/jira/browse/HBASE-21140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605098#comment-16605098 ] Andrew Purtell commented on HBASE-21140: Yes, please leave HBASE-21150 to a separate issue. We are already combining HBASE-15728 and the NPE fix > Backport 'HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime' to > branch-1 . (and backport HBASE-15728 for branch-1) > --- > > Key: HBASE-21140 > URL: https://issues.apache.org/jira/browse/HBASE-21140 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Duo Zhang >Assignee: Xu Cang >Priority: Major > Fix For: 1.5.0 > > Attachments: > HBASE-21140.diff_against_cf198a65e8d704d28538c4c165a941b9e5bac678.branch-1.001.patch > > > There is no computeIfAbsent method on branch-1 as we still need to support > JDK7, so the fix will be different with branch-2+. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21136) NPE in MetricsTableSourceImpl.updateFlushTime
[ https://issues.apache.org/jira/browse/HBASE-21136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605096#comment-16605096 ] Andrew Purtell commented on HBASE-21136: yes, branch-1 patch needs to support JDK 7, thanks. > NPE in MetricsTableSourceImpl.updateFlushTime > - > > Key: HBASE-21136 > URL: https://issues.apache.org/jira/browse/HBASE-21136 > Project: HBase > Issue Type: Bug > Components: metrics >Reporter: Guanghao Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21136-v1.patch, HBASE-21136.patch > > > See https://builds.apache.org/job/PreCommit-HBASE-Build/14260/testReport/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605082#comment-16605082
]
Mingliang Liu commented on HBASE-21138:
---
Thanks, Ted. I will provide a branch-1 patch shortly.
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605079#comment-16605079
]
Ted Yu commented on HBASE-21138:
Planning to commit v3 tomorrow morning, if QA comes back clean.
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20578) Support region server group in target cluster
[ https://issues.apache.org/jira/browse/HBASE-20578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605078#comment-16605078 ] Ted Yu commented on HBASE-20578: Ping [~albertlee...@gmail.com] > Support region server group in target cluster > - > > Key: HBASE-20578 > URL: https://issues.apache.org/jira/browse/HBASE-20578 > Project: HBase > Issue Type: Improvement > Components: Replication, rsgroup >Reporter: Ted Yu >Assignee: Albert Lee >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20578-001.patch > > > When source tables belong to non-default region server group(s) and there are > region server group counterpart in the target cluster, we should support > replicating to target cluster using the region server group mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605070#comment-16605070 ] Zach York commented on HBASE-21070: --- Sorry for being a little absent on this over the past couple days. I found a bug in the original test code [https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java#L261] is calling contains for a set of FileStatus's and trying to find a String which will never return true... I've been trying to wrestle to get the tests to perform as expected, but it has taken more time. I should probably fix it in a follow-up JIRA. > SnapshotFileCache won't update for snapshots stored in S3 > - > > Key: HBASE-21070 > URL: https://issues.apache.org/jira/browse/HBASE-21070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 3.0.0, 2.1.1, 1.4.7 >Reporter: Zach York >Assignee: Zach York >Priority: Critical > Labels: FSRedo > Attachments: HBASE-21070.master.001.patch, > HBASE-21070.master.002.patch > > > The SnapshotFileCache depends on last modified time to determine whether to > update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. > S3 filesystems create a dummy file in place of a folder, but the dummy file > last modified time is not updated when files are changed 'under' it. This > means that the SnapshotFileCache doesn't pick up new snapshot HFiles and > these files aren't removed from the HFileCleaner and can be eligible for > deletion. > > My patch removes the lastmodified assumption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605068#comment-16605068
]
Mingliang Liu commented on HBASE-21098:
---
Thanks [~mtylr]. I guess my suggestion is to compare FS using _scheme_,
_authority_, and _ugi_ instead of {{equals()}} to check if we can rename;
*and* if the comparison is false or FS.rename fails, we fallback to FS.copy.
But I'm fine with {{equals()}} as the copy itself is not costly. The reason we
can not rename without checking the feasibility is that, if the path
{{workingDir}} is not fully qualified URI (e.g. "/user/bob/snap") but is
actually on different FS (e.g. hdfs), the root FS.rename("/user/bob/snap",
"s3a://bucket/hbase/.snapshot/snap1") does not make sense but may still succeed.
Otherwise +1 (non-binding).
> Improve Snapshot Performance with Temporary Snapshot Directory when rootDir
> on S3
> -
>
> Key: HBASE-21098
> URL: https://issues.apache.org/jira/browse/HBASE-21098
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.4.8, 2.1.1
>Reporter: Tyler Mi
>Priority: Major
> Attachments: HBASE-21098.master.001.patch,
> HBASE-21098.master.002.patch, HBASE-21098.master.003.patch,
> HBASE-21098.master.004.patch, HBASE-21098.master.005.patch,
> HBASE-21098.master.006.patch, HBASE-21098.master.007.patch,
> HBASE-21098.master.008.patch, HBASE-21098.master.009.patch,
> HBASE-21098.master.010.patch, HBASE-21098.master.011.patch
>
>
> When using Apache HBase, the snapshot feature can be used to make a point in
> time recovery. To do this, HBase creates a manifest of all the files in all
> of the Regions so that those files can be referenced again when a user
> restores a snapshot. With HBase's S3 storage mode, developers can store their
> data off-cluster on Amazon S3. However, utilizing S3 as a file system is
> inefficient in some operations, namely renames. Most Hadoop ecosystem
> applications use an atomic rename as a method of committing data. However,
> with S3, a rename is a separate copy and then a delete of every file which is
> no longer atomic and, in fact, quite costly. In addition, puts and deletes on
> S3 have latency issues that traditional filesystems do not encounter when
> manipulating the region snapshots to consolidate into a single manifest. When
> HBase on S3 users have a significant amount of regions, puts, deletes, and
> renames (the final commit stage of the snapshot) become the bottleneck
> causing snapshots to take many minutes or even hours to complete.
> The purpose of this patch is to increase the overall performance of snapshots
> while utilizing HBase on S3 through the use of a temporary directory for the
> snapshots that exists on a traditional filesystem like HDFS to circumvent the
> bottlenecks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HBASE-21138:
--
Attachment: HBASE-21138.003.patch
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[ https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605053#comment-16605053 ] Zach York commented on HBASE-20226: --- This is related to the work going on in HBASE-21098. It's likely that this Jira is no longer needed after that work is concluded. > Performance Improvement Taking Large Snapshots In Remote Filesystems > > > Key: HBASE-20226 > URL: https://issues.apache.org/jira/browse/HBASE-20226 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 1.4.0 > Environment: HBase 1.4.0 running on an AWS EMR cluster with the > hbase.rootdir set to point to a folder in S3 >Reporter: Saad Mufti >Priority: Minor > Attachments: HBASE-20226..01.patch > > > When taking a snapshot of any table, one of the last steps is to delete the > region manifests, which have already been rolled up into a larger overall > manifest and thus have redundant information. > This proposal is to do the deletion in a thread pool bounded by > hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the > current single threaded deletion is taking longer than all the rest of the > snapshot tasks when the Hbase data and the snapshot folder are both in a > remote filesystem like S3. > I have a patch for this proposal almost ready and will submit it tomorrow for > feedback, although I haven't had a chance to write any tests yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21107) add a metrics for netty direct memory
[ https://issues.apache.org/jira/browse/HBASE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-21107: - Resolution: Fixed Fix Version/s: 2.2.0 3.0.0 Release Note: Add a new nettyDirectMemoryUsage under server's ipc metrics to show direct memory usage for netty rpc server. Status: Resolved (was: Patch Available) > add a metrics for netty direct memory > - > > Key: HBASE-21107 > URL: https://issues.apache.org/jira/browse/HBASE-21107 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC >Affects Versions: 2.0.1 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21107-master-v1.patch, > HBASE-21107-master-v1.patch, HBASE-21107-master-v2.patch, > HBASE-21107-master-v2.patch, HBASE-21107-master-v2.patch, > HBASE-21107-master-v3.patch, Screen Shot 2018-08-23 at 4.55.01 PM.png > > > This is separated from HBASE-20913 as I realized that netty direct memory > usage needs to show up in metrics instead of the web page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[ https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605051#comment-16605051 ] Tyler Mi commented on HBASE-21098: -- Good point! I also realized that in actuality, the way the snapshot process works, the temporary directory will only have two files in it. Thus, if it defaulted to copying in every case, that would also be acceptable as it would be a "constant" operation. The size of the file does depend on the number of regions in the table being snapshotted, but it is not too much overhead to copy vs rename. Thank you for all the discussion! > Improve Snapshot Performance with Temporary Snapshot Directory when rootDir > on S3 > - > > Key: HBASE-21098 > URL: https://issues.apache.org/jira/browse/HBASE-21098 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 1.4.8, 2.1.1 >Reporter: Tyler Mi >Priority: Major > Attachments: HBASE-21098.master.001.patch, > HBASE-21098.master.002.patch, HBASE-21098.master.003.patch, > HBASE-21098.master.004.patch, HBASE-21098.master.005.patch, > HBASE-21098.master.006.patch, HBASE-21098.master.007.patch, > HBASE-21098.master.008.patch, HBASE-21098.master.009.patch, > HBASE-21098.master.010.patch, HBASE-21098.master.011.patch > > > When using Apache HBase, the snapshot feature can be used to make a point in > time recovery. To do this, HBase creates a manifest of all the files in all > of the Regions so that those files can be referenced again when a user > restores a snapshot. With HBase's S3 storage mode, developers can store their > data off-cluster on Amazon S3. However, utilizing S3 as a file system is > inefficient in some operations, namely renames. Most Hadoop ecosystem > applications use an atomic rename as a method of committing data. However, > with S3, a rename is a separate copy and then a delete of every file which is > no longer atomic and, in fact, quite costly. In addition, puts and deletes on > S3 have latency issues that traditional filesystems do not encounter when > manipulating the region snapshots to consolidate into a single manifest. When > HBase on S3 users have a significant amount of regions, puts, deletes, and > renames (the final commit stage of the snapshot) become the bottleneck > causing snapshots to take many minutes or even hours to complete. > The purpose of this patch is to increase the overall performance of snapshots > while utilizing HBase on S3 through the use of a temporary directory for the > snapshots that exists on a traditional filesystem like HDFS to circumvent the > bottlenecks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21107) add a metrics for netty direct memory
[ https://issues.apache.org/jira/browse/HBASE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605050#comment-16605050 ] huaxiang sun commented on HBASE-21107: -- pushed the patch to branch-2 and master. [~stack], please let me know if this is needed for 2.0 and 2.1, I can reopen and do the push. > add a metrics for netty direct memory > - > > Key: HBASE-21107 > URL: https://issues.apache.org/jira/browse/HBASE-21107 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC >Affects Versions: 2.0.1 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-21107-master-v1.patch, > HBASE-21107-master-v1.patch, HBASE-21107-master-v2.patch, > HBASE-21107-master-v2.patch, HBASE-21107-master-v2.patch, > HBASE-21107-master-v3.patch, Screen Shot 2018-08-23 at 4.55.01 PM.png > > > This is separated from HBASE-20913 as I realized that netty direct memory > usage needs to show up in metrics instead of the web page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605048#comment-16605048 ] Zach York commented on HBASE-20734: --- That'd be great, thanks [~reidchan]! > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.branch-1.002.patch, HBASE-20734.branch-1.003.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch, > HBASE-20734.master.003.patch, HBASE-20734.master.004.patch, > HBASE-20734.master.005.patch, HBASE-20734.master.006.patch, > HBASE-20734.master.007.patch, HBASE-20734.master.008.patch, > HBASE-20734.master.009.patch, HBASE-20734.master.010.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[ https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Mi updated HBASE-21098: - Attachment: HBASE-21098.master.011.patch > Improve Snapshot Performance with Temporary Snapshot Directory when rootDir > on S3 > - > > Key: HBASE-21098 > URL: https://issues.apache.org/jira/browse/HBASE-21098 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 1.4.8, 2.1.1 >Reporter: Tyler Mi >Priority: Major > Attachments: HBASE-21098.master.001.patch, > HBASE-21098.master.002.patch, HBASE-21098.master.003.patch, > HBASE-21098.master.004.patch, HBASE-21098.master.005.patch, > HBASE-21098.master.006.patch, HBASE-21098.master.007.patch, > HBASE-21098.master.008.patch, HBASE-21098.master.009.patch, > HBASE-21098.master.010.patch, HBASE-21098.master.011.patch > > > When using Apache HBase, the snapshot feature can be used to make a point in > time recovery. To do this, HBase creates a manifest of all the files in all > of the Regions so that those files can be referenced again when a user > restores a snapshot. With HBase's S3 storage mode, developers can store their > data off-cluster on Amazon S3. However, utilizing S3 as a file system is > inefficient in some operations, namely renames. Most Hadoop ecosystem > applications use an atomic rename as a method of committing data. However, > with S3, a rename is a separate copy and then a delete of every file which is > no longer atomic and, in fact, quite costly. In addition, puts and deletes on > S3 have latency issues that traditional filesystems do not encounter when > manipulating the region snapshots to consolidate into a single manifest. When > HBase on S3 users have a significant amount of regions, puts, deletes, and > renames (the final commit stage of the snapshot) become the bottleneck > causing snapshots to take many minutes or even hours to complete. > The purpose of this patch is to increase the overall performance of snapshots > while utilizing HBase on S3 through the use of a temporary directory for the > snapshots that exists on a traditional filesystem like HDFS to circumvent the > bottlenecks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605043#comment-16605043
]
Zach York commented on HBASE-21154:
---
I have a small concern (not with the actual change, but with the symptom). The
goal here is to avoid having to handle startup ordering, correct? If we split
meta or introduce a new system table, we will still have to handle the startup
ordering eventually.
Probably not a huge concern here, but how much will this increase the size of
hbase meta? (above what the namespace table used to be)
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605012#comment-16605012
]
Ted Yu commented on HBASE-21138:
Thanks for the new patch.
Can you take a look at
https://builds.apache.org/job/PreCommit-HBASE-Build/14318/artifact/patchprocess/diff-checkstyle-hbase-server.txt
to see if any warning applies to the modified test ?
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[ https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21155: -- Fix Version/s: 2.1.1 Affects Version/s: 2.1.0 Status: Patch Available (was: Open) > Save on a few log strings and some churn in wal splitter by skipping out > early if no logs in dir > > > Key: HBASE-21155 > URL: https://issues.apache.org/jira/browse/HBASE-21155 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Trivial > Fix For: 2.1.1 > > Attachments: HBASE-21155.branch-2.1.001.patch, > HBASE-21155.branch-2.1.002.patch > > > Trivial change to splitlogmanager that saves us a log line at least per WAL > dir when it goes to split. Also saves some not-needed churn in SLM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[ https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21155: -- Attachment: HBASE-21155.branch-2.1.002.patch > Save on a few log strings and some churn in wal splitter by skipping out > early if no logs in dir > > > Key: HBASE-21155 > URL: https://issues.apache.org/jira/browse/HBASE-21155 > Project: HBase > Issue Type: Improvement >Reporter: stack >Assignee: stack >Priority: Trivial > Attachments: HBASE-21155.branch-2.1.001.patch, > HBASE-21155.branch-2.1.002.patch > > > Trivial change to splitlogmanager that saves us a log line at least per WAL > dir when it goes to split. Also saves some not-needed churn in SLM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
[ https://issues.apache.org/jira/browse/HBASE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21155: -- Attachment: HBASE-21155.branch-2.1.001.patch > Save on a few log strings and some churn in wal splitter by skipping out > early if no logs in dir > > > Key: HBASE-21155 > URL: https://issues.apache.org/jira/browse/HBASE-21155 > Project: HBase > Issue Type: Improvement >Reporter: stack >Assignee: stack >Priority: Trivial > Attachments: HBASE-21155.branch-2.1.001.patch, > HBASE-21155.branch-2.1.002.patch > > > Trivial change to splitlogmanager that saves us a log line at least per WAL > dir when it goes to split. Also saves some not-needed churn in SLM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21155) Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
stack created HBASE-21155: - Summary: Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir Key: HBASE-21155 URL: https://issues.apache.org/jira/browse/HBASE-21155 Project: HBase Issue Type: Improvement Reporter: stack Assignee: stack Trivial change to splitlogmanager that saves us a log line at least per WAL dir when it goes to split. Also saves some not-needed churn in SLM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605001#comment-16605001
]
Hadoop QA commented on HBASE-21138:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 55s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} hbase-server: The patch generated 1 new + 28 unchanged
- 23 fixed = 29 total (was 51) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}187m
34s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}228m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21138 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938518/HBASE-21138.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2de24793f499 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bb657c2d2e |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/14318/artifact/patchprocess/diff-compile-javac-hbase-server.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14318/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Resolution: Fixed
Release Note:
Client facing artifacts are now built whenever Maven is run through the
"package" goal. Previously, the client facing artifacts would create
placeholder jars that skipped repackaging HBase and third-party dependencies
unless the "release" profile was active.
Build times may be noticeably longer depending on your build hardware. For
example, the Jenkins worker nodes maintained by ASF Infra take ~14% longer to
do a full packaging build. An example portability-focused personal laptop took
~25% longer.
Status: Resolved (was: Patch Available)
Let me know if the RN isn't loud enough (or if it's too loud).
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Issue Type: Improvement (was: Bug)
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604993#comment-16604993
]
Sean Busbey commented on HBASE-21154:
-
bq. Could also assert that there is no system table but meta.
This would be a big change, no? impacts our ability to clean up persistent
state that's in ZK. thought I had heard chatter of moving that stuff into an
optional system table.
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604987#comment-16604987
]
stack commented on HBASE-21154:
---
bq. also could we document some considerations for the future about how we pick
"new CF in meta" versus "table in the hbase namespace"?
bq. it sounds like "master needs it to start" -> "CF in meta"? anything else
goes in a table, given our scaling constraints on meta?
Sure.
Could also assert that there is no system table but meta.
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604980#comment-16604980
]
stack commented on HBASE-21154:
---
[~mdrob] One thought is that adding to hbase:meta schema is tough given its
hard-coded at the moment (to be fixed so we can split meta). Was thinking we
could put namespace in the new-to-branch-2 'table' column-family. Currently it
has a single column, 'table:state'. Could add a column 'table:namepace'. This
is an ugly hack but better than a table that hangs out in assignment purgatory.
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604970#comment-16604970
]
stack commented on HBASE-21153:
---
+1
Needs RN shouting about longer build time but I'd doubt anyone will notice
(smile).
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20786) Table create with thousands of regions takes too long
[
https://issues.apache.org/jira/browse/HBASE-20786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604967#comment-16604967
]
stack commented on HBASE-20786:
---
[~mdrob] Thanks for taking a look. Can it be backgrounded? Or piecemealed
rather than done in a lump.
Here is what they are at... (We doing this per store? There are maybe 1M on
this cluster Pity we couldn't bulk it up).
{code}
"region-location-4" #552 daemon prio=5 os_prio=0 tid=0x00b43800
nid=0x82cb in Object.wait() [0x7f58e8485000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at
org.apache.hadoop.util.concurrent.AsyncGet$Util.wait(AsyncGet.java:59)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1479)
- locked <0x0005d9a77ce0> (a org.apache.hadoop.ipc.Client$Call)
at org.apache.hadoop.ipc.Client.call(Client.java:1437)
at org.apache.hadoop.ipc.Client.call(Client.java:1347)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy21.getListing(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:654)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
- locked <0x0005d9a77bb0> (a
org.apache.hadoop.io.retry.RetryInvocationHandler$Call)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy22.getListing(Unknown Source)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
at com.sun.proxy.$Proxy23.getListing(Unknown Source)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
at com.sun.proxy.$Proxy23.getListing(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1624)
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1142)
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1125)
at
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1070)
at
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1066)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:1084)
at
org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:2039)
at org.apache.hadoop.fs.FileSystem$5.(FileSystem.java:2168)
at org.apache.hadoop.fs.FileSystem.listFiles(FileSystem.java:2165)
at
org.apache.hadoop.hbase.util.CommonFSUtils.listLocatedStatus(CommonFSUtils.java:702)
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFilesLocatedStatus(HRegionFileSystem.java:268)
at
org.apache.hadoop.hbase.regionserver.HRegion.computeHDFSBlocksDistribution(HRegion.java:1202)
at
org.apache.hadoop.hbase.regionserver.HRegion.computeHDFSBlocksDistribution(HRegion.java:1182)
at
org.apache.hadoop.hbase.master.balancer.RegionLocationFinder.internalGetTopBlockLocation(RegionLocationFinder.java:198)
at
org.apache.hadoop.hbase.master.balancer.RegionLocationFinder$1$1.call(RegionLocationFinder.java:81)
at
org.apache.hadoop.hbase.master.balancer.RegionLocationFinder$1$1.call(RegionLocationFinder.java:78)
at
[jira] [Updated] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-21023: - Summary: Add bypassProcedureToCompletion() API to HbckService (was: Add completeProcedure/s() API to HbckService) > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604951#comment-16604951
]
Hadoop QA commented on HBASE-21153:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} hbase-shaded-client-byo-hadoop in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} hbase-shaded-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hbase-shaded-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21153 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938534/HBASE-21153.0.patch |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux 6e03e66c6b08 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cc414bdeab |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14319/testReport/ |
| Max. process+thread count | 87 (vs. ulimit of 1) |
| modules | C: hbase-shaded/hbase-shaded-client-byo-hadoop
hbase-shaded/hbase-shaded-client
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604948#comment-16604948
]
stack commented on HBASE-20952:
---
As asked-for above, we need a write-up on how this work goes against the
high-level design including learnings, decisions (why), cons, and todos. Asking
us to deduce this info, the model, and intent, by trying to parse a four-pager
patch is a lot to ask. The rb text doesn't satisfy my request above nor yours
that followed after mine. For an example of what I'm after, see the write-up on
HBASE-20708
(https://docs.google.com/document/d/1_872oHzrhJq4ck7f6zmp1J--zMhsIFvXSZyX1Mxg5MA/edit#),
a doc. I've since gone back to a few times trying to ensure that any changes
to the affected area align w/ the architects' intent.
On your comments:
* On the lead-off, thanks for starting to fill in what should have been there
but the point was an imprecise lead-of colors the reading of what follows.
* If the implementation is informed by "Apache DistributedLog", I'd like to
learn how. Do I have to read DL code and then this patch and compare to figure
it? How I figure if DL namespace'ing was mapped to hbase regions or not? Any
other learnings from this review of the outstanding literature?
* "I think Ted was just trying to capture this." <= The comment is about
consensus based WAL implementations... Which is an exclusive club. IIRC, the
high-level design was more ecumenical. Are we dropping target impls?
* "Which do you think is better?" <= Either.
* "I'm struggling to peel away a suggestion from the criticism." <= This area
of the codebase has seen heavy accretion. It is overladen w/ Interface,
Abstracts, Impls, and Utils. I see an opportunity for nice cleanup via
application of a strong design. From review of the commentary (only!), it looks
like we are being freighted with a new set or Infos, Abstracts, and Interfaces,
with an "FS" dimension thrown-in. On "Provider" being a core concept already,
I'd forgotten. A write-up that talked up how existing Provider notion is being
doubled-down on would help with navigation.
* "A smaller review in which only the new WAL API is provided?" <= I'd think
this is how we'd lead off? In a write-up. The shiny new WAL interface that has
had all FS/Path/File purged from it. Tah-dah!
* "I think we need to tease this apart." <= Does the patch say this anywhere?
This is a TODO? Follow-on? No sense from the rb comment.
* "Do you have something in mind in how this abstraction should work?" <= I
have ideas but ideas are cheap ("API has an append that returns a sequence id.
Also has a replay that takes an sequenceid and it gives you back all edits at
and after the sequence id. Ideally, could do sequence id by region or group of
regions and ask for replay by group or region."). I defer to you fellows that
are doing the hard work. I'd like to benefit from what you've learned though
from reviewing existing implementations. I don't think I will get this reading
a fat patch only.
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21091) Update Hadoop compatibility table
[ https://issues.apache.org/jira/browse/HBASE-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604939#comment-16604939 ] Sean Busbey commented on HBASE-21091: - oof the pdf does not render well at all: https://builds.apache.org/job/PreCommit-HBASE-Build/14316/artifact/patchprocess/patch-site/apache_hbase_reference_guide.pdf > Update Hadoop compatibility table > - > > Key: HBASE-21091 > URL: https://issues.apache.org/jira/browse/HBASE-21091 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-20264.001.patch, HBASE-20264.002.patch, > hbase-20264-emoji.png, hbase-20264-html.png > > > [https://lists.apache.org/thread.html/7016d322a07e96dccdb071041c37238e43d3df4f93e9515d52ccfafc@%3Cdev.hbase.apache.org%3E] > covers some discussion around our Hadoop Version Compatibility table. > > A "leading" suggestion to make this more clear is to use a green/yellow/red > (traffic-signal) style marking, instead of using specifics words/phrases (as > they're often dependent on the interpretation of the reader). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21091) Update Hadoop compatibility table
[ https://issues.apache.org/jira/browse/HBASE-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604940#comment-16604940 ] Sean Busbey commented on HBASE-21091: - both the yellow warning sign and the bad rendering of the pdf smell like font problems, fwiw. > Update Hadoop compatibility table > - > > Key: HBASE-21091 > URL: https://issues.apache.org/jira/browse/HBASE-21091 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-20264.001.patch, HBASE-20264.002.patch, > hbase-20264-emoji.png, hbase-20264-html.png > > > [https://lists.apache.org/thread.html/7016d322a07e96dccdb071041c37238e43d3df4f93e9515d52ccfafc@%3Cdev.hbase.apache.org%3E] > covers some discussion around our Hadoop Version Compatibility table. > > A "leading" suggestion to make this more clear is to use a green/yellow/red > (traffic-signal) style marking, instead of using specifics words/phrases (as > they're often dependent on the interpretation of the reader). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21091) Update Hadoop compatibility table
[ https://issues.apache.org/jira/browse/HBASE-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604937#comment-16604937 ] Sean Busbey commented on HBASE-21091: - the java 9 and 10 columns should be (x), no? > Update Hadoop compatibility table > - > > Key: HBASE-21091 > URL: https://issues.apache.org/jira/browse/HBASE-21091 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-20264.001.patch, HBASE-20264.002.patch, > hbase-20264-emoji.png, hbase-20264-html.png > > > [https://lists.apache.org/thread.html/7016d322a07e96dccdb071041c37238e43d3df4f93e9515d52ccfafc@%3Cdev.hbase.apache.org%3E] > covers some discussion around our Hadoop Version Compatibility table. > > A "leading" suggestion to make this more clear is to use a green/yellow/red > (traffic-signal) style marking, instead of using specifics words/phrases (as > they're often dependent on the interpretation of the reader). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604903#comment-16604903
]
Sean Busbey commented on HBASE-21154:
-
also could we document some considerations for the future about how we pick
"new CF in meta" versus "table in the hbase namespace"?
it sounds like "master needs it to start" -> "CF in meta"? anything else goes
in a table, given our scaling constraints on meta?
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604901#comment-16604901
]
Sean Busbey commented on HBASE-21154:
-
given our community desire to make future major versions rolling-upgradable,
I'd really like this to target landing in a branch-2 so that we get upgrade
issues sorted sooner rather than later.
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21154:
Component/s: meta
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
> Components: meta
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Component/s: build
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Status: Patch Available (was: In Progress)
-v0
- remove "release" profile from shaded client artifacts
- add shade-plugin to normal build section of shaded client artifacts
tested locally by running {{mvn -DskipTests package assembly:single}} and
inspecting both the shaded client module jars and the resultant assembly
tarballs.
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.1.0, 3.0.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Attachment: HBASE-21153.0.patch
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21153.0.patch, ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18974) Document "Becoming a Committer"
[ https://issues.apache.org/jira/browse/HBASE-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18974: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) done. thanks for the work [~mdrob] and [~misty] > Document "Becoming a Committer" > --- > > Key: HBASE-18974 > URL: https://issues.apache.org/jira/browse/HBASE-18974 > Project: HBase > Issue Type: Bug > Components: community, documentation >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-18974-copyedit-addendum.patch, HBASE-18974.patch, > HBASE-18974.v2.patch, HBASE-18974.v3.patch, HBASE-18974.v4.patch > > > Based on the mailing list discussion at > https://lists.apache.org/thread.html/81c633cbe1f6f78421cbdad5b9549643c67803a723a9d86a513264c0@%3Cdev.hbase.apache.org%3E > it sounds like we should record some of the thoughts for future contributors > to refer to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18974) Document "Becoming a Committer"
[ https://issues.apache.org/jira/browse/HBASE-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18974: Issue Type: Improvement (was: Bug) > Document "Becoming a Committer" > --- > > Key: HBASE-18974 > URL: https://issues.apache.org/jira/browse/HBASE-18974 > Project: HBase > Issue Type: Improvement > Components: community, documentation >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-18974-copyedit-addendum.patch, HBASE-18974.patch, > HBASE-18974.v2.patch, HBASE-18974.v3.patch, HBASE-18974.v4.patch > > > Based on the mailing list discussion at > https://lists.apache.org/thread.html/81c633cbe1f6f78421cbdad5b9549643c67803a723a9d86a513264c0@%3Cdev.hbase.apache.org%3E > it sounds like we should record some of the thoughts for future contributors > to refer to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration
[
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604875#comment-16604875
]
Hadoop QA commented on HBASE-21150:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} hbase-server: The patch generated 1 new + 78 unchanged
- 0 fixed = 79 total (was 78) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}195m
59s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}237m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21150 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12938496/21150.v7.txt |
| Optional Tests | asflicense javac javadoc unit
[jira] [Work started] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21153 started by Sean Busbey.
---
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604873#comment-16604873
]
Sean Busbey commented on HBASE-21153:
-
Reproduced the error locally and it's even more confusing if you're on a Hadoop
version missing YARN-7190. In that case my 2.1.1-SNAPSHOT build reported the
hbase version as 1.2.6. :(
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Affects Version/s: 3.0.0
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Description:
*edit*:
Now that our assembly directly relies on the shaded clients, failing to build
the actual client jars (e.g. because {{-P release}} is required to fill in
their contents) causes confusing errors for downstream folks about classes not
being found when they run simple commands like {{hbase version}}.
We should always fill in the shaded artifacts to make our build easier to
understand.
*original report:*: When I run the {{hbase version}} command it comes back with:
{code}$ ./bin/hbase version
Error: Could not find or load main class
org.apache.hadoop.hbase.util.GetJavaProperty
Error: Could not find or load main class
org.apache.hadoop.hbase.util.VersionInfo{code}
The two classes are in hbase-commons.
The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars wherever
possible -- may have overstretched expecting version to work with shaded client
([~busbey] ?). If so, fix is < one-liner.
was:
When I run the {{hbase version}} command it comes back with:
{code}$ ./bin/hbase version
Error: Could not find or load main class
org.apache.hadoop.hbase.util.GetJavaProperty
Error: Could not find or load main class
org.apache.hadoop.hbase.util.VersionInfo{code}
The two classes are in hbase-commons.
The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars wherever
possible -- may have overstretched expecting version to work with shaded client
([~busbey] ?). If so, fix is < one-liner.
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Fix Version/s: 2.2.0
3.0.0
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21153) Shaded client jars should always build in relevant phase to avoid confusion
[
https://issues.apache.org/jira/browse/HBASE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21153:
Affects Version/s: 2.2.0
> Shaded client jars should always build in relevant phase to avoid confusion
> ---
>
> Key: HBASE-21153
> URL: https://issues.apache.org/jira/browse/HBASE-21153
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 2.2.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: ls.txt
>
>
> *edit*:
> Now that our assembly directly relies on the shaded clients, failing to build
> the actual client jars (e.g. because {{-P release}} is required to fill in
> their contents) causes confusing errors for downstream folks about classes
> not being found when they run simple commands like {{hbase version}}.
> We should always fill in the shaded artifacts to make our build easier to
> understand.
> *original report:*: When I run the {{hbase version}} command it comes back
> with:
> {code}$ ./bin/hbase version
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.VersionInfo{code}
> The two classes are in hbase-commons.
> The nice shaded refactoring of our bin/hbase -- i.e. using shaded jars
> wherever possible -- may have overstretched expecting version to work with
> shaded client ([~busbey] ?). If so, fix is < one-liner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21154) Remove hbase:namespace table; fold it into hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604868#comment-16604868
]
stack commented on HBASE-21154:
---
[~mdrob] Not sure. We tried to hide all access behind the
TableNamespaceManager. If there were an auto-migration, I don't see why it
couldn't go into branch-2. Haven't looked sir.
> Remove hbase:namespace table; fold it into hbase:meta
> -
>
> Key: HBASE-21154
> URL: https://issues.apache.org/jira/browse/HBASE-21154
> Project: HBase
> Issue Type: Improvement
>Reporter: stack
>Priority: Major
>
> Namespace table is a small system table. Usually it has two rows. It must be
> assigned before user tables but after hbase:meta goes out. Its presence
> complicates our startup and is a constant source of grief when for whatever
> reason, it is not up and available. In fact, master startup is predicated on
> hbase:namespace being assigned and will not make progress unless it is up.
> Lets just add a new 'ns' column family to hbase:meta for namespace.
> Here is a default ns table content:
> {code}
> hbase(main):023:0* scan 'hbase:namespace'
> ROW
>COLUMN+CELL
> default
>column=info:d, timestamp=1526694059106,
> value=\x0A\x07default
> hbase
>column=info:d, timestamp=1526694059461,
> value=\x0A\x05hbase
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
